Belatedly reading this valuable piece but wondering how you separate what you disparage as tactical science from your own work pulling back the curtain on billion-dollar disasters or RCP 8.5. Would you be as apt to publish on those questions if they weren't policy relevant? Another great fresh example is the new Bob Kopp et al paper in Nature Climate Change critiquing tipping points: https://revkin.substack.com/p/a-watchwords-warning-about-tipping
A lot of science is tactical -- that alone does not make it bad science. As I explain in the post:
"The three climate papers above are bad science not simply because they are tactical science, but because they are bad science — the demonstration of which requires employing good science."
There is plenty of tactical science that is also good science.
Roger, I've seen some tweets about the possible influence of HAARP (in the IONOSphere) on weather.
Common understanding is that it too high above the Troposphere, but some people seem to doubt this and started to make links between weather patterns and strength and direction of storms and hurricanes. Could this be a possible subject for an article in THB.
Great piece, sir. Thank you. The discussion is, as usual, fantastic and educational. But one thing missing in all of it is "accountability." The general theme is that "peer-reviewed" is not trusted and fraught with abuse. This is throwing the baby out with the bath water.
Peer-review has done much to improve a vast majority of scientific papers, when done on the basis of good faith and objective science. Peer-review is, in fact, a method of holding an author accountable for his or her publication. What is missing is a method of holding those reviewers equally accountable for their efforts.
Do we remove the "double blind" approach, and require reviewers to publish their names and CV as part of the comment-response process? A reviewer who knowingly passes on an obviously flawed paper stands to tarnish his reputation. Likewise, a reviewer who openly "calls out" a paper for bad science or "decision-based evidence-making" (I love that phrase) will gain respect.
There are undoubtedly better ways of improving accountability in the review and publication process. As one reviewer pointed out, like democracy it's not perfect but a helluva lot better than what's left.
Clickbait title, unnecessary introduction of new terminology (your definition is covered already by your own term of Issue/Stealth Advocate), cherry-picking examples to make point (both papers, and selective quotes).
Yes, peer review is imperfect, has been since the inception of scientific journals 350 years ago (with now some 30k journals out there; 1.5-2M papers published each year; something between 50-100M scholarly pubs in total; non-standardized criteria being applied, and reviewers overburdened). There have always existed bad and bad-faith science (as amply exemplified in Oreskes' and Conway's Merchants of Doubt case studies), and cherry-picking evidence for political ends, hardly new thing.
Peer review is only a first QA step in the validation of scientific knowledge (and the only thing we have to have a reasonable chance that the 2M papers published are not rubbish claims). But it is well-known (increasingly in some mainstream, reputable media outlets) that producing and validating reasonably solid knowledge requires the scientific method + peer review + reproducibility studies + debate post-publication (by scientists, public, and other stakeholders). A lot of this is increasingly implemented at journals and platforms, aided by open access/science efforts.
Claims and arguments in this piece are tendentious.
In my view, stealth advocacy is a far broader concept than is tactical research
For instance, persistent use of RCP8.5 may be stealth advocacy (recognizing that some is not).
Conducting research with the aim of fortifying RCP8.5 for use in advocacy is however a specific type of stealth advocacy — tactical research.
Oreskes and Conway associate bad faith science with that outside peer review and good faith as that inside peer review. I’d hope we can agree that claim is simply incorrect.
"Oreskes and Conway associate bad faith science with that outside peer review and good faith as that inside peer review. I’d hope we can agree that claim is simply incorrect."
I don't see how that claim is made/supported in/by the quotes you picked.
Also
1) Bad science could be seen as distinct from bad-faith science, or at least a higher-level term encompassing bad-faith one (e.g. ignorance/incompetence-driven bad science, of which there's plenty of evidence in the literature... vs intentional distortions, which would indeed be bad-faith one).
2) Publishing in peer reviewed journals is indeed distinct from publishing an opinion piece on a blog [e.g. regarding accountability (which can even include retractions/self-corrections to the record, based on authors' own, editors', scientists, other stakeholders' criticisms) vs non-accountable)].
3) Peer review is a necessary, but not sufficient, aspect of identifying bad (and bad faith) science (see point 2) + my previous response (a more sanguine interpretation of Oreskes/Conway could revolve around this).
Great cautionary warning that peer-review is not gospel, though is often taken like it is. With the revolution of AI, hopefully researchers, journalists, and their readers take the personal responsibility that not everything they read, from any source, is automatic fact.
As I've said before, "peer review is the last refuge of scoundrels" and of the Established.
The NRC has an (unintended) counterpoint to this. Its NUREG dealing with qualification of existing data for use lists four methods for this. They make the point that confirmatory testing is the strongest, and by implication peer review is the weakest. The bottom line to me is that the onus should be on the users of data, studies, etc, to have processes aimed at assuring fitness for use rather than expecting peer review to do this.
Science is a process. Peer review is part of that process. Like many human things, it goes wrong sometimes. But like democracy, it is not perfect but noone has found a better method.
Let's not throw the baby out with the bathwater and suggest that flawed Peer Review means that we have to be sceptical about climate change and the drastic changes we need to make it happen.
If scientists in the 1980s and 1990s had been better at pushing politicians to act on climate change, we might have been in better shape now.
Future generations might blame us/them for not pushing the politics "by any means necessary". Lies, if needed. Manipulation, if needed. The truth of science is complex and has to be distilled to something easier for politics.
I know you dont agree that the end justify the means. But sometimes it does. It a person stands on a bridge ready to jump off it, you tell him everything will be fine. If a madman points a gun at your head, you say whatever is needed to not be shot. The end, justifies the means, at some point on the scale.
Climate wipeout is such a point.
We dont know what will happen yet, of course. But those unborn, who at the moment has no rights and no legal persona, and perhaps shouldn't have any, will know, and might applaud those scientists who tried to make us act on climate change before it was too late.
far from it, I am suggesting that the data is so good that we should not be in doubt of the ability of climate change to significantly wipe out ecosystems and our ability to function as a society. The data is all here, not made by God at all:
There is no data in that report suggesting what you say. There are results of simulations based on very questionable assumptions that suggest something might happen. The range of what might happen is very wide, and the realism in that range goes from virtually zero to something we might believe.
Peer review is literally censorship. Now we have the internet we don't need it - and the wholesale abuses mean it's basically dead already, certainly in any field with the slightest political tension.
If someone has done some work they can publish it. If I can't evaluate it myself I can pick people with expertise who I trust to evaluate it for me.
As a scientist, I was also myself victim of this process. I bring here one of my experiences:
The general topic of a paper of mine was "ecological impact of small hydropower". The article was submitted to a well-known ecology-focused journal, and sent to reviewers (initially two, as I found out later). After a couple of months, the reviews arrived to the editor, and apparently the editor chose to send it for another review, to a third reviewer. After that, I received the reviews (three in total). I was expecting contrasting reviews motivating the sending of the article to three reviewers, but all of them actually recommended publication of the article. The editor rejected the article, with reasons that, at most, could have justified a desk-rejection. It was clear the editor was looking for a review giving a reason to reject the article, but eventually this was done, no matter what.
Trying to understand the reason for this behaviour, I checked publications of the editor, and it resulted they were author or co-author of articles, also well-known within the sector, to which the results I presented in the paper were not in line with.
Peer review is a joke, it is a completely auto-referential system, with no proper "quality" check at any stage.
There were "Publish or Perish" or "Cash or Crash", now we have "Advocate or Abdicate".
By the way, the three articles mentioned (Howarth, Sarofim, Schwalm) are mere expert reports with no scientific investigation that establish new knowledge. What's more, they are lousy experts because of their prejudices. This should be picked by peer review, as long as the peers are not as prejudiced as the authors.
The reader of an expert report expects all aspects of a problem to be presented, not just arguments pointing in one particular direction.
There is a lot of confusion between scientific publications, scientific reviews, and expert reports.
Peer review is a good system, but it is also crap. The intention is all good, but the implementation is crap. First of all, what is with the secrecy? Who came up with the idea that a knowledgeable person should be able to criticize some work without disclosing who they are? Science is about openness. Open methods, open data, open arguments, open conclusions. But peer review should be...closed? That is where the crap starts.
Today there is so much being produced which is worthless. It is not necessarily wrong, but it has no worth. Millions of papers published that carry nothing - no new knowledge, no new data, analyzing what is obvious.
Science is a slow train, but I think by and by the journals will lose their position and open review will happen bacuse it can.
Journals have created a self reinforcing loop to achieve what are considered noble ends. There is only one result from a self reinforcing loop. My hope is the rest of us don't get taken down with it. As a past editorial board chairman for a non technical professional publication, I'll say again, we have to find a different path for peer review. Double blind review is if no use if the primary quest is politics, not science. My idea is that qualified reviewers be chosen at random much like a jury. That the reviews be published and subject to rebuttal. Yes, a flawed idea. But maybe a start.
"First, we should all recognize that peer-review provides a minimal standard of review"
I disagree. Reviewers are free to do as much or as little as they please. They can be vindictive towards the authors or they can be shameless promoters of the authors. There is no accountability for reviewers.
You have found a home on substack because your writing and your scientific integrity is such that people like me are willing to pay you to keep going, because we have concluded that most 'Peer Reviewed' papers have become just another way of suppressing unwanted inputs, regardless of the actual scientific merit. Government funded 'scientists' prepare 'scientific' papers that are then 'peer reviewed' by other government funded 'scientists' - and they all then use the fact that they have published 'peer reviewed' papers to get more government funding.
You and like-minded scientists need to start your own peer-reviewed online scientific 'magazine' where 'peer review' can return to it's original intent - separating good science from bad science. Think of it this way; when the history of this shameful period is written, wouldn't you like to be remembered as the guy (or group of guys) who created the institution that eventually becomes the new gold standard for real climate science - one that even governments can't afford to ignore?
Belatedly reading this valuable piece but wondering how you separate what you disparage as tactical science from your own work pulling back the curtain on billion-dollar disasters or RCP 8.5. Would you be as apt to publish on those questions if they weren't policy relevant? Another great fresh example is the new Bob Kopp et al paper in Nature Climate Change critiquing tipping points: https://revkin.substack.com/p/a-watchwords-warning-about-tipping
A lot of science is tactical -- that alone does not make it bad science. As I explain in the post:
"The three climate papers above are bad science not simply because they are tactical science, but because they are bad science — the demonstration of which requires employing good science."
There is plenty of tactical science that is also good science.
Roger, I've seen some tweets about the possible influence of HAARP (in the IONOSphere) on weather.
Common understanding is that it too high above the Troposphere, but some people seem to doubt this and started to make links between weather patterns and strength and direction of storms and hurricanes. Could this be a possible subject for an article in THB.
Roger, taking this to a sporting metaphor, which your background should help you appreciate.
WG1 is real sport, 100m dash, no doubts as to the outcome vs WG2 is the judged sports like ice dance and trampoline.
It why corruption and fraud clusters where it can best operate
Great piece, sir. Thank you. The discussion is, as usual, fantastic and educational. But one thing missing in all of it is "accountability." The general theme is that "peer-reviewed" is not trusted and fraught with abuse. This is throwing the baby out with the bath water.
Peer-review has done much to improve a vast majority of scientific papers, when done on the basis of good faith and objective science. Peer-review is, in fact, a method of holding an author accountable for his or her publication. What is missing is a method of holding those reviewers equally accountable for their efforts.
Do we remove the "double blind" approach, and require reviewers to publish their names and CV as part of the comment-response process? A reviewer who knowingly passes on an obviously flawed paper stands to tarnish his reputation. Likewise, a reviewer who openly "calls out" a paper for bad science or "decision-based evidence-making" (I love that phrase) will gain respect.
There are undoubtedly better ways of improving accountability in the review and publication process. As one reviewer pointed out, like democracy it's not perfect but a helluva lot better than what's left.
My two cents, adjusted for inflation.
Clickbait title, unnecessary introduction of new terminology (your definition is covered already by your own term of Issue/Stealth Advocate), cherry-picking examples to make point (both papers, and selective quotes).
Yes, peer review is imperfect, has been since the inception of scientific journals 350 years ago (with now some 30k journals out there; 1.5-2M papers published each year; something between 50-100M scholarly pubs in total; non-standardized criteria being applied, and reviewers overburdened). There have always existed bad and bad-faith science (as amply exemplified in Oreskes' and Conway's Merchants of Doubt case studies), and cherry-picking evidence for political ends, hardly new thing.
Peer review is only a first QA step in the validation of scientific knowledge (and the only thing we have to have a reasonable chance that the 2M papers published are not rubbish claims). But it is well-known (increasingly in some mainstream, reputable media outlets) that producing and validating reasonably solid knowledge requires the scientific method + peer review + reproducibility studies + debate post-publication (by scientists, public, and other stakeholders). A lot of this is increasingly implemented at journals and platforms, aided by open access/science efforts.
Claims and arguments in this piece are tendentious.
Thanks DC
I always appreciate hearing from readers
In my view, stealth advocacy is a far broader concept than is tactical research
For instance, persistent use of RCP8.5 may be stealth advocacy (recognizing that some is not).
Conducting research with the aim of fortifying RCP8.5 for use in advocacy is however a specific type of stealth advocacy — tactical research.
Oreskes and Conway associate bad faith science with that outside peer review and good faith as that inside peer review. I’d hope we can agree that claim is simply incorrect.
Thanks
Thanks Roger.
"Oreskes and Conway associate bad faith science with that outside peer review and good faith as that inside peer review. I’d hope we can agree that claim is simply incorrect."
I don't see how that claim is made/supported in/by the quotes you picked.
Also
1) Bad science could be seen as distinct from bad-faith science, or at least a higher-level term encompassing bad-faith one (e.g. ignorance/incompetence-driven bad science, of which there's plenty of evidence in the literature... vs intentional distortions, which would indeed be bad-faith one).
2) Publishing in peer reviewed journals is indeed distinct from publishing an opinion piece on a blog [e.g. regarding accountability (which can even include retractions/self-corrections to the record, based on authors' own, editors', scientists, other stakeholders' criticisms) vs non-accountable)].
3) Peer review is a necessary, but not sufficient, aspect of identifying bad (and bad faith) science (see point 2) + my previous response (a more sanguine interpretation of Oreskes/Conway could revolve around this).
Cheers
Let me add that not all targeted research is stealth advocacy, some of it (eg, WWA) is explicit advocacy 👍
Great cautionary warning that peer-review is not gospel, though is often taken like it is. With the revolution of AI, hopefully researchers, journalists, and their readers take the personal responsibility that not everything they read, from any source, is automatic fact.
Thank you for the clear explanation of "tactical science". Most of the time now I wonder what and whom one still can trust.
As I've said before, "peer review is the last refuge of scoundrels" and of the Established.
The NRC has an (unintended) counterpoint to this. Its NUREG dealing with qualification of existing data for use lists four methods for this. They make the point that confirmatory testing is the strongest, and by implication peer review is the weakest. The bottom line to me is that the onus should be on the users of data, studies, etc, to have processes aimed at assuring fitness for use rather than expecting peer review to do this.
Science is a process. Peer review is part of that process. Like many human things, it goes wrong sometimes. But like democracy, it is not perfect but noone has found a better method.
Let's not throw the baby out with the bathwater and suggest that flawed Peer Review means that we have to be sceptical about climate change and the drastic changes we need to make it happen.
If scientists in the 1980s and 1990s had been better at pushing politicians to act on climate change, we might have been in better shape now.
Future generations might blame us/them for not pushing the politics "by any means necessary". Lies, if needed. Manipulation, if needed. The truth of science is complex and has to be distilled to something easier for politics.
I know you dont agree that the end justify the means. But sometimes it does. It a person stands on a bridge ready to jump off it, you tell him everything will be fine. If a madman points a gun at your head, you say whatever is needed to not be shot. The end, justifies the means, at some point on the scale.
Climate wipeout is such a point.
We dont know what will happen yet, of course. But those unborn, who at the moment has no rights and no legal persona, and perhaps shouldn't have any, will know, and might applaud those scientists who tried to make us act on climate change before it was too late.
You are making the God-argument. "I haven't seen it, can't measure it. But I know it is there"
That is a dangerous path, much more dangerous than your imagined "climate wipeout" whatever that is.
far from it, I am suggesting that the data is so good that we should not be in doubt of the ability of climate change to significantly wipe out ecosystems and our ability to function as a society. The data is all here, not made by God at all:
https://www.ipcc.ch/
There is no data in that report suggesting what you say. There are results of simulations based on very questionable assumptions that suggest something might happen. The range of what might happen is very wide, and the realism in that range goes from virtually zero to something we might believe.
Peer review is literally censorship. Now we have the internet we don't need it - and the wholesale abuses mean it's basically dead already, certainly in any field with the slightest political tension.
If someone has done some work they can publish it. If I can't evaluate it myself I can pick people with expertise who I trust to evaluate it for me.
Nothing stops anyone from publishing their own research on the internet without peer review.
Peer review is censorship, it enables non-experts to rely on a panel of experts to trust the original research more.
It is not perfect, but the solution is not to trust everyone equally if they can post it.
As a scientist, I was also myself victim of this process. I bring here one of my experiences:
The general topic of a paper of mine was "ecological impact of small hydropower". The article was submitted to a well-known ecology-focused journal, and sent to reviewers (initially two, as I found out later). After a couple of months, the reviews arrived to the editor, and apparently the editor chose to send it for another review, to a third reviewer. After that, I received the reviews (three in total). I was expecting contrasting reviews motivating the sending of the article to three reviewers, but all of them actually recommended publication of the article. The editor rejected the article, with reasons that, at most, could have justified a desk-rejection. It was clear the editor was looking for a review giving a reason to reject the article, but eventually this was done, no matter what.
Trying to understand the reason for this behaviour, I checked publications of the editor, and it resulted they were author or co-author of articles, also well-known within the sector, to which the results I presented in the paper were not in line with.
Peer review is a joke, it is a completely auto-referential system, with no proper "quality" check at any stage.
sounds like a problem with the editor, not the peer review.
Sure, but the editor has quite an important role in the peer review process...
Tactical science or "advocacy science" ?
There were "Publish or Perish" or "Cash or Crash", now we have "Advocate or Abdicate".
By the way, the three articles mentioned (Howarth, Sarofim, Schwalm) are mere expert reports with no scientific investigation that establish new knowledge. What's more, they are lousy experts because of their prejudices. This should be picked by peer review, as long as the peers are not as prejudiced as the authors.
The reader of an expert report expects all aspects of a problem to be presented, not just arguments pointing in one particular direction.
There is a lot of confusion between scientific publications, scientific reviews, and expert reports.
Peer review is a good system, but it is also crap. The intention is all good, but the implementation is crap. First of all, what is with the secrecy? Who came up with the idea that a knowledgeable person should be able to criticize some work without disclosing who they are? Science is about openness. Open methods, open data, open arguments, open conclusions. But peer review should be...closed? That is where the crap starts.
Today there is so much being produced which is worthless. It is not necessarily wrong, but it has no worth. Millions of papers published that carry nothing - no new knowledge, no new data, analyzing what is obvious.
Science is a slow train, but I think by and by the journals will lose their position and open review will happen bacuse it can.
Journals have created a self reinforcing loop to achieve what are considered noble ends. There is only one result from a self reinforcing loop. My hope is the rest of us don't get taken down with it. As a past editorial board chairman for a non technical professional publication, I'll say again, we have to find a different path for peer review. Double blind review is if no use if the primary quest is politics, not science. My idea is that qualified reviewers be chosen at random much like a jury. That the reviews be published and subject to rebuttal. Yes, a flawed idea. But maybe a start.
"First, we should all recognize that peer-review provides a minimal standard of review"
I disagree. Reviewers are free to do as much or as little as they please. They can be vindictive towards the authors or they can be shameless promoters of the authors. There is no accountability for reviewers.
Roger,
You have found a home on substack because your writing and your scientific integrity is such that people like me are willing to pay you to keep going, because we have concluded that most 'Peer Reviewed' papers have become just another way of suppressing unwanted inputs, regardless of the actual scientific merit. Government funded 'scientists' prepare 'scientific' papers that are then 'peer reviewed' by other government funded 'scientists' - and they all then use the fact that they have published 'peer reviewed' papers to get more government funding.
You and like-minded scientists need to start your own peer-reviewed online scientific 'magazine' where 'peer review' can return to it's original intent - separating good science from bad science. Think of it this way; when the history of this shameful period is written, wouldn't you like to be remembered as the guy (or group of guys) who created the institution that eventually becomes the new gold standard for real climate science - one that even governments can't afford to ignore?
Just my (recurrent) thoughts :)