In 1963 I was living on the top floor of Ellworth Hall at KU. KU is located on Mount Oread which is the highest point in Eastern Kansas, and Ellsworth in the highest place there. There was a magnificent view of farms and woodlands to the West. One Spring afternoon there was a storm front that approached from the West. That afternoon, you could see 7 tornados at the same time from the dorm windows. We were all Kansans and most had seen tornados before, but even we remarked that surely there would be a big write up in the local papers. There was nothing. A note in a small town local paper from mentioned that some farmer had lost a small out building.
Today the entire horizon as seen from Ellsworth is ranches homes, small businesses, schools and churches. If that event happed today, if would provide some Weather Channel specials that would last the rest of the decade. In 2011 one large tornado and several smaller ones tore thru Joplin Missouri. Killing several and doing millions of dollars worth of damage. I have bid and built a lot of work in the area. My best friend in 1962 was born there and we visited several times. I know where the tornados struck. In t4he 1960's a few cows would have died and no one would have noticed.
Sorry I came to this late. I'd prefer if you focused on specific aspects of "scientific integrity" and defined the concept. It might be an interesting grad student project to look at different agency's definitions. Ideas like "transparency of public information" were around long before the Data Quality Act or whatever is linked to current concepts of "scientific integrity."
I think we need look no further than some of the government scientist "let's not use our work emails" discussions during Covid to know there is a major transparency issue in government science, above and beyond other behaviors that might be called "against scientific integrity." But as you may recall I always thought the way that "scientific integrity" was promoted was basically political.. err.. signaling without much substance.
NOAA also helps fuel (pun intended) the climate change - fire narrative. They ignore a major driver of fire size and severity - fuel loads (which have increased, in part, due to fire exclusion) and only look at several decades of fire data to support the narrative, which is too short to evaluate fire regimes with > 150 year fire return intervals. From their website: "Climate change, including increased heat, extended drought, and a thirsty atmosphere, has been a key driver in increasing the risk and extent of wildfires in the western United States during the last two decades. Wildfires require the alignment of a number of factors, including temperature, humidity, and the lack of moisture in fuels, such as trees, shrubs, grasses, and forest debris. All these factors have strong direct or indirect ties to climate variability and climate change."
As a person from a wildfire-adjacent community, NOAA and NASA are just trying to increase their market share of science budgets. "Things we study are more important to solving problems than other factors" and the media are unskeptical.
Roger... outstanding, as always, and the deep concerns re data integrity, reproducibility, and transparency are applicable across multiple Fed agencies, including outputs by FEMA and the US Army Corps of Engineers on NFIP and flood control infrastructure. I discussed your paper w/ a colleague and he asked whether you intend to make a formal "request for correction" (RFC) to NOAA under its guidelines implementing the Information Quality Act of 2001 (Sec. 515 of PL 106-554) and relavent OMB guidance from 2002 and 2019 (updated). Your footnotes on p. 3 (nos. 9 and 11) link to the NOAA RFC criteria (pursuant to OMB-M-19-15). Many commenters here understandably lament the lack of agency accountability. A formal RFC might help change things. They're required by law. You've inspired me to examine doing same w/ FEMA and USACE.
Don't forget I was a fellow in a NOAA Cooperative Institute (CIRES) for 15 years. I have former students in important positions in NOAA. I have discussed this issue many times with many in NOAA. Many people are aware of the scientific issues here, even independent of my critiques -- they are pretty obvious. Why aren't they fixed? I don't have a good answer. Probably the same reason why the media writ large refuses to acknowledge any criticism of the NOAA dataset.
My sense is that the BDD dataset easily travels into the media and policy settings, and agencies like that a lot. The president cited the dataset!!! Yay!!
I think the issues here are more about turf and federal budgets than the hot politics of climate (though the hot politics of climate are what makes the BDD dataset travel so well).
NOAA has painted themselves into a corner -- what happens if they correct/revise the dataset now? Nothing good for them, other than scientific integrity. So best to double down. My two cents.
That makes sense, including the doubling down. Can you think of past Zombie metrics - across government - where methodology is demonstrably not scientifically valid but lives on forever?
I appreciate the opportunity to read the paper. The changing information and lack of transparency is truly troubling. I support your efforts to get them to improve the documentation of sources, methods and adjustments.
I live on the Gulf of Mexico coast and follow hurricane season intently. My experience and observation is over the last 30 years, in the counties here in Florida, there has been a significant increase in population and residential and commercial building. Clearly the same storm in 2023 vs 1993 would have a significant increase in monetary damage….. but wouldn’t be an indicator of climate change impacts.
I grew up in Oklahoma in tornado alley. When I visit now I see population growth and increased building. In a similar fashion I don’t believe increased $ damage would be a good indicator of climate impacts on the frequency or severity of tornadoes.
Keep up the good work. As we used to say when I worked as an engineer, show me your data. There is too much journalistic and political malpractice occurring to compound it with less than our best efforts with science.
hotair.com already has a link to your substack article. That could be fodder to label it as right wing spin. If realclearpolitics puts up a link to it that will make it very hard for Nature to drop it. A lot more people will read it from a RCP link than in Nature.
If I criticize Boeing for its numerous 737 Max issues, no one will expect me to write “the point here is not to call into question the reality of gravity.” I hope at some point we won’t have to write “The point here is not to call into question the reality or importance of human-caused climate change – it is real, and it is important.”
Good Luck Roger. You have exactly zero chance of influencing NOAA or having them adhere to their own scientific integrity policy. I tried multiple times and the only way to get them to change is by strong legal action and I doubt even that would work.
The group at NOAA NCEI, formerly NCDC, has a bunker mentality. Plus it brings lots of headlines and even though Tom Karl is gone, they can’t quit his approaches. I tried to force them to document everything by formulating a Climate Data Record (CDR) Maturity index that does all and more of what you suggest https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2012EO440006
This CDR maturity index has been widely adopted, including by the European Union, Committee on Earth Observation Satellites, and others. The NCEI surface products folks rejected it and will never ever adopt it; it didn’t come from them.
Let’s take a look how to properly document the world’s most used and cited climate data set – the International Cloud Climatology Project (ISCCP). This is where you get HurSat data also. Bill Rossow and I worked to ensure this important data set lived beyond our careers - https://www.ncei.noaa.gov/products/climate-data-records/cloud-properties-isccp
Everything needed is documented; algorithm, data flow, maturity index, manuals, and source code.
Virtually no details. Yes, some of the peer reviewed publications give some steps, but there is no chance you can re-produce their result. Hell, they can’t even reliably reproduce their result. I know since I was in charge of running the code for a bit. They took it away when I refused to let them make undocumented changes to the operational code.
Only a lawsuit, injunction, and then real consequences will have any chance of forcing NOAA NCEI to do the right thing.
Well done. The primary authors, and their management, must be aware of a significant fraction of problems you identify with their analysis. You must have many contacts at NOAA and I would assume you have provided some of the arguments to those contacts in the past (especially the very sensible GDP normalization) . Why do they persist with this? I hope this gets published close to its present form, and that productive discussions follow. Ron
The list is useful as a case-picker for "important" weather events, for little undergrad research projects on the meteorology of each one. Not so much for its growing length or trends, as you say. I wonder who touts up the latter? NOAA itself, or some independent layer of activists?
The same people who tell everyone to trust the science are the same people who ignore scientific integrity. Science without integrity is just propaganda aimed at supporting an agenda. I can understand why NGOs lack scientific integrity, but it's infuriating when taxpayer funded agencies pull this stunt.
You mentioned that your paper has not yet been peer-reviewed. Would it be possible for you to publish the peer reviewers' comments, without violating the integrity of the process? I would also be curious about what ire you might raise among the alarmists' crowd (I have visions of another Alimonti debacle).
Thanks for sharing the article. As an economist, I find the inflation-adjusted methodology amusingly egregious. While useful to express everything in comparable current dollars, when you combine it with the arbitrary nominal 1 billion $ threshold it seems expressly designed to constantly inflate the cumulative headline number, including by pulling in past disasters that were not previously counted.
Indeed, as you note, a more appropriate methodology would be to measure it all as a share of GDP. This would avoid the very counterintuitive phenomenon of old disasters inflating their way into the database; it would also give a better measure of the real economic damage. A richer economy has bigger “targets” – the same disaster will likely cause more dollars in damage, as you point out, but it also has a better capacity to absorb the cost. It’s the same rationale by which we look at public debt as % of GDP and not in nominal terms: the capacity to repay/absorb the cost matters.
Again, congratulations on an excellent and very important contribution; the extent to which we are expected to take on faith scientific arguments in the climate change debate is one of the most deleterious developments of the last many years.
I suppose it could be argued that any GDP-based metric is more or less egregiously flawed (whether inflation adjusted or as a share of GDP).
If whichever country affected by disaster simply lacks the necessary skilled labour to rebuild destroyed infrastructure then 'GDP's capacity to absorb the cost' is completely irrelevant.
What is relevant is whether your country has been training and providing a suitably remunerative economy for, for example, electrical linemen over the years. How many financial wizards and private sector debt creators you have 'produced' over the years to goose your GDP numbers will be of strictly zero value when it comes to the task at hand.
From a slightly older KS friend:
In 1963 I was living on the top floor of Ellworth Hall at KU. KU is located on Mount Oread which is the highest point in Eastern Kansas, and Ellsworth in the highest place there. There was a magnificent view of farms and woodlands to the West. One Spring afternoon there was a storm front that approached from the West. That afternoon, you could see 7 tornados at the same time from the dorm windows. We were all Kansans and most had seen tornados before, but even we remarked that surely there would be a big write up in the local papers. There was nothing. A note in a small town local paper from mentioned that some farmer had lost a small out building.
Today the entire horizon as seen from Ellsworth is ranches homes, small businesses, schools and churches. If that event happed today, if would provide some Weather Channel specials that would last the rest of the decade. In 2011 one large tornado and several smaller ones tore thru Joplin Missouri. Killing several and doing millions of dollars worth of damage. I have bid and built a lot of work in the area. My best friend in 1962 was born there and we visited several times. I know where the tornados struck. In t4he 1960's a few cows would have died and no one would have noticed.
Sorry I came to this late. I'd prefer if you focused on specific aspects of "scientific integrity" and defined the concept. It might be an interesting grad student project to look at different agency's definitions. Ideas like "transparency of public information" were around long before the Data Quality Act or whatever is linked to current concepts of "scientific integrity."
I think we need look no further than some of the government scientist "let's not use our work emails" discussions during Covid to know there is a major transparency issue in government science, above and beyond other behaviors that might be called "against scientific integrity." But as you may recall I always thought the way that "scientific integrity" was promoted was basically political.. err.. signaling without much substance.
Roger perfect timing given yet another breathless take on weather disasters and insurance in WSJ today https://www.wsj.com/business/insurance-home-auto-rate-increases-climate-change-03b806f3. We refer to your work but what else can we do??
The idea that somehow insurance rates should increase based on the modeled future rather than the observed past needs to be addressed and possibly litigated IMHO. Perhaps litigate insurance regulators for not performing oversight? https://www.hcn.org/issues/56.1/wildfire-homeowners-insurance-is-going-up-in-smoke
NOAA also helps fuel (pun intended) the climate change - fire narrative. They ignore a major driver of fire size and severity - fuel loads (which have increased, in part, due to fire exclusion) and only look at several decades of fire data to support the narrative, which is too short to evaluate fire regimes with > 150 year fire return intervals. From their website: "Climate change, including increased heat, extended drought, and a thirsty atmosphere, has been a key driver in increasing the risk and extent of wildfires in the western United States during the last two decades. Wildfires require the alignment of a number of factors, including temperature, humidity, and the lack of moisture in fuels, such as trees, shrubs, grasses, and forest debris. All these factors have strong direct or indirect ties to climate variability and climate change."
As a person from a wildfire-adjacent community, NOAA and NASA are just trying to increase their market share of science budgets. "Things we study are more important to solving problems than other factors" and the media are unskeptical.
Roger... outstanding, as always, and the deep concerns re data integrity, reproducibility, and transparency are applicable across multiple Fed agencies, including outputs by FEMA and the US Army Corps of Engineers on NFIP and flood control infrastructure. I discussed your paper w/ a colleague and he asked whether you intend to make a formal "request for correction" (RFC) to NOAA under its guidelines implementing the Information Quality Act of 2001 (Sec. 515 of PL 106-554) and relavent OMB guidance from 2002 and 2019 (updated). Your footnotes on p. 3 (nos. 9 and 11) link to the NOAA RFC criteria (pursuant to OMB-M-19-15). Many commenters here understandably lament the lack of agency accountability. A formal RFC might help change things. They're required by law. You've inspired me to examine doing same w/ FEMA and USACE.
Have you reached out to NOAA directly about this in the past? Thinking of Mike Smith's efforts to improve NWS tornado warning system, which aren't just taking place through his blog but also direct outreach. If you do (or have done) that, it can help with FOIAs etc. https://www.mikesmithenterprisesblog.com/2023/10/nws-tornado-warning-problems-continue.html
Yes
Don't forget I was a fellow in a NOAA Cooperative Institute (CIRES) for 15 years. I have former students in important positions in NOAA. I have discussed this issue many times with many in NOAA. Many people are aware of the scientific issues here, even independent of my critiques -- they are pretty obvious. Why aren't they fixed? I don't have a good answer. Probably the same reason why the media writ large refuses to acknowledge any criticism of the NOAA dataset.
Is part of the failure a lack of cross talk between FEMA and NOAA?
No
My sense is that the BDD dataset easily travels into the media and policy settings, and agencies like that a lot. The president cited the dataset!!! Yay!!
I think the issues here are more about turf and federal budgets than the hot politics of climate (though the hot politics of climate are what makes the BDD dataset travel so well).
NOAA has painted themselves into a corner -- what happens if they correct/revise the dataset now? Nothing good for them, other than scientific integrity. So best to double down. My two cents.
That makes sense, including the doubling down. Can you think of past Zombie metrics - across government - where methodology is demonstrably not scientifically valid but lives on forever?
Plenty
I use this paper in my grad seminars to introduce the notion of zombie science:
https://journals.sagepub.com/doi/full/10.1177/0306312714535679
Perhaps most significant is the current idea in several agencies that a Covid lab leak has been disproven with certainty
RCP8.5 is of course another
I appreciate the opportunity to read the paper. The changing information and lack of transparency is truly troubling. I support your efforts to get them to improve the documentation of sources, methods and adjustments.
I live on the Gulf of Mexico coast and follow hurricane season intently. My experience and observation is over the last 30 years, in the counties here in Florida, there has been a significant increase in population and residential and commercial building. Clearly the same storm in 2023 vs 1993 would have a significant increase in monetary damage….. but wouldn’t be an indicator of climate change impacts.
I grew up in Oklahoma in tornado alley. When I visit now I see population growth and increased building. In a similar fashion I don’t believe increased $ damage would be a good indicator of climate impacts on the frequency or severity of tornadoes.
Keep up the good work. As we used to say when I worked as an engineer, show me your data. There is too much journalistic and political malpractice occurring to compound it with less than our best efforts with science.
hotair.com already has a link to your substack article. That could be fodder to label it as right wing spin. If realclearpolitics puts up a link to it that will make it very hard for Nature to drop it. A lot more people will read it from a RCP link than in Nature.
You got a link on RealClearScience. That's pretty good cancellation insurance.
Wow!
Already my paper has been downloaded >10k times
Just for the record, I'll keep track of all of the climate beat reporters who have been in touch to ask me about it. Here is the list:
“Here is the list:”. What’s your point? I love the sarcasm.
I'm shocked that a Nature journal invited this publication. There must be some other influential people in this field that are unhappy with NOAA.
To be clear, they invited me to submit a paper, they did not suggest the topic. I appreciate the invitation.
If I criticize Boeing for its numerous 737 Max issues, no one will expect me to write “the point here is not to call into question the reality of gravity.” I hope at some point we won’t have to write “The point here is not to call into question the reality or importance of human-caused climate change – it is real, and it is important.”
The same sort of mantra caveat was demanded of Galileo, Darwin, early Geology, Medicine, literature, etc
Good Luck Roger. You have exactly zero chance of influencing NOAA or having them adhere to their own scientific integrity policy. I tried multiple times and the only way to get them to change is by strong legal action and I doubt even that would work.
The group at NOAA NCEI, formerly NCDC, has a bunker mentality. Plus it brings lots of headlines and even though Tom Karl is gone, they can’t quit his approaches. I tried to force them to document everything by formulating a Climate Data Record (CDR) Maturity index that does all and more of what you suggest https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2012EO440006
https://journals.ametsoc.org/view/journals/bams/97/9/bams-d-15-00015.1.xml
This CDR maturity index has been widely adopted, including by the European Union, Committee on Earth Observation Satellites, and others. The NCEI surface products folks rejected it and will never ever adopt it; it didn’t come from them.
Let’s take a look how to properly document the world’s most used and cited climate data set – the International Cloud Climatology Project (ISCCP). This is where you get HurSat data also. Bill Rossow and I worked to ensure this important data set lived beyond our careers - https://www.ncei.noaa.gov/products/climate-data-records/cloud-properties-isccp
Everything needed is documented; algorithm, data flow, maturity index, manuals, and source code.
Compare and contrast this with the source of all, yes all, surface temperature analysis data - https://www.ncei.noaa.gov/products/land-based-station/noaa-global-temp
Virtually no details. Yes, some of the peer reviewed publications give some steps, but there is no chance you can re-produce their result. Hell, they can’t even reliably reproduce their result. I know since I was in charge of running the code for a bit. They took it away when I refused to let them make undocumented changes to the operational code.
Only a lawsuit, injunction, and then real consequences will have any chance of forcing NOAA NCEI to do the right thing.
Well done. The primary authors, and their management, must be aware of a significant fraction of problems you identify with their analysis. You must have many contacts at NOAA and I would assume you have provided some of the arguments to those contacts in the past (especially the very sensible GDP normalization) . Why do they persist with this? I hope this gets published close to its present form, and that productive discussions follow. Ron
The list is useful as a case-picker for "important" weather events, for little undergrad research projects on the meteorology of each one. Not so much for its growing length or trends, as you say. I wonder who touts up the latter? NOAA itself, or some independent layer of activists?
The same people who tell everyone to trust the science are the same people who ignore scientific integrity. Science without integrity is just propaganda aimed at supporting an agenda. I can understand why NGOs lack scientific integrity, but it's infuriating when taxpayer funded agencies pull this stunt.
Agreed
I apply far different standards of scientific integrity to politicians and NGOs than to public science agencies.
A good post, as usual. Thank you.
You mentioned that your paper has not yet been peer-reviewed. Would it be possible for you to publish the peer reviewers' comments, without violating the integrity of the process? I would also be curious about what ire you might raise among the alarmists' crowd (I have visions of another Alimonti debacle).
I look forward to reading your paper.
Yes, I will be happy to make this whole process as transparent as possible without violating the journal’s policies
I am not sure, but it may be that npjNH will publish the reviews themselves (they should!)
thank you.
Thanks for sharing the article. As an economist, I find the inflation-adjusted methodology amusingly egregious. While useful to express everything in comparable current dollars, when you combine it with the arbitrary nominal 1 billion $ threshold it seems expressly designed to constantly inflate the cumulative headline number, including by pulling in past disasters that were not previously counted.
Indeed, as you note, a more appropriate methodology would be to measure it all as a share of GDP. This would avoid the very counterintuitive phenomenon of old disasters inflating their way into the database; it would also give a better measure of the real economic damage. A richer economy has bigger “targets” – the same disaster will likely cause more dollars in damage, as you point out, but it also has a better capacity to absorb the cost. It’s the same rationale by which we look at public debt as % of GDP and not in nominal terms: the capacity to repay/absorb the cost matters.
Again, congratulations on an excellent and very important contribution; the extent to which we are expected to take on faith scientific arguments in the climate change debate is one of the most deleterious developments of the last many years.
I suppose it could be argued that any GDP-based metric is more or less egregiously flawed (whether inflation adjusted or as a share of GDP).
If whichever country affected by disaster simply lacks the necessary skilled labour to rebuild destroyed infrastructure then 'GDP's capacity to absorb the cost' is completely irrelevant.
What is relevant is whether your country has been training and providing a suitably remunerative economy for, for example, electrical linemen over the years. How many financial wizards and private sector debt creators you have 'produced' over the years to goose your GDP numbers will be of strictly zero value when it comes to the task at hand.