In 1963 I was living on the top floor of Ellworth Hall at KU. KU is located on Mount Oread which is the highest point in Eastern Kansas, and Ellsworth in the highest place there. There was a magnificent view of farms and woodlands to the West. One Spring afternoon there was a storm front that approached from the West. That afternoon, you could see 7 tornados at the same time from the dorm windows. We were all Kansans and most had seen tornados before, but even we remarked that surely there would be a big write up in the local papers. There was nothing. A note in a small town local paper from mentioned that some farmer had lost a small out building.
Today the entire horizon as seen from Ellsworth is ranches homes, small businesses, schools and churches. If that event happed today, if would provide some Weather Channel specials that would last the rest of the decade. In 2011 one large tornado and several smaller ones tore thru Joplin Missouri. Killing several and doing millions of dollars worth of damage. I have bid and built a lot of work in the area. My best friend in 1962 was born there and we visited several times. I know where the tornados struck. In t4he 1960's a few cows would have died and no one would have noticed.
Sorry I came to this late. I'd prefer if you focused on specific aspects of "scientific integrity" and defined the concept. It might be an interesting grad student project to look at different agency's definitions. Ideas like "transparency of public information" were around long before the Data Quality Act or whatever is linked to current concepts of "scientific integrity."
I think we need look no further than some of the government scientist "let's not use our work emails" discussions during Covid to know there is a major transparency issue in government science, above and beyond other behaviors that might be called "against scientific integrity." But as you may recall I always thought the way that "scientific integrity" was promoted was basically political.. err.. signaling without much substance.
Roger... outstanding, as always, and the deep concerns re data integrity, reproducibility, and transparency are applicable across multiple Fed agencies, including outputs by FEMA and the US Army Corps of Engineers on NFIP and flood control infrastructure. I discussed your paper w/ a colleague and he asked whether you intend to make a formal "request for correction" (RFC) to NOAA under its guidelines implementing the Information Quality Act of 2001 (Sec. 515 of PL 106-554) and relavent OMB guidance from 2002 and 2019 (updated). Your footnotes on p. 3 (nos. 9 and 11) link to the NOAA RFC criteria (pursuant to OMB-M-19-15). Many commenters here understandably lament the lack of agency accountability. A formal RFC might help change things. They're required by law. You've inspired me to examine doing same w/ FEMA and USACE.
I appreciate the opportunity to read the paper. The changing information and lack of transparency is truly troubling. I support your efforts to get them to improve the documentation of sources, methods and adjustments.
I live on the Gulf of Mexico coast and follow hurricane season intently. My experience and observation is over the last 30 years, in the counties here in Florida, there has been a significant increase in population and residential and commercial building. Clearly the same storm in 2023 vs 1993 would have a significant increase in monetary damage….. but wouldn’t be an indicator of climate change impacts.
I grew up in Oklahoma in tornado alley. When I visit now I see population growth and increased building. In a similar fashion I don’t believe increased $ damage would be a good indicator of climate impacts on the frequency or severity of tornadoes.
Keep up the good work. As we used to say when I worked as an engineer, show me your data. There is too much journalistic and political malpractice occurring to compound it with less than our best efforts with science.
hotair.com already has a link to your substack article. That could be fodder to label it as right wing spin. If realclearpolitics puts up a link to it that will make it very hard for Nature to drop it. A lot more people will read it from a RCP link than in Nature.
If I criticize Boeing for its numerous 737 Max issues, no one will expect me to write “the point here is not to call into question the reality of gravity.” I hope at some point we won’t have to write “The point here is not to call into question the reality or importance of human-caused climate change – it is real, and it is important.”
Good Luck Roger. You have exactly zero chance of influencing NOAA or having them adhere to their own scientific integrity policy. I tried multiple times and the only way to get them to change is by strong legal action and I doubt even that would work.
The group at NOAA NCEI, formerly NCDC, has a bunker mentality. Plus it brings lots of headlines and even though Tom Karl is gone, they can’t quit his approaches. I tried to force them to document everything by formulating a Climate Data Record (CDR) Maturity index that does all and more of what you suggest https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2012EO440006
This CDR maturity index has been widely adopted, including by the European Union, Committee on Earth Observation Satellites, and others. The NCEI surface products folks rejected it and will never ever adopt it; it didn’t come from them.
Let’s take a look how to properly document the world’s most used and cited climate data set – the International Cloud Climatology Project (ISCCP). This is where you get HurSat data also. Bill Rossow and I worked to ensure this important data set lived beyond our careers - https://www.ncei.noaa.gov/products/climate-data-records/cloud-properties-isccp
Everything needed is documented; algorithm, data flow, maturity index, manuals, and source code.
Virtually no details. Yes, some of the peer reviewed publications give some steps, but there is no chance you can re-produce their result. Hell, they can’t even reliably reproduce their result. I know since I was in charge of running the code for a bit. They took it away when I refused to let them make undocumented changes to the operational code.
Only a lawsuit, injunction, and then real consequences will have any chance of forcing NOAA NCEI to do the right thing.
Well done. The primary authors, and their management, must be aware of a significant fraction of problems you identify with their analysis. You must have many contacts at NOAA and I would assume you have provided some of the arguments to those contacts in the past (especially the very sensible GDP normalization) . Why do they persist with this? I hope this gets published close to its present form, and that productive discussions follow. Ron
The list is useful as a case-picker for "important" weather events, for little undergrad research projects on the meteorology of each one. Not so much for its growing length or trends, as you say. I wonder who touts up the latter? NOAA itself, or some independent layer of activists?
The same people who tell everyone to trust the science are the same people who ignore scientific integrity. Science without integrity is just propaganda aimed at supporting an agenda. I can understand why NGOs lack scientific integrity, but it's infuriating when taxpayer funded agencies pull this stunt.
You mentioned that your paper has not yet been peer-reviewed. Would it be possible for you to publish the peer reviewers' comments, without violating the integrity of the process? I would also be curious about what ire you might raise among the alarmists' crowd (I have visions of another Alimonti debacle).
Thanks for sharing the article. As an economist, I find the inflation-adjusted methodology amusingly egregious. While useful to express everything in comparable current dollars, when you combine it with the arbitrary nominal 1 billion $ threshold it seems expressly designed to constantly inflate the cumulative headline number, including by pulling in past disasters that were not previously counted.
Indeed, as you note, a more appropriate methodology would be to measure it all as a share of GDP. This would avoid the very counterintuitive phenomenon of old disasters inflating their way into the database; it would also give a better measure of the real economic damage. A richer economy has bigger “targets” – the same disaster will likely cause more dollars in damage, as you point out, but it also has a better capacity to absorb the cost. It’s the same rationale by which we look at public debt as % of GDP and not in nominal terms: the capacity to repay/absorb the cost matters.
Again, congratulations on an excellent and very important contribution; the extent to which we are expected to take on faith scientific arguments in the climate change debate is one of the most deleterious developments of the last many years.
From a slightly older KS friend:
In 1963 I was living on the top floor of Ellworth Hall at KU. KU is located on Mount Oread which is the highest point in Eastern Kansas, and Ellsworth in the highest place there. There was a magnificent view of farms and woodlands to the West. One Spring afternoon there was a storm front that approached from the West. That afternoon, you could see 7 tornados at the same time from the dorm windows. We were all Kansans and most had seen tornados before, but even we remarked that surely there would be a big write up in the local papers. There was nothing. A note in a small town local paper from mentioned that some farmer had lost a small out building.
Today the entire horizon as seen from Ellsworth is ranches homes, small businesses, schools and churches. If that event happed today, if would provide some Weather Channel specials that would last the rest of the decade. In 2011 one large tornado and several smaller ones tore thru Joplin Missouri. Killing several and doing millions of dollars worth of damage. I have bid and built a lot of work in the area. My best friend in 1962 was born there and we visited several times. I know where the tornados struck. In t4he 1960's a few cows would have died and no one would have noticed.
Sorry I came to this late. I'd prefer if you focused on specific aspects of "scientific integrity" and defined the concept. It might be an interesting grad student project to look at different agency's definitions. Ideas like "transparency of public information" were around long before the Data Quality Act or whatever is linked to current concepts of "scientific integrity."
I think we need look no further than some of the government scientist "let's not use our work emails" discussions during Covid to know there is a major transparency issue in government science, above and beyond other behaviors that might be called "against scientific integrity." But as you may recall I always thought the way that "scientific integrity" was promoted was basically political.. err.. signaling without much substance.
Roger perfect timing given yet another breathless take on weather disasters and insurance in WSJ today https://www.wsj.com/business/insurance-home-auto-rate-increases-climate-change-03b806f3. We refer to your work but what else can we do??
Roger... outstanding, as always, and the deep concerns re data integrity, reproducibility, and transparency are applicable across multiple Fed agencies, including outputs by FEMA and the US Army Corps of Engineers on NFIP and flood control infrastructure. I discussed your paper w/ a colleague and he asked whether you intend to make a formal "request for correction" (RFC) to NOAA under its guidelines implementing the Information Quality Act of 2001 (Sec. 515 of PL 106-554) and relavent OMB guidance from 2002 and 2019 (updated). Your footnotes on p. 3 (nos. 9 and 11) link to the NOAA RFC criteria (pursuant to OMB-M-19-15). Many commenters here understandably lament the lack of agency accountability. A formal RFC might help change things. They're required by law. You've inspired me to examine doing same w/ FEMA and USACE.
Have you reached out to NOAA directly about this in the past? Thinking of Mike Smith's efforts to improve NWS tornado warning system, which aren't just taking place through his blog but also direct outreach. If you do (or have done) that, it can help with FOIAs etc. https://www.mikesmithenterprisesblog.com/2023/10/nws-tornado-warning-problems-continue.html
I appreciate the opportunity to read the paper. The changing information and lack of transparency is truly troubling. I support your efforts to get them to improve the documentation of sources, methods and adjustments.
I live on the Gulf of Mexico coast and follow hurricane season intently. My experience and observation is over the last 30 years, in the counties here in Florida, there has been a significant increase in population and residential and commercial building. Clearly the same storm in 2023 vs 1993 would have a significant increase in monetary damage….. but wouldn’t be an indicator of climate change impacts.
I grew up in Oklahoma in tornado alley. When I visit now I see population growth and increased building. In a similar fashion I don’t believe increased $ damage would be a good indicator of climate impacts on the frequency or severity of tornadoes.
Keep up the good work. As we used to say when I worked as an engineer, show me your data. There is too much journalistic and political malpractice occurring to compound it with less than our best efforts with science.
hotair.com already has a link to your substack article. That could be fodder to label it as right wing spin. If realclearpolitics puts up a link to it that will make it very hard for Nature to drop it. A lot more people will read it from a RCP link than in Nature.
Wow!
Already my paper has been downloaded >10k times
Just for the record, I'll keep track of all of the climate beat reporters who have been in touch to ask me about it. Here is the list:
I'm shocked that a Nature journal invited this publication. There must be some other influential people in this field that are unhappy with NOAA.
If I criticize Boeing for its numerous 737 Max issues, no one will expect me to write “the point here is not to call into question the reality of gravity.” I hope at some point we won’t have to write “The point here is not to call into question the reality or importance of human-caused climate change – it is real, and it is important.”
Good Luck Roger. You have exactly zero chance of influencing NOAA or having them adhere to their own scientific integrity policy. I tried multiple times and the only way to get them to change is by strong legal action and I doubt even that would work.
The group at NOAA NCEI, formerly NCDC, has a bunker mentality. Plus it brings lots of headlines and even though Tom Karl is gone, they can’t quit his approaches. I tried to force them to document everything by formulating a Climate Data Record (CDR) Maturity index that does all and more of what you suggest https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2012EO440006
https://journals.ametsoc.org/view/journals/bams/97/9/bams-d-15-00015.1.xml
This CDR maturity index has been widely adopted, including by the European Union, Committee on Earth Observation Satellites, and others. The NCEI surface products folks rejected it and will never ever adopt it; it didn’t come from them.
Let’s take a look how to properly document the world’s most used and cited climate data set – the International Cloud Climatology Project (ISCCP). This is where you get HurSat data also. Bill Rossow and I worked to ensure this important data set lived beyond our careers - https://www.ncei.noaa.gov/products/climate-data-records/cloud-properties-isccp
Everything needed is documented; algorithm, data flow, maturity index, manuals, and source code.
Compare and contrast this with the source of all, yes all, surface temperature analysis data - https://www.ncei.noaa.gov/products/land-based-station/noaa-global-temp
Virtually no details. Yes, some of the peer reviewed publications give some steps, but there is no chance you can re-produce their result. Hell, they can’t even reliably reproduce their result. I know since I was in charge of running the code for a bit. They took it away when I refused to let them make undocumented changes to the operational code.
Only a lawsuit, injunction, and then real consequences will have any chance of forcing NOAA NCEI to do the right thing.
Well done. The primary authors, and their management, must be aware of a significant fraction of problems you identify with their analysis. You must have many contacts at NOAA and I would assume you have provided some of the arguments to those contacts in the past (especially the very sensible GDP normalization) . Why do they persist with this? I hope this gets published close to its present form, and that productive discussions follow. Ron
The list is useful as a case-picker for "important" weather events, for little undergrad research projects on the meteorology of each one. Not so much for its growing length or trends, as you say. I wonder who touts up the latter? NOAA itself, or some independent layer of activists?
The same people who tell everyone to trust the science are the same people who ignore scientific integrity. Science without integrity is just propaganda aimed at supporting an agenda. I can understand why NGOs lack scientific integrity, but it's infuriating when taxpayer funded agencies pull this stunt.
A good post, as usual. Thank you.
You mentioned that your paper has not yet been peer-reviewed. Would it be possible for you to publish the peer reviewers' comments, without violating the integrity of the process? I would also be curious about what ire you might raise among the alarmists' crowd (I have visions of another Alimonti debacle).
I look forward to reading your paper.
Thanks for sharing the article. As an economist, I find the inflation-adjusted methodology amusingly egregious. While useful to express everything in comparable current dollars, when you combine it with the arbitrary nominal 1 billion $ threshold it seems expressly designed to constantly inflate the cumulative headline number, including by pulling in past disasters that were not previously counted.
Indeed, as you note, a more appropriate methodology would be to measure it all as a share of GDP. This would avoid the very counterintuitive phenomenon of old disasters inflating their way into the database; it would also give a better measure of the real economic damage. A richer economy has bigger “targets” – the same disaster will likely cause more dollars in damage, as you point out, but it also has a better capacity to absorb the cost. It’s the same rationale by which we look at public debt as % of GDP and not in nominal terms: the capacity to repay/absorb the cost matters.
Again, congratulations on an excellent and very important contribution; the extent to which we are expected to take on faith scientific arguments in the climate change debate is one of the most deleterious developments of the last many years.