Scientific integrity and U.S. “Billion Dollar Disasters”

A high-profile opportunity for self-correction in climate science and policy

Pielke 2024

Today, npj Natural Hazards, a journal in the Nature family of journals, officially published my new paper, “Scientific integrity and U.S. “Billion Dollar Disasters.”

The paper shows — irrefutably in my view — that the “billion dollar disaster” tabulation of the National Oceanic and Atmospheric Administration (NOAA), fails to meet the agency’s standards for information quality and scientific integrity.¹

For reasons I describe in detail in the paper, the “billion dollar disaster” tabulation is not suitable as a “database” (scare quotes — it is not data by any standard) for the detection and attribution of trends in extreme weather. Similarly, the tabulation is not suitable for identifying the consequences of changes or variability in climate on the costs of disasters. The dataset has been widely misused inscience, by the media, and in policy.

It is, in a word, misinformation.²

Here is how the paper starts:

In the late 1990s, the U.S. National Oceanic and Atmospheric Administration (NOAA) began publishing a tally of weather and climate disasters that each resulted in more than $1 billion in damage, noting that the time series had become “one of our more popular web pages”¹. Originally, the data was reported in current-year U.S. dollars. In 2011, following criticism that the dataset was misleading, NOAA modified its methods to adjusted historical losses to constant-year dollars by accounting for inflation.

By 2023, the billion dollar disaster time series had become a fixture in NOAA’s public outreach, was highlighted by the U.S. government’s U.S. Global Change Research Program (USGCRP) as a “climate change indicator,” was a cited as evidence in support of a “key message” of the Fifth U.S. National Climate Assessment showing that “extreme events are becoming more frequent and severe.” The time series is often cited in policy settings as evidence of the effects of human-caused climate change to increase the frequency and intensity of extreme weather events and associated economic damage, including in federal agencies, Congress and by the U.S. President. In addition to being widely cited in justifications of policy, as of March, 2024, NOAA’s billion dollar dataset has been cited in almost 1000 articles according to Google Scholar.

NOAA’s “billion dollar disaster” tabulation began as a simplistic but clever way to market NOAA and to attract the attention of reporters with a clickbaity listicle. At some point along the way, the “billion dollar disaster” list was somehow transformed into “data” used in peer-reviewed research, an official indicator of human-caused climate change featured by the U.S. National Climate Assessment, and used by the administration of President Joe Biden to justify a wide range of regulations and policy.

It is a remarkable story of how science can get off track and how misinformation can exist in plain sight, just like the emperor’s new clothes.

Here are some examples of such misinformation in plain sight that I cite in my paper:

• The NOAA official responsible for overseeing the dataset claimed that the dataset showed: “Climate change is supercharging many of these extremes that can lead to billion-dollar disasters.”

• At the press conference where the 2022 dataset was released, the NOAA Administrator claimed that the dataset indicated that, “Climate change is creating more and more intense extreme events that cause significant damage.”

• In 2021 the U.S. Department of Treasury identified increasing billion dollar disasters as evidence of the effects of climate change on financial risks.

• The Fifth U.S. National Climate Assessment cited the NOAA dataset as evidence that “Climate change is not just a problem for future generations, it’s a problem today,” and claimed that the dataset, in part, demonstrated “the increasing frequency and severity of extreme events” due in part to “human-caused climate change.”

• In 2023, President Biden attributed weather and climate-related disaster costs in the U.S. in 2022 to climate change, citing the NOAA dataset: “[C]limate change related extreme weather events still pose a rapidly intensifying threat – one that costs the U.S. at least $150 billion each year … This year set a record for the number of climate disasters that cost the United States over $1 billion. The United States now experiences a billion-dollar disaster approximately every three weeks on average, compared to once every four months during the 1980s.”

The failures of the “billion dollar disaster” dataset provide a high-profile test of NOAA — and the broader scientific community, the major media, and the Biden Administration — to correct course when science gets off track. I conclude the paper with some specific recommendations:

Identifying the reasons why NOAA’s billion dollar disaster dataset has departed so significantly from the agency’s own standards of scientific integrity goes well beyond the scope of this paper. However, the steps necessary to bring the dataset back into conformance with NOAA’s information quality criteria are straightforward:

• Publish all data, including all versions of the dataset;

• Document and publish baseline loss estimates and their provenance;

• Clearly describe all methodologies employed to adjust baseline data;

• Document every change made to the dataset, give each successive version of the dataset a unique name, and publish all versions of the data;

• Maintain all historical versions of the dataset in a publicly accessible archive;

• Subject the methods and results to annual peer review by experts, including economists and others with subject matter expertise, who are independent of NOAA. Make the peer review reports public;

• Align NOAA’s practices with federal government policies for disseminating statistical information that are applied to other agencies;

• Align claims with IPCC methods and standards for any claims of detection and attribution, or justify why the claims are at odds with those of the IPCC.

I enjoyed working in a NOAA cooperative institute at the University of Colorado Boulder for 16 years. I have many friends, colleagues, and collaborators in the agency. NOAA does excellent work — across both research and operations — and every minute of every day contributes positively to making the nation safer and furthering its economy. That makes the agency’s failures on the “billion dollar disasters” so remarkable and troubling.

Here is how I end the paper:

NOAA is a crucially important agency that sits at the intersection of science, policy and politics. It has a long and distinguished history of providing weather, climate, water, ocean and other data to the nation. These data have saved countless lives, supported the economy and enabled significant scientific research. The agency is far too important to allow the shortfalls in scientific integrity documented in this paper to persist. Fortunately, science and policy are both self-correcting.

I’ll be watching closely to see how NOAA responds to this new paper, how (and if) the media continues in its promotion of the “billion dollar disaster” tabulation in light of this new peer-reviewed analysis, and what sorts of self-correction might take place among policymakers who have used the dataset to justify specific policies.³

You can read the paper in full here and access and an enhanced PDF here — all open access.

❤️Click the heart to help THB be seen by more Substack readers!

I welcome your comments, critique, alternative views, and discussion. Please share the paper and this post widely. THB is reader supported — In addition to what you read here, your support is what enables me to write peer-reviewed papers such as this one. I have no other support for research other than THB subscribers — and I like it that way. Please consider joining those who support the work of THB with a paid subscription. Thank you for your support!

1. I am happy to hear counter-arguments. The review process was smooth and the peer-reviewers offered very constructive advice that improved the paper, but no counter-arguments related to NOAA’s scientific integrity failures.

2. The longer the “billion dollar disaster” tabulation goes without correction to uphold standards of scientific integrity, the greater the risk to NOAA and effective climate action.

3. Let me add a teaser — thanks to a tip I received after my paper was accepted and in press, I have learned that there is much more to this story which I’ll be revealing later in June.