RIP NOAA's Billion Dollar Disasters
The curious end to a troubled tabulation
Last June, my peer-reviewed paper on problems with NOAA’s Billion Dollar Disaster (BDD) tabulation was published. Today, NOAA announced that the BDD tabulation would no longer be updated by the agency, explaining that it has been “retired.”
While some media have chosen to make the BDD retirement about the Trump administration, there has also been some excellent reporting indicating that the retirement of the tabulation is a bit more complicated.
CBS News has excellent reporting:
According to a current NOAA employee, who spoke on condition of anonymity out of fear of losing their job, part of the reason the billion-dollar database is being decommissioned is because it is "not core" to NOAA's mission, since its focus is on economics and an analysis of various datasets. Additionally, the lead researcher of the database resigned last month by taking a separation incentive package. . .
Criticism had been mounting against the billion-dollar database over the years, driven by Roger Pielke Jr., a senior fellow at the American Enterprise Institute and faculty at the University of Colorado-Boulder. Pielke argued it needs better metadata, review and documentation. He published a paper in the journal Nature last year about the database, saying it failed to meet NOAA's "criteria of procedure and substance" based on the agency's "information quality and scientific integrity policies."
In a phone interview with CBS News, Pielke said, "I'm a huge fan of NOAA and the National Weather Service, they play a huge public service role, and I'm not happy with the sledgehammer that is being taken to the agency."
He said having climatologists in charge of the data was problematic and resulted in some quality-control issues that could be addressed by proper economists. He also said the current dataset isn't fixable at NOAA, but that it would be very important for another federal agency to take disaster losses seriously and track the data.
Interestingly, according to the CBS News report, NOAA leadership under President Biden was apparently quite sympathetic to my critique.
The Washington Post also recognizes the nuance here, building on the paper’s earlier excellent deep dive (by Harry Stevens, now at the NYT) into the Billion Dollar Disaster tabulation:
Roger Pielke, a senior fellow at the American Enterprise Institute, said tracking disaster costs is essential but added that there are better approaches than the one NOAA has since taken. As a professor emeritus at the University of Colorado at Boulder, Pielke studied the database and said he found inconsistencies in how loss estimates were calculated or inflation was factored in over the nearly five decades of disasters the agency tracked.
“It’s absolutely important for the federal government to track the cost of disasters; there’s no doubt about that,” Pielke said. “Perhaps this is an opportunity to do things right.”
The Post includes an important nugget of information:
The many recent departures at the agency include Adam Smith, who led the billion-dollar disaster program for 15 years before leaving last week.
Because the methodologies used by NOAA to generate loss estimates are neither public nor (to my knowledge) written down, it may be that with Smith’s departure the agency may have lost capacity to carry on. Keep a lookout for the tabulation to reappear at a climate advocacy group, which is probably where this incarnation best fits.
Interestingly, sometime after my paper was published last year, NOAA added a disclaimer of sorts to the BDD homepage:
This product has no focus on climate event attribution.
After years of saying the opposite, encouraging media to report the opposite, and watching policy makers claim the opposite, it turned out that it was too late for NOAA to get the horse back in the barn.
One other behind-the-scenes detail — Next Wednesday the House Science Committee was scheduled to hold a hearing on the BDD tabulation and NOAA’s efforts to correct its many substantive and procedural issues. I had agreed to testify. That hearing is now off the schedule for next week.
I have no idea who else may have been invited to testify, but I am absolutely certain that with the retirement of NOAA’s most knowledgeable employee on the BDD tabulation, the prospects of trying to defend it before Congress could not have been that appealing to the agency. The same goes for any external expert.
I suspect that today’s announcement was more about NOAA opportunistically getting rid of some dirty laundry as the Trump Administration takes a sledgehammer to the agency — Something that I strongly oppose.
NOAA, and its National Weather Service, sit in the Department of Commerce for a reason. There are mission-oriented agencies that rely on scientific research to serve all facets of the economy, from supply chains to aviation to agriculture to public safety. I could go on. The Trump administration’s decimation of the agency in the absence of a policy plan makes little sense, and appears to be more about retribution than anything else.
The scientific integrity lapses of the BDD tabulation are in my view an anomaly, and evidence of what happens when politicized science takes on a life of its own. Ignore scientific integrity along the way and at the end you get a car crash, which is what we see today.
What should happen now?
Congress — remember them? — should pass legislation directing one of the nation’s statistical and economic agencies to develop a scientifically sound methodology for tabulating disaster costs to meet the needs of a range of stakeholders. I would propose the Bureau of Economic Analysis, but there are other candidates.
There are different ways to estimate disaster costs — direct, indirect, insured, agricultural, commodity effects, public, federal, etc.. There is no such thing as a single metric, which is one reason why economists should be in charge of loss estimation as they can tailor a suite of products based on the needs of those who use the data. The demise of NOAA BDD can serve as an impetus for better data.
One thing that makes science science is that it is self-correcting. Hopefully NOAA’s announcement today to step away from its BDD tabulation represents an important step in that direction. But it is just one step.
If you want to take a deeper dive into my journey with the BDD tabulation here is a timeline:
2012: Bad Economics at NOAA
2012: The Washington Post, 2011 billion dollar weather disaster record: legit or bad economics?
2019: Forbes, Everything You Hear About Billion-Dollar Disasters Is Wrong
2024: npj Natural Hazards, Scientific integrity and U.S. “Billion Dollar Disasters”
Comments, questions, critique always welcomed!
The easiest thing you can do to support THB is to click that “♡ Like”. More likes mean that THB gets in front of more readers!
If you value the efforts here at THB to contextualize science in policy and politics, please consider subscribing, sharing, and supporting. THB exists because of your support. Remember, honest broking is a group effort.
I understand (probably) your concern about the Administration’s draconian approach to change - especially as that applies to NOAA (where my father-in-law worked years ago). But perhaps the upside of all this would be found in your call for Congress to do their job with respect to taking more control over the obese and inefficient-ineffective federal bureaucracy…? Of course, one wonders if Congress would be any good at this job considering the limited talent pool found in the Congress…!
Thanks for the the report. My only quibble is with the implied thesis that BDD tabulation can be computed without bias, and is fact. I think that is extremely optimistic and difficult to prove. In the same way you don't like the Trump admin taking a sledge hammer to NOAA and other federal science policy agencies (which should not be cause for celebration by Republicans) conservatives find fault with belittling and dismissing scientists who claim that broadly, climate science "consensus " is based in part on assumptions built into modeling and even data acquisition. Essentially confirmation bias is rampant in the marketplace. As well, ,the seeming disinterest of 21st century climate haranguers to understand that the climate itself is a complex system predicated on and interacting with other complex systems is pretty amazing here in the cheap seats. Conflating complication with complexity is logic 101 stuff. Ergo making prognostications about what the future looks like is at best fraught and at worst...well I would suggest everyone involved examine the predictions of climate science since the 1970s - we were about to enter a new ice age then and are in the middle of global death now.
I appreciate your ardor and candor. It is refreshing for someone involved in rhetoric and review about a very politicized field to show some measure of humility regarding the limitations of your own predictive abilities. It would be fantastic if likewise the science PhDs doing the hard math, etc did the same. The sky is falling crowd inevitably get it wrong because of the gang warfare they employ that ruins whatever "science" they claim domain over.