The 2022 Seasonal Hurricane Forecast Bust
The 2022 Seasonal Hurricane Forecast Bust
What lessons can we take?
Predictions are a dime a dozen.
Who is going to win the next election? The next Super Bowl? The World Cup? What will tomorrow’s high temperature be? What will GDP growth be next year?
The possibilities are endless. And there are endless prognostications.
Seasonal hurricane forecasting is a thing. Every year, dozens of groups around the world issue forecasts for the coming year’s Atlantic hurricane activity. In 2022, these groups — as a collective — were wrong, and comprehensively so.
The Barcelona Supercomputing Center has collected seasonal hurricane forecasts from 29 organizations which allows a detailed analysis of just how wrong the community was in 2022. Of the 19 organizations that issued forecasts last spring, 17 of them predicted an above-average frequency of “named storms” (i.e., those that the U.S. National Hurricane Center names based on estimates of intensity). By early summer, 19 of 19 of these forecasts called for an above-average season.
The actual hurricane season was below average by most metrics, as you can see in the table below, via the one and only Phil Klotzbach at Colorado State University.
Seasonal hurricane forecasting of hurricanes was developed by the late William Gray (Klotzbach’s mentor and the first scientist to invite me to speak at a university, back in 1993) and has developed into a cottage industry.
But are they any good?
Forecasts can be evaluated based on a metric called “skill” which refers to how a forecast compares to the use of a naïve baseline expectation. If your methodology cannot improve upon a naïve baseline then it has no skill and its use should be questioned.
Seasonal hurricane forecasting is challenging because naïve baseline forecasts tend to do quite well. Consider that a persistence forecast of hurricane activity — assuming that the next year will follow the previous year in terms of being more or less active than the median — would have correctly anticipated more than 2/3 of hurricane activity in years since 1990.
Hurricane activity is streaky. The past 6 years have all been above median activity, but 2022 breaks that streak. Forecasting above median activity would have been a winner each of the past 6 years, but failed in 2022. And 1970 to 1994 (except two years) were all below median.
Some forecasters routinely miss the mark, For instance, Penn State (Michael Mann) issues forecasts of “named storms” but reality has fallen outside their wide forecast range in each of the past 4 years. In general, I do not like “named storms” as a metric because it involves judgement calls by NOAA hurricane experts (on whether to “name” a storm) and is thus less rigorous a metric than is satellite-derived estimates of hurricanes, major hurricanes or accumulated cyclone energy (ACE).
Predictions of ACE for 2022 made in August — after the hurricane season had started but before the bulk of activity — were also off target. Of the 10 groups forecasting seasonal ACE in August, all predicted that activity would be above the median of activity (of 1990 to 2021). In fact, 2022 ACE came it at about the 40th percentile of years since 1990.
Klotzbach at CSU does a very nice job looking back at their seasonal forecasts and assessing how well they performed. The figure below shows a fairly strong correlation between their forecasts and observations, but as I’ve noted, they have a very high bar to show skill, based on the strength of persistence and other naïve techniques.
In a 2018 paper led by Klotzbach — updating a discovery we first made in 1999 — we showed that landfalling major hurricanes occurred twice as often in La Niña years as compared to El Niño years, and 4 times as often for Florida. With Ian’s landfall in Florida as a Category 4 storm in 2022, this relationship gets even stronger. Climate change is important, of course, but its projected effects on hurricanes are measured in the few percentage points over a century — ENSO has effects that are orders of magnitude larger in just years. Don’t lose sight of climate variability.
So what lessons should we take from the seasonal forecasting bust of 2022?
Prediction is hard, especially about the future
That said, there are some indications of skill in seasonal hurricane forecasts, based on simple metrics and relationships
You can safely ignore all forecasting efforts that do not rigorously evaluate their past efforts
CSU and Phil Klotzbach remain the head of the class
If you care about landfalls and damage, then focus directly on correlates of landfalls and damage
Here is how I concluded a 2011 paper on the failures to generate 1 to 5 year predictions of U.S. hurricane activity:
Given that the climate system is known to be non-stationary on various timescales, there are of course good reasons to expect that uncertainties may be larger than the variability observed in the past, given that the climate system can assume modes of behaviour not observed over the past century and a half. Each decision maker should carefully evaluate how unknown unknowns might influence their judgements. In addition to decision making under conditions of uncertainty, decision makers need also to make judgements under conditions of ignorance, where uncertainties cannot be known with certainty.
Decision makers will continue to make bets on the future and, just like in a casino, some bets will prove winners and some will be losers. But over the long term those who do the best in the business of decision making related to hurricane landfalls and their impacts will be those who best match their decisions to what can and cannot be known about the uncertain future.
After the jump, paid subscribers can download the following three papers:
Pielke Jr, R. A. (2009). United States hurricane landfalls and damages: Can one-to five-year predictions beat climatology?. Environmental Hazards, 8(3), 187-200.
Klotzbach, P. J., Bowen, S. G., Pielke, R., & Bell, M. (2018). Continental US hurricane landfall frequency and associated damage: Observations and future risks. Bulletin of the American Meteorological Society, 99(7), 1359-1376.
Pielke Jr, R. A., & Landsea, C. N. (1999). La nina, el nino, and atlantic hurricane damages in the united states. Bulletin of the American Meteorological Society, 80(10), 2027-2034.
If you like what you read here please consider sharing, liking, subscribing, giving a gift subscription (free!) and commenting. All are welcome here and the community is growing.