The Unstoppable Momentum of Outdated Science
Much of climate research is focused on implausible scenarios of the future, but implementing a course correction will be difficult
A 2015 literature review found almost 900 peer-reviewed studies published on breast cancer using a cell line derived from a breast cancer patient in Texas in 1976. But in 2007 it was confirmed that the cell line that had long been the focus of this research was actually not a breast cancer line, but was instead a skin cancer line. Whoops.
Even worse, from 2008 to 2014 — after the mistaken cell line was conclusively identified — the review identified 247 peer-reviewed articles putatively on breast cancer that were published using the misidentified skin cancer cell line. A cursory search of Google Scholar indicates that studies continue to be published in 2020 mistakenly using the skin cell line in breast cancer research.
The lesson from this experience is that science has momentum, and that momentum can be hard to change, even when obvious and significant flaws are identified. This is particularly the case when the flaws exist in databases that underlie research across an entire discipline.
In 2020, climate research finds itself in a similar situation to that of breast cancer research in 2007. Evidence indicates the scenarios of the future to 2100 that are at the focus of much of climate research have already diverged from the real world and thus offer a poor basis for projecting policy-relevant variables like economic growth and carbon dioxide emissions. A course-correction is needed.
In a new paper of ours just out in Environmental Research Letters we perform the most rigorous evaluation to date of how key variables in climate scenarios compare with data from the real world (specifically, we look at population, economic growth, energy intensity of economic growth and carbon intensity of energy consumption). We also look at how these variables might evolve in the near-term to 2040.
We find that the most commonly-used scenarios in climate research have already diverged significantly from the real world, and that divergence is going to only get larger in coming decades. You can see this visualized in the graph below, which shows carbon dioxide emissions from fossil fuels from 2005, when many scenarios begin, to 2045. The graph shows emissions trajectories projected by the most commonly used climate scenarios (called SSP5-8.5 and RCP8.5, with labels on the right vertical axis), along with other scenario trajectories. Actual emissions to date (dark purple curve) and those of near-term energy outlooks (labeled as EIA, BP and ExxonMobil) all can be found at the very low end of the scenario range, and far below the most commonly used scenarios.
Our paper goes into the technical details, but in short, an important reason for the lower-than-projected carbon dioxide emissions is that economic growth has been slower than expected across the scenarios, and rather than seeing coal use expand dramatically around the world, it has actually declined in many regions. It is even conceivable, if not likely, that in 2019 the world has passed “peak carbon dioxide emissions.” Crucially, the projections in the figure above are pre-Covid19, which means that actual emissions 2020 to 2045 will be even less than was projected in 2019.
Our study builds upon a growing literature — notably that led by our co-author Justin Ritchie of the University of British Columbia — indicating that commonly used climate scenarios are already well off track and will become increasingly off track. As Zeke Hausfather and Glen Peters write in Nature, the highest emissions scenario commonly used in research to represent a “business as usual” trajectory into the future “becomes increasingly implausible with every passing year.”
Another new paper, led by Brian O’Neill at the University of Denver and co-authored by many involved in scenario development, has also recognized that the real world and scenario architecture have drifted apart in the years since the scenarios were first developed. That is of course not surprising, as projecting the future is always challenging. Correspondingly, the authors “recommend establishing a process for regular updates” to the scenarios and recommend that key variables in the scenarios “be updated now to be consistent with new historical data.”
While it is excellent news that the broader community is beginning to realize that scenarios are increasingly outdated, voluminous amounts of research have been and continue to be produced based on the outdated scenarios. For instance, O’Neill and colleagues find that “many studies” use scenarios that are “unlikely.” In fact, in their literature review such “unlikely” scenarios comprise more than 20% of all scenario applications from 2014 to 2019. They also call for “re-examining the assumptions underlying” the high-end emissions scenarios that are favored in physical climate research, impact studies and economic and policy analyses. As a result of such high prevalence of such studies in the literature, they are also the most commonly cited within scientific assessments of the Intergovernmental Panel on Climate Change. O’Neill and colleagues find that the highest emission scenarios comprise about 30% of all applications in studies over the past five years, from a family of 35 different scenarios that they surveyed.
As a result of the growing recognition of the divergence between scenarios and the real-world, the climate research community is in a similar place to where the breast cancer research community was in 2007. Evidence is now undeniable that the basis for a significant amount of research has become untethered from the real world. The issue now is what to do about it.
The challenges for climate research are significant. Consider that in contrast to the 900 articles that misused a skin cancer line as a breast cancer line, our literature review found almost 17,000 peer-reviewed articles that use the now-outdated highest emissions scenario. That particular scenario is also by far the most commonly cited in recent climate assessments of the IPCC and the U.S. National Climate Assessment. And every day new studies are published using outdated scenarios.
The elevated role of scenarios across climate research means that there is a huge momentum behind their continued use. A research reset would be a massive endeavor and would require essentially writing off the policy, economic or other real-world relevance of thousands of studies, and perhaps even their scientific utility. Though to be fair, there are reasons to use exploratory scenarios in modeling or theoretical studies, but such uses shouldn’t be confused with practical relevance — just as studies of skin cancer lines should not be confused with breast cancer lines.
Make no mistake. The momentum of outdated science is powerful. Recognizing that a considerable amount of climate science to be outdated is, in the words of the late Steve Rayer, “uncomfortable knowledge” — that knowledge which challenges widely-held preconceptions. According to Rayner, in such a context we should expect to see reactions to uncomfortable knowledge that include:
denial (that scenarios are off track),
dismissal (the scenarios are off track, but it doesn’t matter),
diversion (the scenarios are off track, but saying so advances the agenda of those opposed to action) and,
displacement (the scenarios are off track but there are perhaps compensating errors elsewhere within scenario assumptions).
Such responses reinforce the momentum of outdated science and make it more difficult to implement a much needed course correction.
Responding to climate change is critically important. So too is upholding the integrity of the science which helps to inform those responses. Identification of a growing divergence between scenarios and the real-world should be seen as an opportunity — to improve both science and policy related to climate — but also to develop new ways for science to be more nimble in getting back on track when research is found to be outdated.