There are plenty of statistics out there that we know just aren’t true — like 10,000 hours of practice at something will make you a world expert and we have just 10 years left to save the planet. Such falsehoods are mostly harmless expressions that arise in popular discourse. Peer-reviewed research, however, seeks to holds itself to a much more rigorous standard. As Ole Bjørn Rekdal explains, “Academic publications should have different standards and requirements regarding truth and accuracy from other mediums, and should not be a playground for rumors and urban legends.” And yet, academic urban legends are all too common. Identifying and correcting them is difficult, but matters more, because as you’ll see below, they can be impactful in policy and politics.
Earlier this week Professor Orla Hegarty, of University College Dublin posted this interesting thread that allows us to track an academic urban legend to its source. Follow along as I go down the rabbit hole.
The academic urban legend at issue here is the claim that only 0.1% — 1 in 1,000 — of COIVD-19 cases is the result of outdoor transmission. Prof. Hegarty identified several peer reviewed papers making this claim, including BMC Infectious Diseases and Environmental Health Perspectives, two widely respected journals.
The most recent peer-reviewed assessment of studies seeking to quantify the proportion of COVID-19 spread that occurs outdoors suggests that outdoor spread is indeed much lower than spread that occurs indoors. “The proportion of infections happening outdoors is likely much lower than 10%; and some of the studies we have reviewed thus far suggest that it is likely less than 1%.”
The 0.1% statistic was created by a journalist, Ronan McGreevy of the Irish Times, who was kind enough to contact me after I had retweeted Prof. Hegarty’s thread. Here is where the 0.1% came from, according to McGreevy.
On 1 April 2021 he sent a query to Ireland’s Health Protection Surveillance Centre (HSPC) asking if the agency “produced figures for Covid-19 infections acquired at an outdoor setting.” The HSPC response was that “262 confirmed linked cases associated with locations or activities associated with outdoor locations” included the following statement (emphasis added):
Locations which are primarily associated with outdoor activities i.e. outdoor sports and construction sites, or outbreaks that specifically mention in comments that an outdoor location/activity was involved were included in this query. It must be noted however, that we cannot determine where transmission occurred.
I asked McGreevy exactly where the 0.1% number came from, as it was not in the response to his query that he received from the HSPC. He told me the he calculated it himself based on the total number of official COVID-19 cases recorded by the HSPC since the start of the pandemic: “it was me that extrapolated the figures by dividing 232,164 by 262 which gives a figure of 0.00112 which is rounded down to 0.01. I decided on the one in a 1,000 based on that figure.”
McGreevy wrote a news article on the statistic, which was published 6 April 2021, with the headline, “Outdoor transmission accounts for 0.1% of State’s Covid-19 cases.” You can see the top of the article below.
McGreevy included in the body of the story a bit of a disclaimer: “The HSPC said, however, that it “cannot determine where transmission occurred”. In addition 20 per cent of all cases in the State result from community transmission where the source of the infection is not known.” Of the headline, McGreevy told me, “I didn’t write the headline. I don’t know who did. It would have gone through many iterations with sub-editors etc.”
From there, the statistic traveled around the world. On 5 May 2021 a prominent U.S. observer of public health promoted the statistic on Twitter (below), linking to the Irish Times article and elevating the source to an “analysis.” This Tweet was cited on 11 May 2021 by David Leonhardt of the New York Times who further elevated the stature of the statistic to being the result of a “study” with apparently more rigor than other such studies: “A study from Ireland, which seems to have been more precise about the definition of outdoors, put the share of such transmission at 0.1 percent.” Of course, there was no such study.
Leonhardt’s column took strong issue with the claim by the U.S. Centers for Disease Control that outdoor COVID-19 transmission was less than 10%, and cited the 0.1% number twice in his critique, linking back to the article in the Irish Times:
In truth, the share of transmission that has occurred outdoors seems to be below 1 percent and may be below 0.1 percent, multiple epidemiologists told me. The rare outdoor transmission that has happened almost all seems to have involved crowded places or close conversation.
Saying that less than 10 percent of Covid transmission occurs outdoors is akin to saying that sharks attack fewer than 20,000 swimmers a year. (The actual worldwide number is around 150.) It’s both true and deceiving.
There is something deeply ironic and troubling when a major media outlet like the New York Times scolds the CDC for being “deceiving” by both invoking and helping to create an academic urban legend. Because the New York Times is the New York Times, the claim that CDC was deceiving the public was quickly amplified - by Fox News (below), the New York Post and others, and in the process the origins of the 0.1% statistics were lost.
From there the statistic continued to travel. On 11 May 2021 the 0.1% statistic made it to a Congressional hearing with the head of the CDC, where a Senator attributed the statistic to epidemiologists and cited it to suggest important policy implications: “There are some epidemiologists who say that it could be 0.1% of cases. That's a really important difference, and I assume the difference between 5% and 0.1% would likely educate decisions you would make about what recommendations you make to summer camps.” No one questioned the source or validity of the 0.1% figure. It had achieved truthiness.
Then, on 27 July it appeared in a peer-reviewed paper, presented as the result of “study” and now with 2 decimal point precision.
The creator of the statistic, McGreevy of the Irish Times, Tweeted that he is not responsible if medical journals misrepresent information in their publications: “I am flattered. It is not my fault if medical journals cite what I have written as if it was an academic study. That’s on them not me.”
While we might question McGreevy’s methods of analysis in producing the original 0.1% statistic, it is clear that the rise of this particular academic urban legend was given a huge boost by major media and later scientists and the journals they publish in. Academic urban legends can be consequential, in this particular instance perhaps contributing to reduced public trust in the CDC and poorly-informed policies. Be careful — sometimes misinformation is hidden in plain sight.
And yet I’ve not heard of data countering this figure or close to it. Is there actually a proper study? Pronouncements of the CDC don’t count.