Bombshell: World Athletics Admits its Research Underpinning DSD Regulations is "Potentially Misleading"

Caster Semenya and several other athletes were banned from certain events in Tokyo, but in a scientific correction published today World Athletics admits flaws in research used to support the ban

Today, the British Journal of Sports Medicine quietly publish a correction to a 2017 paper. Corrections are common in research, as scientists are human and make mistakes, like anyone else. But one of the most important features of science is that it is self-correcting, and mistakes are identified, admitted and corrected. But the correction published today is not simply the admission an error in a inconsequential paper, it is an admission of error by World Athletics in the only empirical analysis which underpins its eligibility regulations for female athletes. The implications are massive.

The original 2017 paper was authored by World Athletics (WA, then called IAAF) employees Stéphane Bermon and Pierre Yves Garnier (hereafter BG17). The paper argued a causal relationship between testosterone levels and athletic performance among women, but just for certain events (which happened to include the events run by Caster Semenya, the South African runner targeted by the regulations). In 2018, when WA issued its regulations governing eligibility for female athletes the BG17 analysis was the only data and evidence provided in support of the events that were being regulated, from 400m to one mile. (RP: Over at my blog I explain in more detail why BG17 is so important.)

This is clearly explained in the FAQ that WA issued to accompany the regulations and reproduced below. The citation to BG17 as the only empirical basis appears in the regulations (which I reproduce in this Twitter thread).

In the correction published today, WA now admits that its claims of a causal relationship between testosterone levels and athletic performance among female athletes in the restricted events was incorrect.

To be explicit, there is no confirmatory evidence for causality in the observed relationships reported. . . With this in mind, we recognise that statements in the paper could have been misleading by implying a causal inference.

There are other fatal problems with the WA research, which you can read about here and here. There are still more errors for WA correct, but this is a good start.

While it must have been difficult for BJSM and WA to admit the error in a paper at the center of important regulations governing sport, it was absolutely the right thing to do. For its part, in its correction WA has called for an independent scientific review to “to establish confirmatory scientific evidence for the causal relationships between the variables analysed.” This also is the right thing to do.

The (flawed) evidence used to support the WA regulations was recently cited as being authoritative by Sebastian Coe, WA president, at the Tokyo Olympics after Christine Mboma of Namibia won a silver medal in the 200m. Mboma fell afoul of the regulations at her preferred distance of 400m, which she was barred from running. Coe’s invocation of the (now admittedly) flawed research illustrates its continuing weight in the regulations and publics justifications by WA for their necessity.

Given this remarkable turn of events, the following steps should now be taken:

• An independent review of the scientific claims found within the regulations should be undertaken, as called for by WA. Obviously, independence means that WA nor IOC should have absolutely no role in such a review. WA and IOC could request an authoritative, independent organization such as UNESCO or the InterAcademy Partnership to oversee such a review.

• While such a review is taking place, the WA female eligibility regulations should be suspend, with immediate effect. Today.

• BMJ, the publisher of BJSM, should itself conduct an internal review to understand how flawed science that supported important regulations remained uncorrected for years, long after egregious errors were widely reported. It is curious that the correction appears just a few weeks after the Tokyo Olympics.

• If WA and IOC do not immediately suspend the regulations contingent upon a review, then Caster Semenya’s legal team should request the immediate intervention of CAS. The judgment that CAS rendered in the Semenya case explicitly noted that future evidence would be cause for revisiting their decision. As well, WA’s note the importance of new evidence for implementation of the regulations, “If future evidence or new scientific knowledge indicates that there is good justification to expand or narrow the number of events affected by the Regulations, it will propose such revisions to the IAAF Council.”

• If such an appeal is brought, CAS should immediately suspend the regulations.

The WA correction offered today provides a very public test of the integrity of World Athletics. The organization chose to base its 2018 regulation on a set of scientific claims. It now admits that those claims were wrong and potentially misleading. Doing the right thing in support of the athletes that it represents means changing course when the facts warrant. Let’s see what happens.