When Bad Science Serves Injustice Scientists Should Act

The British Journal of Sports Medicine should retract fatally flawed research used to justify human rights violations

In a hard-hitting and incredibly disturbing report released today, Human Rights Watch documents humans right violations that have resulted from the “sex testing” policies of World Athletics, the international body that governs track and field. HRW documents in detail how the so-called “sex testing” policies violate fundamental and widely held human rights to privacy, health and non-discrimination.

Here I explain how these human rights violations have been enabled by the inexplicable inaction of editors of a scientific journal, the British Journal of Sports Medicine (BJSM). The research community should call upon the BJSM to uphold standards of scientific integrity that it professes to follow. By not upholding these standards, the research community is complicit in enabling the continuing human rights violations.

Here are the details: World Athletics justifies its “sex testing” policies based on a scientific paper that was published in 2017, written by two of its employees, Stéphane Bermon and Pierre Yves Garnier. That paper argued that females with certain biological characteristics associated with high testosterone levels “tended to have distinct advantages in races from a quarter mile to a mile,” as explained by a New York Times article on today’s HRW report.

World Athletics mischaracterizes such women as “biological males” and bans them from competing at these distances unless they undergo medically unnecessary treatments or surgeries. HRW explains:

Sex testing violates a range of internationally protected fundamental rights including to privacy, dignity, health, non-discrimination, freedom from ill-treatment, and employment rights. These punitive regulations push them into unnecessary medical procedures that are conducted in coercive environments in which humiliated women are forced to choose between their careers and their basic rights.

In response to the HRW report, World Athletics repeated to the New York Times its frequent claim that its research published in BJSM justifies the regulations:

In a statement, World Athletics rejected the accusation that biological limits set on athletes competing in certain women’s events were based on race or gender stereotypes. Rather, the governing body said, the testosterone regulations provide “an objective and scientific measure” to preserve equitable competition.

In 2018, upon World Athletics release of the then-proposed “sex testing” regulations and their basis in the 2017 BJSM paper, I (along with Ross Tucker, University of Cape Town and Erik Boye, University of Oslo) requested and were provided by Dr. Bermon a subset of the data used in his 2017 paper.

What we found when we received that data was simply shocking.

The data was rife with errors. In the track and field events between 400m and one mile that are the focus of the “sex testing” regulations, we found that a significant number of data points were incorrect, duplicated or unrelated to actual performances in competition. At 400m, for instance, of the 67 data points used in the paper, 22 (one third) were bad data.

Upon discovery of the flawed data, World Athletics (then IAAF) quickly sent a letter to BJSM, which — again inexplicitly — was rushed into publication without peer review. The letter admitted to the data errors (and in the process dropping 220 flawed data points of only about 2000 total) and purported to conduct a “do over” of the original analysis. The “do over” analysis employed new methods which changed the results significantly, leading us to conclude, “the large magnitude of performance changes on all athletes when correcting these problematic data points significantly undermines any conclusions that can be drawn.”

Because the World Athletics analyses was so obviously problematic and also the centerpiece of the proposed regulations, we contacted the BJSM editors. What happened next is unprecedented in my 25+ years in research:

As we explored the issues associated with the data and results of [the 2017 study] and [the 2018 letter], we were in extensive contact with the editors of BJSM. Our request that the journal work to secure the release of the balance of performance data of [the studies] was refused. Similarly, once we identified errors in the data that were provided to us by Dr. Bermon, the editors of BJSM rejected our call for the paper to be retracted. Further, after reviewing a submission from us on the flaws in [the 2017 study] and the irregularities in the process, BJSM refused to publish our analysis after more than 3 months of deliberation—not due to any scientific shortfalls (indeed, the BJSM peer reviewers judged our analysis to be scientifically sound) but because our analysis was critical of how the issue was handled by BJSM.

World Athletics continues to justify its regulations on the fatally flawed 2017 study. Meanwhile, important international bodies have continued to document how implementation of the regulations violates fundamental human rights and medical ethics. These bodies include not just Human Rights Watch and its report today, but also the World Medical Association and the United Nations Human Rights Council among other organizations.

The flawed research published in BJSM enables and empowers these violations because World Athletics continues to invoke the authority of science as justification for the legitimacy and appropriateness of its “sex testing” regulations. The continuing failure of BJSM to uphold fundamental standards of research integrity thus allows bad science to be exploited in service of injustice.

One good thing about science is that it is, in principle, self-correcting. It is never too late to correct flawed science. BJSM retraction of the flawed World Athletics research is long overdue. It should be obvious to any independent observer that the research is sufficiently flawed to warrant retraction. At the very least, the parent organization of BJSM — The BMJ —should immediately convene an independent evaluation of the World Athletics research and determine whether retraction is indeed warranted.

The BMJ states that it adheres to the Singapore Statement on Research Integrity, which, among other principles, states that when “irresponsible research practice is confirmed, appropriate actions should be taken promptly, including correcting the research record.” That has not happened here. But it is not too late.

Correcting the research record by itself won’t necessarily lead to changes in World Athletics implementation of its policies that violate fundamental principles of human rights and medical ethics. However, it would eliminate the role of the scientific community as legitimizing these violations by continuing to allow deeply flawed science stand uncorrected. Correcting the research record is, simply put, the right thing to do.