Fishy Science
Fishy Science
Implications of the retraction of a recent paper for scientific integrity in the White House
Recently, the Proceedings of the National Academy of Sciences (PNAS) retracted a highly influential paper on marine protected areas and fishing due to identification of significant errors that undercut the paper’s results as well as significant irregularities in the peer review process. What makes this particular retraction of unusual interest is that the irregularities in the PNAS peer review process involve Dr. Jane Lubchenco, the White House official who is currently overseeing President Biden’s Scientific Integrity Task Force.
The paper, A global network of marine protected areas for food (Cabral et al. 2020, hereafter C20), was published by PNAS in October 2020, and has been highly reported on due to its perceived policy relevance. Dr. Lubchenco served as its editor for PNAS. That means that she was responsible for overseeing the paper’s journey through the peer review process, including the selection of reviewers. We now know that Dr. Lubchenco violated PNAS guidelines for conflict of interest, and not unknowingly or in a small way.
The details matter here, so I am going to explain them.
The issues in peer review of C20 at PNAS are not subtle. The authors of C20 included seven researchers who Dr. Lubchenco was collaborating with on a different paper — Sala et al. 2021 (published in Nature, hereafter S21) — that built upon the results of C20. Even though C20 was published first (26 Oct 2020 vs 17 Mar 2021), S21 was actually submitted three weeks prior to C20 (17 Dec 2019 vs. 6 Jan 2020). So at the time that Dr. Lubchenco assumed the role of editor for C20, she had just submitted a different paper with seven authors of C20 that built upon the S21 results — S21 thus depended upon the successful publication of C20.
Already, this is an egregious violation of scientific integrity. It gets worse. One of Dr. Lubchenco’s co-authors on S21 who was also a co-author of S20 — the paper she was editing for PNAS — was her brother-in-law. These various conflicts were called to the attention of PNAS in April 2021 by Dr. Magnus Johnson, of the University of Hull in the UK, prompting an investigation.
The PNAS guidelines are completely clear (emphasis added):
A competing interest due to a personal association arises if you are asked to serve as editor or reviewer of a manuscript whose authors include a person with whom you had an association, such as a thesis advisor (or advisee), postdoctoral mentor (or mentee), or coauthor of a paper, within the last 48 months. When such a competing interest arises, you may not serve as editor or reviewer.
A competing interest due to personal association also arises if you are asked to serve as editor or reviewer of a manuscript whose authors include a person with whom you have a family relationship, such as a spouse, domestic partner, or parent–child relationship. When such a competing interest arises, you may not serve as editor or reviewer.
While Dr. Lubchenco should not have edited C20, to be completely fair to her, the “competing interest due to a personal association” guideline does not appear to be much enforced by PNAS. I was able to quickly identify multiple violations of this guideline via a simple search — here are one, two, three examples — the third of which also involves Dr. Lubchenco as editor. If PNAS retracts papers that violate its competing interests guidelines, they will no doubt find a rather large set of papers.
There is more to the story.
On November 17, 2020 Dr. Lubchenco testified before the House Natural Resources Committee, in support of Congressional legislation to establish protected areas in marine ecosystems, relying on C20. She did not disclose that she had shepherded the paper through peer review, nor did she disclose that she was a collaborator in the research. The unavoidable impression that this sequence of events gives is the creation of “policy-based evidence” — that is, evidence that is created for the explicit purpose of supporting particular policy or political outcomes, like the passing of legislation.
The impression of “policy-based evidence” is further supported by the fact that the failures of the peer review process in this instance are not just procedural, they are substantive as well. It turns out that the science of C20 is also fatally flawed. In an excellent and comprehensive post, Max Mossler goes into detail on the errors of C20, how they were identified, and how they also call into question the validity of S21 — which has also received an incredible amount of media and policy attention. Here I’ll just report his excellent bottom line:
Regardless of any conflict of interest, the science in both Cabral et al. and Sala et al. is critically flawed, but being used to advocate for public policy. Both follow a recent trend of publishing predictions that use a limited set of assumptions (in a very uncertain world) to produce global maps that get published in high-profile journals and garner considerable media and political attention.
Computer models are essential tools for science and management, but the accuracy of their predictions depends on both the quality of the data and the assumptions they are based on. Often, a problem is so complex that several assumptions may be equally plausible; readers need to be made aware when different assumptions lead to vastly different outcomes.
The Cabral et al. and Sala et al. papers disregard uncertainty in favor of set values for their model parameters. They don’t account for the enormous uncertainty in these parameters and don’t provide strong evidence that their choice of values was correct. The assumptions and parameters produce big headlines, but are fundamentally unhelpful for the future of ocean governance and sustainability. We expect policy-makers and resource managers to make decisions based on the best available science. Inconsistent and unrealistic assumptions are not that.
And if all of that is not bad enough, it still gets worse. S21 reports (inaccurately) that its projections are based on the IPCC SRES A2 scenario — which for anyone who knows anything about climate scenarios, would have been an incredibly odd choice, not least because that scenario is more than 20 years old and rarely (if ever) used in research today. It turns out (if you dig deep enough) that C20 and S21 are in fact not based on the IPCC SRES A2 scenario, but instead on the implausible RCP8.5. That the authors don’t know the difference between A2 and RCP8.5 is itself problematic. That RCP8.5 is being used to generate predictions for use in marine/fisheries policy is even more problematic.
So we have quite a mess here. Going forward, here are some recommendations:
Nature should immediately evaluate S21 for retraction, as it is based on C20, which is now retracted. It is difficult to see how S21 can stand unretracted.
PNAS properly retracted C20, but the journal should also do a comprehensive audit to assess the extent of other violations of its conflict of interest guidelines. A cursory look suggests that such violations are not uncommon.
Given Dr. Lubchenco’s significant violations of PNAS policies to publish flawed research, and then using that flawed research to advocate for policy, the White House should reconsider her leadership role in its Scientific Integrity Task Force. Otherwise, it would be fair to ask if scientific integrity guidelines are optional, depending on your politics.
This episode provides good news and bad news. The good news is that it underscores that science is indeed self-correcting, even if that process takes a while. In the long run, better science defeats bad science. The bad news is that in the short term, leadership and institutions failed. This episode of fishy science is not over yet — PNAS, Nature and the White House still have important roles to play in ensuring scientific integrity. Watch this space.
Jane was properly penalized for her human mistake, i.e., being overly enthusiast about MPAs to cut some ethical corners. But Stev, I'd be careful about throwing stones, as most people live in glass houses. Fortunately, science is more self-correcting than other disciplines:
Vadas, R.L. Jr. 1994. The anatomy of an ecological controversy: honey bee searching
behavior. Oikos 69: 158-166 (https://www.beesource.com/threads/the-anatomy-of-an-ecological-controversy-honey-bee-searching-behavior.365460).
And MPAs are valid mgmt. tools, though we need to stay honest & transparent about such research.
-Bob V.
Though Jane Lubchenco has been caught up in this sort of brouhaha is disturbing given her role in an “integrity task force,” it is much less so than some of her past support for censorship on population issues. In the late 1980s she was appointed chair of the Ecological Society of America (ESA) committee that eventually produced the “landmark” Sustainable Biosphere Initiative (1991) white paper. While still in draft form that was criticized by some highly reputable reviewers for giving essentially zero attention to the issue of overpopulation as the biggest obstacle to “sustainability.” But Lubchenco and her committee declined to address the deficiency. Indirectly they admitted that the whole report was basically a gussied-up request for more funds for ecological research, so no offending of congressmen would be risked. Following publication of the report, other scientists chimed in with the same complaint. More-or-less complete suppression of the topic by ESA continued for the next couple of decades, until I and a few other anti-censorship, pro-environment scientists managed to put together and have approved a half-day session for ESA’s 2012 annual meeting, titled on “Population, Environment and Sustainability Issues in the U.S.”. Documentation of all of the above, with additional information on Lubchenco, ESA, AAAS, and a Californians for Population Stabilization (CAPS) exhibitor booth at the 2012 ESA meeting, can be found in “Population Camel Gets Its Nose into Ecologists’ Tent: Hope Is High That the Rest Will Follow.” https://www.thesocialcontra...
CAPS applied to have an exhibitor booth at the 2012 AAAS annual meeting in Vancouver and was turned down. That was one factor that incentivized the creation of a new organization, Scientists and Environmentalists for Population Stabilization (SEPS), dedicated primarily to operating just such booths at scientific meetings.It has now done this with great success for 26 different societies. But when SEPS applied to have a booth at the 2014 and 2016 AAAS meetings it was also turned down. AAAS’s ever-changing pretexts were “diverse, disingenuous, and puzzling,” and every time we objected to the entire AAAS board of directors. And every time they pleaded, in essence, “not in our job description – it’s a staff issue.” A preliminary account of these battles has been published: “AAAS Wields the Censor’s Hammer on U.S. Population Issues” https://capsweb.org/blog/aa...
May Berenbaum was a AAAS board member at the time, and never replied when a SEPS advisory board member who had gone to grad school with her asked for help or advice. Lubchenco was AAAS president in 1997, so her future antipathy towards SEPS was not in evidence at the time. However, in 2015 she was “outed” by AAAS CEO Rush Holt who forwarded to me several private email messages by others (without their permission).
In preparation for its application for a booth at the 2016 AAAS meeting, and anticipating continued resistance, SEPS individually invited many current and past presidents of scientific societies to endorse our booth application. Lubchenco was invited but never responded. However, 40 others did sign on, including Peter Raven, Paul Ehrlich, and Simon Levin. Rush Holt gave another curt rejection and, when challenged, put out a more detailed but highly falsehood-ridden response on December 12, 2015. He cc’d that to, among others, May Berenbaum, Peter Raven and Simon Levin. Immediately, before I could rebut that response, Levin forwarded Holt’s response to Raven and Ehrlich, with “Rush Holt’s brilliant response” in the subject line. Raven, Ehrlich and Levin all then asked to be removed from the list of endorsers of SEPS application for a booth. Raven insisted that he had been “duped,” and passed his message on to Lubchenco and possibly others. Lubchenco responded, “Peter -- thanks for sharing Rush’s excellent response. It’s good to call a spade a spade! Jane Lubchenco.” Shades of her biased, censorious attitude and actions during the late 1980s! No population booths at AAAS meetings! Away with you!
A fuller account of this failure of the scientific community, with more detailed documentation, is in preparation.
The scientific and academic communities in the U.S. have never, in my long lifetime, been so politicized and censorious – and, in some ways, totalitarian. Alejandro Mayorkas and George Soros can only be ecstatic to have so many strong “let’s not talk about population” allies in the nominally “education community.” They are winning on matters of population policy