Scientific Fraud is on the rise, and it will get worse (Patreon)

[This is a transcript with links to references.]

Science has a big problem and it’s been getting rapidly worse in the past two years or so, to no small part because of recent advances in artificial intelligence. Fraudulent papers are getting published more than ever, and the fraudsters are getting increasingly aggressive. In this episode I want to give you an update on the recent developments.

According to data collected by Nature magazine the number of retracted papers hit an all time record in 2023 with more ten thousand.  Most of these papers were not retracted because of honest mistakes, but because they contain fabricated crap. Sham data, AI generated text, repurposed figures and images.

The number of retractions is rising faster than the total number of publications. About 2 in a thousand scientific papers is now being retracted. The number of retracted papers isn’t the same the number of fraudulent papers, but it is unlikely that the identification of fraud has suddenly gotten much better. More likely it’s become more difficult as AI gets better. This also means that the number of fraudulent papers has been skyrocketing.

But this number might look more alarming than it is, so let me give you some context. Most retractions happen in Saudia Arabia followed by Pakistan Russia China and Egypt. It is predominantly an Eastern problem. And most of these retractions come from one publisher, Hindawi, and they mostly come from special issues. Special issues have become a special issue in publishing so to speak.

The idea of special issues was that you’d have collections of papers on one particular topic, typically some kind of recent development for which there wasn’t a dedicated journal. This makes sense. The problem is that the editorial process of these special issues was outsourced to “guest editors” who then basically invited their friends to submit papers that were essentially guaranteed to get published. As time went on, special issues became basically junkyards of scam papers that were waved through by those guest editors which were not accountable to anyone or anything.

The journals didn’t do much about it, because you see they sell subscriptions to their content, regardless of what that content is. So researchers were constantly getting spammed with calls to contribute to those special issues, and if you had a need to get some paper published without much effort this was the way to do it.

Now Hindawi is a subsidiary of the publisher Wiley and Wiley has meanwhile recognized the special issue issue. They have announced some major changes and say they’ll stop using the Hindawi brand altogether. I’m not sure that’s going to solve the problem, but this makes me think that the rapid increase in retractions will probably not continue like this.

However, there’s more trouble at the door. A lot of this increase in rubbish publications in driven by what’s become known as “paper mills” in academic publishing. These are semi-legal networks of people who produce scam papers and guide them to publication. They usually do this for academics who pay money. Typically, they’ll be offering authorship on a paper with a particular topic, and the price depends on where you want to be in the author list.

These paper mills are believed to first have been originated in China where academics are often paid for papers or even if not, papers are required for promotions. But the practice has since spread to Russia and India, and reportedly also to Iran and eastern Europe.

A lot of these papers are published in areas that concern public health, such as drug development or psychology. Drug development in particular seems to have been a target because papers on this topic all look and sound more or less the same, you just need to swap out the name of the drug. This makes these paper mills very dangerous because these fake papers can get cited in support of useless drugs, as seems to have happened with the controversial drug Ivermectin .

In the past they have been easy to spot because the language is sometimes funny. This has become known as tortured phrases, probably stemming from automated systems trying to rewrite technical terms. For example in some cases the word “magnetic resonance” became “attractive reverberations”. An article from Times higher education has more funny examples. Fuzzy logic, a research area in mathematics, turned to “fluffy rationale”, breast cancer into “bosom peril”, renal failure became a “kidney disappointment” and an ant colony turned into a “subterranean insect province”.

The most recent worrying trend is that the paper mills evidently make enough money to simply bribe journal editors into accepting papers. Frederik Joelving from the database Retraction Watch recently wrote an article for Science magazine in which he reports an alarming trend. Among recent retractions, those related to bribed editors or other peer review manipulations, such as simply pretending to review a paper with AI generated text, have steeply increased. And this problem doesn’t just affect niche publishers you’ve never heard of. According to Joelving, “A spokesperson for Elsevier said every week its editors are offered cash in return for accepting manuscripts. Sabina Alam, director of publishing ethics and integrity at Taylor & Francis, said bribery attempts have also been directed at journal editors there.”

The problem is ultimately driven by a scientific system that values the quantity of results and publications over the quality.

This is not a new insight of course, but despite it being well known not much has been done to address it. And so I’m afraid that as AI becomes better, fraudulent work will creep into more and more scientific disciplines and become increasingly hard to identify. You know maybe we’re actually better off if we just leave science to AIs entirely.