On November 8, 2021, the American Heart Association journal Circulation published a 300-word abstract of a research paper warning that mRNA COVID-19 vaccines caused heart inflammation in study subjects. An abstract typically summarises and accompanies the full paper, but this one was published by itself.
According to Altmetrics, the abstract was picked up by 23 news outlets and shared by more than 69,000 Twitter users. On the basis of that abstract, a video on BrandNewTube, a social media outlet that circumvents YouTube’s anti-misinformation policies, pronounced COVID-19 vaccinations “murder.”
Sixteen days later, the American Heart Association added an “expression of concern,” noting that the abstract might not be reliable, and on December 21 it issued a correction that changed the title to indicate that the study did not establish cause and effect, noting there was no control group nor a statistical analysis of the results.
This incident underscores a flaw at the centre of the scientific enterprise. It’s all too easy to make outsize claims that sidestep the process of peer review. No publication should carry a standalone abstract, particularly one making such a bold claim, and particularly during a pandemic. But the problem goes much deeper than that: Even scientific papers that have passed through the intended safeguards of peer review can become vectors for confusion and unsubstantiated claims.
As we’ve seen again and again over the past two years, COVID-19 hasn’t been just a viral pandemic, but also a pandemic of disinformation—what the World Health Organisation calls an “infodemic.” Many scientists blame social media for the proliferation of COVID-related falsehoods, from the suggestion that COVID-19 could be treated by drinking disinfectants to the insistence that masks don’t help prevent transmission. Facebook, Twitter, TikTok, and other platforms have indeed propagated dangerous misinformation. However, social media is a symptom of the problem more than the cause. Misinformation and disinformation often start with scientists themselves.
Facebook and Twitter app icons on a smartphone. Photo: Reuters
Institutions incentivise scientists going for tenure to focus on quantity rather than quality of publications and to exaggerate study results beyond the bounds of rigorous analysis. Scientific journals themselves can boost their revenue when they are more widely read. Thus, some journals may pounce
on submissions with juicy titles that will attract readers. At the same time, many scientific articles contain more jargon than ever, which encourages misinterpretation, political spin, and a declining public trust in the scientific process. Addressing scientific misinformation requires top-down changes to promote accuracy and accessibility, starting with scientists and the scientific publishing process itself.
The history of the scientific journal goes back hundreds of years. In 1731 the Royal Society of Edinburgh launched the first fully peer-reviewed publication, Medical Essays and Observations, initiating what has become the gold standard of credibility: vetting by experts. In the traditional model, scientists conduct original research and write up their findings and methodology, including data, tables, images, and any other relevant information. They submit their article to a journal, whose editors send it to other experts in the field for review. Those peer reviewers evaluate the scientific soundness of the study and advise the journal editors whether to accept it. Editors may also ask authors to revise and resubmit, a process that takes anywhere from weeks to months.
By 2010, most traditional scientific journals also had digital counterparts. The “open access” movement makes roughly 1/3rd of those available to the public for free. Meanwhile, the number of science publications and the number of published papers increased dramatically, and most academic institutions established themselves on social media to help promote the work of their researchers.
In this new world, scientific journals and scientists compete for clicks just like mainstream publications. Articles that are downloaded, read, and shared the most receive a high “impact factor” or Altmetric Attention Score. Studies show that people are more likely to read and share articles with short, positively worded, or emotion-invoking titles.
The rating system can’t help but affect scientists’ publications and their careers. “Many [scientists] are required to achieve certain metrics in order to progress their career, obtain funding, or even keep their jobs,” according to PhD candidate and researcher Benjamin Freeling of the University of Adelaide, who was lead author of a study on the topic, published in the Proceedings of the National Academy of Sciences in 2019.
“There’s less room for a scientist to work on a scientific question of immense importance to humanity if that question won’t lead to a particular quantity of publications and citations,” he wrote in an email to OpenMind. Valuing exposure above the scientific process incentivises sloppy and unethical practices and demonstrates British economist Charles Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.”
University of Washington data scientist Jevin West, who studies the spread of misinformation, says that university public relations offices responsible for press releases and other media interactions “also play a role in the hype machine. Universities want their scientists to win prestigious grants and funding, and to do that, “the research has to be flashy and boundary-pushing.” PR offices may add to that flash by exaggerating the certainty or implications of findings in press releases, which are routinely published almost verbatim in media outlets.
The demand for headline-worthy publications has led to a surge in research studies that can’t be replicated. Results of a reproducibility project published by the Center for Open Science found that only 26% of the top cancer studies published between 2010 and 2021 could be replicated, often because the original papers lacked details about data and methodology. Compounding the problem, researchers cite these nonreplicable studies more often than those that can be replicated, perhaps because they tend to be more sensational and therefore get more clicks.
Most readers, including journalists, can’t discern the quality of the science. Yet it’s “taken forever for the publishing community to provide banners on the original papers” to signal they “might not reach the conclusion readers think,” West says. Tentative or unsubstantiated claims can have profound social impacts. West references a one-paragraph letter written by two physicians and published in the New England Journal of Medicine in 1980, which he regards as largely responsible for the current opioid crisis. The authors asserted that “addiction is rare in patients treated with narcotics,” but they provided no supporting evidence.
It took 37 years before the New England Journal of Medicine added an editorial note warning that the letter had been “heavily and uncritically cited,” but neither a warning nor a retraction can put the misinformation genie back in the bottle, especially given the letter’s decades-long influence on narcotics prescriptions. What’s more, readers can still access misleading studies, and researchers continue to cite them even after they’ve been retracted because they either don’t know about a retraction, or don’t care.
The rise of preprints, scientific papers that have yet to be peer-reviewed, has generated further debate about the proper way to communicate scientific research. Some people celebrate preprints as a way to invite advance feedback and disseminate findings faster. Others argue that so much unvetted material adds to the misinformation glut.
Preprints accounted for roughly 25% of COVID-19–related studies published in 2020. Of those preprints, 29% were cited at least once in mainstream news articles. Take the infamous example of ivermectin, a drug developed for treating parasitic infections. A preprint touting its efficacy in treating COVID-19 patients appeared on the Social Science Research Network (SSRN) server in April 2021, prompting widespread interest in and approval of the drug, including by governments in Bolivia, Brazil, and Peru. As people began taking ivermectin to treat or prevent COVID-19, scientists expressed concern about the data used in the preprint—data supplied by Surgisphere, a health-care analytics company whose unreliable data had previously led to retractions of papers in The Lancet and The New England Journal of Medicine. The paper was removed from SSRN, and shortly thereafter Surgisphere shut down its website and disappeared.
You must be logged in to post a comment.