New research from the University of Georgia suggests most people don’t understand the difference between a preprint and a published academic journal article.
Preprints are research papers that haven’t undergone peer review, the process by which studies’ findings are validated by experts who weren’t involved with the research themselves.
The study found the majority of readers have little to no understanding of what a preprint actually is. That lack of understanding could lead to public distrust in science since findings and how those findings are described can change between the preprint phase and publication following peer review. Frequent reporting of scientific preprints could also hurt trust in news.
If people are basing attitudes, for example, about a new drug on evidence from a preprint … they should be able to have a sense of its preliminary nature.” —Chelsea Ratcliff, Franklin College of Arts and Sciences
Preprints used to mainly circulate within scientific communities, but the COVID-19 pandemic led to unprecedented numbers of preprints flooding the internet.
The desire to get information out as quickly as possible was understandable, the researchers said. But it also sets a problematic precedent.
“With preprints, there are still uncertainties that haven’t been ironed out,” said Chelsea Ratcliff, lead author of the study and an assistant professor in the Franklin College of Arts and Sciences department of communication studies. “A lot of preprints never even get published. I really think it’s important that the public understands that.
“If people are basing attitudes, for example, about a new drug on evidence from a preprint or if they’re basing health decisions on a preprint, they should be able to have a sense of its preliminary nature.”
75% of people don’t really know what a preprint is
The researchers assigned 415 people to read news articles about COVID-19 preprint research. The preprint focused on the relationship between COVID-19 vaccine side effects and vaccine efficacy.
One group read stories where the study was described as “a preprint study recently posted online and not yet evaluated by outside experts,” while the other version simply described it as a “study.”
The news article also either presented the conclusions of the research tentatively, such as saying the findings “suggest” or “could” mean that people are protected from COVID-19 regardless of whether they experienced side effects from the vaccines, or portrayed them as being certain.
[Preprints] weren’t meant to influence public policy, attitudes or behaviors. And that’s something worth keeping in mind for readers.” —Chelsea Ratcliff
Unsurprisingly, participants did rate the findings as less certain when the story said the findings were tentative. But using the word “preprint” in the text and mentioning that the research hadn’t undergone peer review had no effect on participants’ interpretation of the study. Both groups rated the certainty of the research the same.
When the researchers asked participants to describe what they think “preprint” means when it appears in a science news story, 75% gave a definition that showed they didn’t really understand the concept.
Chelsea Ratcliff
“What I tell my students is to think about any one study as just a drop in the bucket of knowledge about a phenomenon,” Ratcliff said. “No single study proves or disproves anything, and we may need an extra degree of caution when it’s a preprint study. I see value in preprints, but just telling the public that it’s a preprint is not enough to give them that sense that it’s preliminary evidence.”
“We need to find other strategies for communicating about preprints effectively,” added Alice Fleerackers, co-author of the study and a doctoral candidate at Simon Fraser University. “Simply labeling research as a ‘preprint’—even with a brief definition—doesn’t seem to move the needle.”
Journalists who report on preprint studies should briefly explain the process of academic peer review and warn readers that the findings of preprints are subject to change, the researchers said.
“Before the pandemic, which is when reporting on preprints really ballooned, the main purpose of a preprint was for scientists to share their findings with other scientists,” Ratcliff said. “They weren’t meant to influence public policy, attitudes or behaviors. And that’s something worth keeping in mind for readers.”
Published in Health Communication, the study was co-authored by Rebekah Wicke, Blue Harvill, Andy King and Jakob Jensen.
