Reforming the ‘scoop’ system that hurts science

Science is the best way in society to understand the world, but many people in the field are dissatisfied with the way it works. Rules and procedures aimed at innovative research can promote devastating side effects that harm scientists and scientists. One of these – the ‘rule of precedence’ – rewards scientists who discover with better reputations, rewards and career opportunities, taking away the second prize of such. Researchers at Eindhoven University of Technology (TU / e) and Arizona State University in the US have developed a new model to better understand this rule, and see if conventional reforms to improve the system make sense . Their study was published in Human Behavior Nature.

“Over the past decade, there have been growing concerns that there is something” ugly “in the state of science,” said Leonid Tiokhin, a researcher at the Human Technology Interaction group at TU. / e and the lead author of the paper. “Scientists understand that many rules and procedures are dictated by historical norms and precedents, rather than principled reasons that serve efficiency and reliability. Worse, there is growing evidence that some practices have an adverse effect that can hurt both scientists and scientists. “

A well-known example is the general preference for positive rather than negative results. If journals publish only positive results, this creates a ‘publication bias’, where the same published results do not show a correlation or cause, leaving out negative results which is equally relevant.

Another practice that can do more harm than good is valuing search priority, which rewards a scientist who is first to publish their results with unparalleled reputation, rewards and career opportunities. .

“Many scientists have sleepless nights worrying about being‘ scooped ’- for fear that their work will not be considered a‘ novel ’enough for most major scientific journals. victory, “said Tiokhin. The rule of precedence has been around for centuries. In the 17th century, Newton and Leibniz defeated the inventor of calculus. And in the 19th century, Charles Darwin chased the publication of his work on evolution through natural selection so that Alfred Russel Wallace could not be taken out.

“The priority of evaluation is easy to understand and there are some benefits. However, it comes at a cost,” Tiokhiin says, “Rewards for priority can attract scientists to sacrifice the quality of their research and cut corners.”

“This is partly why some academic publishers, such as PLOS and eLife, have decided to offer ‘scoop protection’, allowing researchers to publish results that are similar to those published as- yes within a specified time. The problem is that we are not good yet. wonder if these reforms make sense. “

To determine the impact of award prioritization (and whether recent reforms offer any solution for potential benefits) Tiokhin and his colleagues developed a ‘model agent-based evolution ’. This computer model simulates how a group of scientists study or omit research questions, depending on factors such as the type of results (positive or negative) and the innovation of the research (cost of being scooped ‘).

The scientists have been portrayed as “representatives” who are more likely to advance their careers if they do research in a way that rewards them sufficiently (such as money or fame). By changing the way science was structured and what types of outcomes were most rewarding, the researchers were able to study how representatives changed their behavior over time and what the impact was. which these changes had on science as a whole.

The researchers found that a culture that rewards a priority – where there is a high cost to build it – can have a detrimental effect. Among other things, he encourages scientists to conduct ‘quick and dirty’ studies, so that they can be published first. This reduces the quality of their work and undermines the overall reliability of science.

The model also suggests that some form of scoop protection, as introduced by PLOS and eLife, works. “It reduces the hassle of tearing up the research and gives researchers more time to gather more data,” Tiokhin said. “However, we should keep in mind that panacea is not a defense. ‘in scoop protection. “

This is because scoop protection encourages some scientists to continue with a line of research even after publishing several results, which reduces the total number of interesting questions studied. The model also shows that, even with scoop protection, scientists can be attracted to run many small studies if the costs of starting a new project (start-up costs) are low and the benefits for negative results are high. .

“So while scoop defense reforms are helpful, they are not enough to encourage high-quality research or reliable published literature,” according to Tiokhin. “Our results suggest that we should also consider incurring some start-up costs, such as requiring scientists to pre-register their studies or criticize their research plans before they begin to data collection. ”

“We also learned that inefficiency in science is not always a bad thing. On the other hand: inefficiency forces researchers to think twice before embarking on a new study.”

Other solutions that may correct the adverse effects of the priority rule are to reduce the inefficiency of data collection (to make it easier to collect large amounts of data) and to reward the quality of analysis (to support studies ’with support statistically avoided).

Tiokhin has a personal purpose for his work, called ‘meta-research’, the use of the scientific method to study research itself. “When I was doing my PhD research in the US, I was overwhelmed by the way science works. Like a lot of researchers, I was very thoughtful when I got into But it was not long before I discovered that research is not often about discovering the nature of reality, but about playing by the rules of the game. means publishing your work in the “right” magazines, and would be better off with results that are new and statistically significant. ”

“The problem is that I don’t like playing with arbitrary rules. With my research, I want to understand what the rules are for recognizing and rewarding scientists who makes sense and generates better results for science as a whole. “

###

Leonid Tiokhin, Minhua Yan, Thomas JH Morgan, Competition for priority will undermine the reliability of science but reforms can help, Human Behavior Nature (DOI: 10.1038 / s41562-020-01040-1)