Registered Reports: a process to safeguard high-quality evidence
- PDF / 494,579 Bytes
- 2 Pages / 595.276 x 790.866 pts Page_size
- 83 Downloads / 233 Views
LETTER TO THE EDITOR
Registered Reports: a process to safeguard high‑quality evidence Anne M. Scheel1 Accepted: 5 November 2020 © Springer Nature Switzerland AG 2020
From the perspective of an individual researcher, Registered Reports may appear as little more than yet another article format—a format with an unusual workflow, perhaps, but ultimately just a slightly different route for one’s research to enter the published record. From the perspective of a scientific claim, however, Registered Reports are much more than that: They establish a new standard for evidence quality. A hypothesis that is upheld in a Registered Report has survived a process that was highly potent at finding any flaws with it. First, the method used to test the hypothesis was vetted in peer review and judged capable of providing an informative test before the results were known. Second, the criteria the data had to pass to be counted as supporting the hypothesis were predefined and left minimal room for the evidence to be presented as stronger than warranted (e.g. due to capitalising on chance). And third, the results would have been published in the same place even if they had contradicted the hypothesis. Compare this to a hypothesis supported in a regular empirical article: Typically, this means that the method only received independent criticism once the results were known to all parties, which may have coloured the reviewers’ judgement; that neither the test criteria nor the hypothesis itself was publicly defined a priori and (knowingly or unknowingly) may have been chosen to fit random patterns in the data; and that contradictory results might never have been submitted for publication or survived the review process. This description is not intended to accuse the research community of habitually misrepresenting their findings and massaging their publication records. Yet the fact of the matter is that our regular publication system puts almost no barriers in the way of such dynamics. Clearly, the hypothesis published in a Registered Report has passed a much more ‘severe’ test, a test that had a high
* Anne M. Scheel [email protected] 1
Human‑Technology Interaction Group, Eindhoven University of Technology, Eindhoven, The Netherlands
probability of refuting the hypothesis if it were false [1, 2]. Because Registered Reports have transparent, resultsindependent selection criteria and strategically placed barriers that prevent authors and editors from getting fooled by their own biases, the evidence for scientific claims published through this route will on average be of far higher quality than that of their counterparts in the regular literature. Critics might object that quality standards for regular articles vary across journals and that some outlets have always required exceptionally high standards of evidence for the scientific claims they publish. This may be true, but taking the perspective of the scientific claim shows where the regular publication system fails: For every supporting piece of evidence published in a high-quali
Data Loading...
