Addressing the “Replication Crisis” in the Field of Parkinson’s Disease

An ongoing phenomenon known as the “replication crisis” – the systematic failure to attempt to, let alone actually, reproduce published study results – poses a significant concern in scientific research. Here we highlight the extent and pervasiveness of the replication crisis across various scientific disciplines as an impetus for the Parkinson’s disease (PD) research community to recognize the need for replication studies to examine reproducibility.

The term replication crisis was initially coined in 2012 to describe issues observed in psychology and medicine studies [1]. However, subsequent reports have indicated that this crisis extends to the natural and social sciences as well. Many research studies are never being replicated, in part, because it is often not even tried. Replicating earlier findings is generally less attractive to researchers – and indeed to society – than delivering novel research discoveries. When attempts are actually being made to replicate earlier work, these are often not successful. A survey conducted by the journal Nature, involving 1,576 researchers, revealed the widespread nature of this phenomenon. Specifically, this survey found that 70% of these researchers attempted and failed to replicate the data from other scientists’ studies, including 87% of chemists, 77% of biologists, 69% of physicists and engineers, 67% of medical researchers, 64% of earth and environmental scientists, and 62% from other fields [2]. Additionally, more than half of the researchers failed to replicate their own experiments. In the field of cancer research, “the Reproducibility Project: Cancer Biology” was initiated in 2021 to examine 53 cancer papers published between 2010 and 2012 in high-impact factor journals. The project demonstrated that even when results were successfully replicated, the effect size was approximately 85% smaller on average compared to the original papers [3]. It is important to note that the replication crisis is not limited to academic research alone. The pharmaceutical company Bayer attempted to replicate in-house research studies and achieved a success rate of only 25% [4]. Similarly, a paper published by Amgen in 2012 reported that only 11% of the 53 pre-clinical cancer studies examined could be successfully reproduced [5]. Formal studies examining the reproducibility of PD research are currently needed, but the field is certainly not immune to the replication crisis.

Multiple causes have been proposed as responsible for this crisis. The survey of researchers conducted by Nature revealed that the two most common factors are the pressure to publish and selective reporting. Poor oversight, insufficient peer review, and statistical mistakes were also reported among several others causes [2]. For example, insufficiently powered studies or the misuse of data analysis to obtain statistically significant results, such as “p-hacking”, are widely diffused in the scientific community [6]. Conversely, the minority of researchers referred to fraud as a possible explanation in this survey [2].

In response to the replication crisis, numerous strategies have been implemented to promote rigorous, standardized, and transparent scientific practices. One notable development is the emergence of the discipline known as “metascience.” Metascience utilizes the scientific method to investigate the practice of science itself, analyzing everyday scientific processes to identify potential pitfalls and opportunities for improvement [7]. Several additional approaches have been adopted to foster more rigor in science. These include pre-registration of scientific studies and clinical trials, which involve registering study protocols and hypotheses in advance to mitigate issues like selective reporting. The availability of open data and repositories has become increasingly prevalent, facilitating data sharing (so other research groups can analyze the same data set) and promoting transparency. Result-blind peer review, in which reviewers are unaware of the authors’ identities or previous findings, helps reduce bias and enhances the objectivity of evaluations. Efforts to address statistical misuse, such as promoting appropriate statistical analysis and interpretation, further contribute to research integrity. Finally, meticulous description of the materials and methods employed in research studies as well as involvement of the original authors might help ensure clarity and facilitate reproducibility.

To address the replication crisis and promote scientific integrity, the Journal of Parkinson’s Disease is introducing a new manuscript submission category called “Replication Studies.” This initiative aligns with the commitment of other journals (e.g., Royal Society Open Science) to prioritize replication studies. This category welcomes methodologically-sound attempts to replicate previously published reports in PD research. Further information for authors will be provided within the submission guidelines on the journal’s website. The replication crisis has started to reshape the perception of replication studies, transforming them from being perceived as unexciting and lacking novelty to being recognized as essential works that are equally important and informative as any new study published [8, 9]. By establishing this new category, the Journal of Parkinson’s Disease aims to encourage researchers to conduct and submit replication studies, acknowledging their equal importance and contribution to advancing knowledge. At the heart of this initiative is the goal of fostering a culture of rigorous scientific inquiry and enhancing the overall quality of research in our field.