Peer review in the 21st century
Abstract
This article reviews new requirements for peer review in scholarly publishing. New models include open, transparent, and post-publication peer review. These have emerged in response to increasing expectations around peer review. Notable other issues are reviewer recognition and process efficiency.
1.Tradition and new requirements in peer review
Peer review enables journal editors to select the best content. Key issues are the validity, significance, and originality of the research results written up for publication. All key participants draw benefits from peer review:
– editors are aided in their decision what to publish;
– authors improve their manuscripts;
– peer reviewers have first access to new results.
Yet, expectations towards peer review have shifted and grown. A general critique is that peer review is a slow process that includes too much subjective bias. More specifically, the lack of formal training for reviewers is noted. Peer review is also described as burdensome and inefficient for reviewers and authors alike. The debate on research ethics and the reproducibility of research has shown that peer review can be open to abuse and that plagiarism and fraud are not necessarily detected. Also, there is a lack of credit for the task of reviewing.
Publishers and the scholarly community are addressing these issues. In what follows, I would like to, firstly, present significant new initiatives to change peer review. Secondly, I will turn to efforts for improving the efficiency of the process. Thirdly, I will look at how reviewers are getting more recognition and support.
2.New models
Traditionally in biology and medicine, peer review has been single-blind, that is the reviewer is anonymous. Double-blind peer review, more commonly used in the humanities and social sciences, has become increasingly common in science, where the authors of the manuscript remain anonymous too.
Initiatives focused on changing and improving peer review have developed three new models:
1. Open peer review with signed reports that are available with the published article, where a response by the author may be included.
2. Transparent peer review by which the unsigned review reports are made available alongside the published article, and a response by the author may be included.
3. Post-publication peer review, which takes place only after publication and is usually fully open.
2.1.Open peer review
The key requirement of open peer review is that the authors and reviewers are aware of each other’s identities, and that the reports and any interaction between reviewers and authors is visible to the readers. ‘Open’ may extend to the publishing platform and include open contributions by the community reviewing or commenting on the article.
BioMed Central has been conducting open peer review since the early 2000s, and has 70 journals publishing the full publication history including reviews. This effort has enhanced the transparency and integrity of the process – except for the drawback that it can be harder to recruit reviewers. Some notable other examples from the industry are The EMBO journal, the PeerJ platform, and BMJ Open.
2.2.Transparent peer review
The key requirement of transparent peer review is that the review file is published alongside the accepted article. The file contains the reports by the reviewers, as well as any rebuttal by the author(s), and may also include the reviews and rebuttals of any resubmission. This opens the editorial decision-making process for scrutiny.
For example, Nature Communications introduced this type of peer review in January 2016. Data shows that on average 60% of authors opt to have the review file published alongside the article. This practice is adopted more frequently by authors in the life sciences than in chemistry and physics.
2.3.Post publication peer review
The key requirement of post-publication peer review is that a first editorial decision is made to publish the manuscript, and that the peer review process follows. The established tradition of giving papers at seminars and conferences is a precedent, albeit one designed to enhance papers before submitting them for formal peer review. This model principally accelerates publication, but also means that the author must be confident, since any rejection of the paper would be publicly visible.
Notable examples include Copernicus Publications, especially the journal Atmospheric Physics and Chemistry (with Discussions), as well as ScienceOpen or F1000 Research.
3.More efficiency?
Technical peer review focuses on determining if the submitted article is sound in method and results. Validity is prioritized and significance or originality is not a key criterion influencing the decision to publish or not. The objective is to accelerate publishing and lighten the responsibilities and workload of the reviewers. A notable example is PLoS One, which has nevertheless published articles with significant impact.
Cascading peer review is based on the principle of reviewing a submitted manuscript only once (if possible) and offering to the author(s) a suitable publication venue in a tiered structure. The Nature group of journals have been a notable example. The objective is also to accelerate publishing and reduce the review burden.
Results-free review means that in a first step the paper is evaluated only for its rationale and method, not the results. If the former is deemed suitable for publication, then this is offered in principle. In a second step, the results are reviewed too. Publication may only be rejected if the results deviate unjustifiably from the stated aims and methods. The objective of results-free review is to tackle bias in the reviewer process that results from looking at the results and their potential significance first and foremost.
More experiments are under way that include, for example, making peer review independent of the journal and providing it as a service; or automating parts of the peer review with the support of artificial intelligence, e.g. pre-checking manuscripts for authorship, permissions and plagiarism. Early evidence shows that machines pick up more detail than human reviewers.
4.More recognition and training?
Publishers seek to provide recognition and rewards for reviewers, which range from thank you notes to free subscriptions. Recently, more systematic efforts are made at mentoring and training reviewers. A mentoring pilot at BMC Medicine is set to determine if structured feedback from a mentor improves the quality of peer review reports produced.
More generally, recognizing and rewarding peer reviewers has become a priority for scholarly societies, publishers, and service providers. For example, societies publish lists of the most prolific and helpful reviewers; publishers give public credit and provide additional rewards; and service providers enable the collection of data on reviews and reviewers to enhance reviewer visibility and rewards. Further still, Publons is a start-up dedicated to publicly recognizing reviewers for their contribution, enabling reviewers to track and showcase their activities.
Overall, peer review has seen much experimentation around the issue of transparency, and much improvement in terms of efficiency and recognition.
