Sponsoring Transparency and Open Science: The Case for Open Peer Review
By: Joseph A. Smith, Jr., MD; D. Robert Siemens, MD, FRCSC | Posted on: 01 Dec 2021
In October 2021, my editorial, “Determination of the Worthiness for Publication,” was published in The Journal of Urology®.1 This editorial served as an announcement that The Journal would begin to embrace the transparent concept of open peer review. In collaboration with my successor, Dr. Robert Siemens, who begins his term in January 2022, and with the support of our Editorial Board, we are pleased to announce exactly how we will move forward with this initiative.
Historically, The Journal has used “single-anonymous” peer review. This model obscures the identity of a submission’s peer reviewers to authors; however, the reviewers have access to the identities of authors. Although single-anonymous peer review is used widely across scholarly publishing, its critics argue it is subject to bias. Because reviewers are able to see a submission’s authors and institutions, opponents of single-anonymous peer review believe that it is difficult for the peer review process to remain impartial and without prejudice.
One alternative to single-anonymous peer review is double-anonymous peer review. This model conceals the identities of both authors and peer reviewers from one another. The benefits of double-anonymous peer review are that any unfairness and preconceptions a reviewer might have about an author and/or institution are removed. However, double-anonymous peer review is very challenging from a practical perspective. Editorial office members must read each submission carefully to remove identifying information from a manuscript. Often, such details are missed, despite thorough checks. We want to equip our reviewers with all relevant information, including data about the provenance of submissions. Reviewers are keenly aware of the potential for bias but do a great job of attempting objective commentary and recommendations.
After careful consideration, Dr. Siemens and I reached the conclusion that open peer review is the right decision for The Journal of Urology®. There are multiple levels of what constitutes truly “open” peer review, so with the support of our Editorial Board, we have committed to working toward the ideal of fully open peer review with the following guidelines:
- We will only publish peer review reports for accepted, published articles.
- We will publish reviewer comments on decisions for accepted papers only, along with the authors’ responses. Comments and feedback shared by reviewers with the Editor will not be published for any articles. Editorial comments and names of the reviewers writing them will continue to be published at the end of articles, exactly as they are now.
- We will offer reviewers an “opt-out” button, which will allow their name to be obscured from the published peer review report. However, we will still publish their comments anonymously.
- All reviewer names will be presented in aggregate, along with the full peer review report. No specific reviews will be connected to specific reviewers.
- All of the above will be explained in a clear and straightforward manner in our reviewer invitation letters.
Recognizing that our volunteer peer reviewers are the backbone of The Journal’s success, we thank them for their steadfast commitment. At the same time, we want The Journal to be a leader in medical publishing while providing a window into the peer review process and how decisions are made in accepting an article.
We have valued the thoughts and comments of the community as we announced this initiative. As we work toward fully open peer review, where we offer transparency regarding the entire decision process, we realize that we must take small steps as we move toward that ultimate goal. In hearing the respectful opinions on this change from our Editorial Board, and following along with personal conversations and feedback we have received on social media and in writing, we believe that the principle of what we are trying to accomplish will be embraced by the urology community. However, we also see that we must move slowly so that we can alleviate the well-founded concerns that reviewers, especially those who are junior and/or international volunteers, will be protected when their peer review comments are shared publicly.
We are not instituting this policy to check a box. Rather, we intend to keep the conversation around open peer review fluid and ongoing. This process will evolve over time, and we encourage continued feedback from each of you. We will continuously check in on progress, measuring such metrics as reviewer declines, number of reviewers who opt out of sharing their names on published peer review reports and anecdotal reactions from our reviewers to this initiative. We will also send followup emails to our reviewers to survey them to determine whether the quality of their reviews and/or the thought processes around them were affected by open peer review. These findings will be reported periodically as editorials in The Journal; furthermore, it is expected that the guidelines will be tweaked and updated frequently over time.
The Journal of Urology® has been a leader in adopting and launching new initiatives. Modification of content, improving visual presentation and incorporation of innovative social media concepts have all been introduced and remain ongoing. Moving toward more transparency in the review process will provide an educational opportunity for readers and, once fully understood and implemented, improve the quality of reviews and discourse with authors.
- Smith JA Jr: Determination of the worthiness for publication. J Urol 2021; 206 802.