Opening the Black Box of Scientific Peer Review: Preliminary Results from the New Frontier

Background: Peer review is foundational to the conduct of science. However, a number of recent episodes of failure, misconduct, or at the very least, problematic peer review mechanisms have contributed to calls for a reconsideration of the rigor and quality of the process. Members of PEERE (New Frontiers in Peer Review) is a COST (CO-operation in Science and Technology) Action, a pan-European platform for networking, are using the COST Network to study peer review processes to improve efficiency, transparency and accountability of peer review through a trans-disciplinary, cross-sectorial collaboration. Methods: The objectives of this Action are: (i) to analyse peer review in different scientific areas by integrating quantitative and qualitative research and incorporating recent experimental and computational findings; (ii) to evaluate implications of different models of peer review and to explore new incentive structures, rules and measures to improve collaboration in all stages of the peer review process; (iii) to involve science stakeholders in data sharing and testing initiatives, (iv) to define collaboratively a joint research agenda that points to an evidence-based peer review reform. Results: Since the goals of the Action are to bring together researchers for networking rather than being a specific research project, the poster will point to some of the results of specific projects conducted by PEERE members and how they are meeting the objectives of the Action.  Projects highlighted including a network analysis of scholarly literature on peer review, an agent-based model of referee behavior, the evidence based for evaluating the success or failure of review, and analysis of peer review across disciplines. Conclusion: Our goals are twofold: to introduce PEERE to the scientific community more broadly and gain more perspective on peer review as a practice worthy of study. Since the project is ongoing, we describe our efforts to define collaboratively (with researchers, editors, and publishers) a joint research agenda that uses multiple methods and disciplines to open up the black box of peer review and move to an evidence-based peer review reform. Not only can a better peer review system improve the self- regulation processes of science to benefit all science stakeholders, it can also increase the social recognition and credibility of science in Europe and globally.