The use of artificial intelligence (AI) tools in peer review has emerged as a controversial topic. With the rise of platforms like ChatGPT, researchers have increasingly turned to AI to evaluate others’ work and modify peer-review reports. In this article, I will provide an analysis of the extent and significance of AI-generated peer-review reports and their implications for scientific publishing, based on an article published on Nature.
First, it is crucial to understand the scale of this phenomenon. Peer review is a pivotal part of the scientific process, ensuring the accuracy and reliability of research findings before they are published. Recent evidence indicates that approximately 17% of peer-review reports may have been substantially affected by AI tools. However, it is essential to determine whether these tools were used to construct complete reviews or simply to edit and enhance existing drafts.
To uncover the extent of AI-generated peer-review reports, researchers employed a novel methodology. By comparing the usage of certain adjectives in peer reviews submitted before and after the release of ChatGPT, they identified a substantial increase in the frequency of positive adjectives, such as ‘commendable,’ ‘innovative,’ and ‘versatile.’
The implications of this trend are concerning. Expectations in scientific publishing assume that human researchers assess the work. However, AI systems have been shown to “hallucinate,” producing misleading or fabricated information. The uncertainty surrounding the involvement of AI in peer review calls into question the trustworthiness and credibility of the scientific literature.
Moreover, legal and ethical considerations arise when discussing AI-generated peer review reports. Debora Weber-Wulff, a renowned computer scientist from the HTW Berlin-University of Applied Sciences, shares this concern. She views the use of AI for peer review as “very shocking” and emphasizes the importance of transparency and accountability in the scientific publishing process.
Additionally, the use of AI in peer review could involve copyright issues. Giving AI access to confidential, unpublished material may infringe upon researchers’ intellectual property rights. It is essential that publishers, researchers, and policymakers consider the implications and safeguards necessary to protect the interests of authors and maintain the integrity of scientific publishing.
Through a collaborative effort, the science community must address the challenges presented by AI-generated peer-review reports. Transparency and accountability should be ensured, and researchers must be educated about ethical implications and best practices. Furthermore, it is imperative to identify and develop robust and reliable detection methods to preserve the credibility and trustworthiness of the scientific literature.