Key points are not available for this paper at this time.
As researchers use Large Language Models (LLMs) for rapid manuscript feedback, a key question is whether they can function as reliable peer reviewers in biotechnology. This study tested AI peer review using 763 preprints (398 with open peer reviews) and 12 grant proposals provided by the authors, including three resubmissions. We found that AI reviewers (GPT-5, Qwen-Plus, and Gemini 2.5 Pro) all provided substantive and well-structured comments, with a strong emphasis on experimental design and statistical analysis; however, they tended to be more lenient overall than human reviewers. AI reviewers are less likely than humans to critique paper positioning or ask for more citations. LLMs often rate grant proposals more favorably than humans (i.e. clustering at 3.2-3.8 vs human average 2.5 in scale of 1-4) and have less variation in word choices. AI detectors failed to reliably identify AI-generated text in review comments, as simple rewording bypassed them and detectors usually lagged behind fast-evolving LLMs. Our results suggest that: (1) AI can serve as a valuable and less biased ad hoc reviewer; (2) the use of public LLMs in peer review introduces privacy and copyright concerns; (3) it is important to develop a review agent capable of identifying AI-generated content and verifying that all scientific claims are rigorously evidence-based; and (4) clearer guidelines and sustained human oversight are essential, along with greater transparency through open peer review. Nevertheless, as artificial general intelligence continues to advance, future AI systems may match, or even surpass, human researchers in evaluating scientific manuscripts.
Building similarity graph...
Analyzing shared references across papers
Loading...
Wenyu Li
Runyu Zhao
Pei-Ti Sun
New Biotechnology
Washington University in St. Louis
RAND Corporation
Clayton Foundation
Building similarity graph...
Analyzing shared references across papers
Loading...
Li et al. (Thu,) studied this question.
www.synapsesocial.com/papers/69fd3754cb5f5b5ce35d040b — DOI: https://doi.org/10.1016/j.nbt.2026.03.007
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: