In Annals of Internal Medicine, Kieran Quinn et al. ask, how can we re-think how we disseminate medical research? Can we expand the status and role of preprint archives? And see their Citation 8, Liang et al., in NEJM-AI. They empirically study how LLM contributes to peer review, and the percentage of reviewers who are favorable is high. 80% of reviewers said LLM review was more helpful than at least some of the available human reviews.
###
AI CORNER
Summary of Quinn et al., 2025 (“On a Slow Boat to Publication”)
Quinn and colleagues argue that traditional medical publishing is inefficient and outdated, with median publication times exceeding six months due to sequential journal submissions and peer review delays. They note that peer review consumes an estimated 15,000 person-years of unpaid labor annually, yet often yields minimal changes from preprints.
The authors advocate for broader use of preprint servers (e.g., medRxiv) to disseminate findings quickly and equitably, eliminating paywalls and high article processing fees. They propose that success in academia should shift away from citation counts toward measures of impact, such as data sharing, policy uptake, and trainee development. Citing Liang et al. (2024), they highlight how AI tools could streamline peer review, producing feedback comparable in quality to humans while enhancing speed and accessibility
Summary of Liang et al., 2024 (“Can Large Language Models Provide Useful Feedback on Research Papers?”)
Liang and co-authors conducted the first large-scale empirical study of GPT-4-generated scientific feedback, analyzing over 4,800 papers from Nature family journals and the ICLR conference.
They found that GPT-4’s feedback overlapped with human reviewer comments at rates (≈30–39%) comparable to overlap between two human reviewers, indicating similar reliability. In user surveys of 308 researchers, 57% found AI feedback helpful, and more than 80% said it was at least as beneficial as some human reviews.
GPT-4 tended to highlight major methodological issues and offer non-generic, paper-specific critiques, though it lacked domain-specific depth. The authors conclude that LLMs can complement, not replace, human peer review by providing rapid, early-stage feedback to authors
Conclusion
Together, Quinn et al. and Liang et al. envision a transformation of scholarly publishing: preprints and AI-assisted feedback could accelerate knowledge sharing, democratize access, and reduce the inefficiencies of traditional peer review. While human expertise remains indispensable, integrating LLM-based feedback systems could help researchers—especially those in under-resourced settings—improve their work more quickly and make scientific communication more open and effective.
