arXiv:2605.03202v2 Announce Type: replace Abstract: Large language models offer a tempting solution to address the peer review crisis. This position paper argues that today's AI systems should not be used to produce paper reviews. We ground this position in an empirical comparison of human- versus AI-generated ICLR 2026 reviews and an evaluation of the effect of automated paper rewriting on different AI reviewers. We identify two critical issues: 1) AI reviewers exhibit a hivemind effect of excessive agreement within and across papers that reduces perspective diversity. 2) AI review scores are
The proliferation of large language models is leading to widespread attempts to automate various processes, including academic peer review, making rigorous evaluation of their efficacy critical right now.
The integrity of scientific communication and the quality of academic research are foundational, and their erosion by poorly implemented AI automation could have long-term societal consequences.
This paper establishes empirical evidence against the immediate widespread deployment of AI for academic peer review, potentially slowing or redefining its integration in scientific publishing.
- · Human peer reviewers
- · Academic journals focused on quality control
- · Researchers prioritizing review integrity
- · Platforms rushing to automate peer review
- · AI developers overstating current capabilities
- · Academic publishing facing 'review crisis'
The call for rigorous evaluation will likely lead to increased scrutiny and slower adoption of AI in critical academic processes.
This could spur the development of more nuanced AI agents capable of diverse perspectives and less 'hivemind' behavior, specifically for complex tasks like qualitative assessment.
Delayed or careful AI integration in peer review might preserve the human element in scientific gatekeeping, subtly influencing the future structure of academic institutions and the validation of knowledge.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI