arXiv:2603.20450v2 Announce Type: replace-cross Abstract: A number of scientific conferences and journals have recently enacted policies that prohibit LLM usage by peer reviewers, except for polishing, paraphrasing, and grammar correction of otherwise human-written reviews. But, are these policies enforceable? To answer this question, we assemble a dataset of peer reviews simulating multiple levels of human-AI collaboration, and evaluate five state-of-the-art detectors, including two commercial systems. Our analysis shows that all detectors misclassify a non-trivial fraction of LLM-polished re
The proliferation of LLMs creates an immediate need for policies governing their use, and this research provides timely evidence on the practical enforceability of such policies in critical academic processes.
This highlights the pervasive and often undetectable integration of AI assistance in workflows, challenging governance structures and necessitating new approaches to ensure integrity and fairness.
The ability to reliably detect LLM-assisted ghostwriting for review polishing is significantly undermined, forcing institutions to reconsider their enforcement strategies or accept this untraceable integration.
- · LLM developers
- · Academics using LLMs for review polishing
- · AI detection tool developers (those who improve functionality)
- · Academic journals and conferences
- · Peer review integrity enforcement
- · Current AI detection tool providers
Scientific conferences and journals confront the ineffectiveness of their current LLM usage policies for peer reviews.
This could lead to a shift towards policies that either explicitly permit or implicitly tolerate LLM-assisted polishing, or prompt investment in more sophisticated, albeit still fallible, detection methods.
The blurring line between human and AI writing in critical contexts like peer review might erode trust in academic processes if not addressed with transparency and revised standards.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI