arXiv:2606.08251v1 Announce Type: cross Abstract: Bold projections that artificial intelligence will accelerate scientific discovery have raced ahead of evidence from working scientists, and the field still lacks large-scale, scientist-in-the-loop tests of these claims. Here we mount the largest such evaluation to date and map what AI cannot yet do for science. We invited authors of 121,640 recent preprints across biology, medicine, chemistry, and the social sciences to judge follow-up ideas that large language models (LLMs) generated from the context and puzzles of their own papers. 6,749 sci
The proliferation of increasingly capable large language models has led to a surge in unverified claims about AI's potential in scientific discovery, necessitating concrete evaluation.
This research provides critical empirical evidence challenging widespread assumptions about AI's current capabilities in scientific imagination and divergence, tempering expectations and guiding future development.
The perception of AI's immediate role in independent scientific ideation shifts from expansive to constrained, highlighting current limitations in abstract reasoning and novel hypothesis generation.
- · Human scientists
- · AI ethics researchers
- · AI systems focused on augmentation
- · Over-optimistic AI developers
- · Investors in 'fully autonomous science AI'
- · Hyperscalers promoting 'AI will solve everything'
More realistic timelines and expectations will be set for AI's integration into scientific discovery workflows.
Funding and research efforts may pivot towards developing AI that enhances human creativity and problem-solving, rather than replacing it.
A deeper understanding of human cognitive processes in scientific imagination might emerge from studying where current AI falls short.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI