The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers
arXiv:2606.16974v1 Announce Type: new Abstract: The reproducibility crisis has directed the AI research community toward improving documentation practices. Several studies have identified methodological issues, and in response, the most impactful venues in the field have introduced reproducibility checklists. We seek to understand whether documentation practices have changed over time by assessing all published papers at five leading AI conferences over the past decade. Seven reproducibility variables were identified, quality-assured and used to analyse 56 800 publications. Our analysis reveal
The AI research community, facing a reproducibility crisis, is actively improving documentation practices, leading to a decade-long analysis of conference papers to understand policy effectiveness.
Improved reproducibility in AI research enhances the trustworthiness and reliability of AI systems, impacting their real-world application and adoption across industries.
The systematic analysis of AI conference papers over a decade provides empirical evidence of shifts in documentation practices, influencing future research methodologies and policies.
- · AI research community
- · AI ethics and safety organizations
- · Developers of AI standards
- · Industries adopting AI
- · Research that lacks transparency
- · Fringe AI research venues
AI research becomes more verifiable and robust, accelerating legitimate progress and mitigating unfounded claims.
Increased trust in AI outputs could lead to faster integration of advanced AI models into critical infrastructure and decision-making systems.
Standardized reproducibility practices could foster greater international collaboration in AI development, potentially leading to global benchmarks and shared knowledge bases.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI