arXiv:2606.31112v1 Announce Type: new Abstract: ASR systems have been often reported to underperform on atypical speech. An often conflated compounding factor is the existence of two valid transcription references: verbatim (actual produced speech, including repetitions/prolongations) and intended (the canonical form of the text with disfluencies removed) in atypical speech recognition depending on context and use-case. Most ASR evaluations conflate this duality into a single ground truth and reward systems that delete disfluencies, ignoring verbatim faithfulness. We benchmark 11 ASR models fr
The increasing sophistication and proliferation of ASR systems in various applications necessitate more precise and nuanced evaluation methods, particularly for challenging speech. This research addresses a fundamental limitation in current ASR benchmarking, aligning with the ongoing drive for robust AI performance.
Improving ASR evaluation metrics for atypical speech is critical for the reliable deployment of voice AI in critical applications, especially those involving human-computer interaction, healthcare, and accessibility, where disfluencies are common. It also impacts the fairness and effectiveness of AI systems across diverse user populations.
This research introduces a more refined approach to evaluating ASR systems by distinguishing between verbatim and intended transcription, allowing for context-dependent performance assessment. It enables developers to build more context-aware and robust ASR models that can handle natural human speech variations more effectively.
- · AI developers
- · Speech recognition companies
- · AI researchers
- · Users of voice AI
- · ASR systems with poor atypical speech handling
- · Benchmarks relying solely on single-ground truth metrics
ASR systems will be developed and evaluated with more sophisticated metrics, leading to improved performance on natural and atypical speech.
Enhanced ASR accuracy will improve the reliability and user experience of voice interfaces across various industries, from customer service to medical dictation.
More robust ASR could accelerate the adoption of voice-based human-AI interaction models, potentially influencing the design of future AI agents and interfaces.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL