Speech-only spoken language models (SLMs) lag behind text and text-speech models in performance, with recent discrete autoregressive (AR) SLMs indicating significant computational and data demands to match text models. Since discretizing continuous speech for AR creates bottlenecks, we explore whether continuous diffusion (CD) SLM is more viable. To quantify the SLMs linguistic quality, we introduce the phoneme Jensen-Shannon divergence (pJSD) metric. Our analysis reveals CD SLMs, mirroring AR behavior, exhibit scaling laws for validation loss and pJSD, and show optimal token-to-parameter…
The continuous exploration of foundational models for AI, especially in modalities other than text, drives research into more efficient architectures and scaling properties for speech. Apple, a major player in speech-enabled devices, is naturally pushing this frontier.
Improving speech-only models to match text model performance is critical for ubiquitous, natural language interfaces and agentic systems, especially in environments where text input is impractical or impossible. This research could broaden the accessibility and utility of AI significantly.
The potential viability of continuous diffusion models for spoken language, with their favorable scaling properties, suggests a new pathway to overcome current bottlenecks in speech AI development. This could lead to a significant uplift in speech AI capabilities.
- · Apple
- · Speech AI developers
- · Voice assistant sectors
- · AI hardware manufacturers
- · Companies relying purely on discrete AR SLMs
- · Legacy speech recognition companies
More accurate and efficient speech-to-text and speech generation systems become feasible, reducing computational overhead for similar performance levels.
Enhanced speech AI could accelerate the development of more capable and fluid AI agents, as natural voice interaction becomes a primary input/output modality.
Widespread adoption of highly advanced, low-latency speech AI could fundamentally reshape human-computer interaction, making interfaces invisible and ubiquitous, potentially diminishing the role of keyboards and screens in many use cases.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at Apple Machine Learning Research