SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech
arXiv:2606.06037v2 Announce Type: cross Abstract: Large audio language models (LALMs) are increasingly deployed in real-world applications, yet their safety alignment is still primarily evaluated on monolingual, text-based harmful prompts. This leaves their generalizability under multilingual and spoken settings, particularly code-switched speech, largely underexplored. To address this gap, we introduce SpeechJBB, an audio jailbreak dataset for benchmarking across multiple state-of-the-art LALMs. The extent of safety weaknesses is further probed by introducing an augmented setting where phonol
The increasing deployment of Large Audio Language Models (LALMs) in real-world applications highlights an urgent need for robust safety evaluations beyond text-based and monolingual benchmarks.
This research introduces a critical method for probing safety weaknesses in LALMs under more realistic and complex conditions, particularly code-switched speech, which is crucial for preventing harmful outputs and ensuring reliable AI deployment.
The introduction of SpeechJBB allows for more comprehensive and realistic safety alignment evaluations of LALMs, moving beyond their current limitations and exposing vulnerabilities previously overlooked.
- · AI safety researchers
- · LALM developers focused on multilingual capabilities
- · Users of secure LALM applications
- · LALM developers with inadequate safety protocols
- · Monolingual LALM evaluation methods
- · Applications vulnerable to audio-based jailbreaks
LALM developers will need to update their safety alignment methodologies to account for multilingual and code-switched audio inputs.
Improved LALM safety in diverse linguistic contexts will increase user trust and accelerate global adoption of audio AI technologies.
The development of more sophisticated audio jailbreak datasets could spur an 'arms race' between AI red-teaming and safety engineering.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL