arXiv:2605.15118v2 Announce Type: replace-cross Abstract: We introduce a reusable framework for auditing whether LLM attack benchmarks collectively cover the threat surface: a 4$\times$6 Target $\times$ Technique matrix grounded in STRIDE, constructed from a 507-leaf taxonomy -- 401 data-populated and 106 threat-model-derived leaves -- of inference-time attacks extracted from 932 arXiv security studies (2023--2026). The matrix enables benchmark-external validation -- auditing collective coverage rather than individual benchmark consistency. Applying it to six public benchmarks reveals that the
The rapid deployment and increasing sophistication of Large Language Models (LLMs) necessitate a robust understanding and categorization of their vulnerabilities to ensure secure development and deployment.
A comprehensive taxonomy of LLM attacks is crucial for developers, security researchers, and policymakers to systematically identify, assess, and mitigate emerging threats to AI systems.
This research provides a standardized framework, the 4x6 Target x Technique matrix, for auditing the coverage of LLM attack benchmarks, moving beyond ad-hoc evaluations to a more systematic security posture.
- · LLM developers
- · Cybersecurity firms
- · AI safety researchers
- · Organizations deploying LLMs
- · Malicious actors
- · Unsecured LLM applications
Security hardening of LLMs will become more systematic and effective due to better-defined threat models.
Reduced incidence of successful attacks against LLMs will increase public and institutional trust in AI applications.
The comprehensive security framework could influence future AI regulations and standards, promoting a more secure AI ecosystem.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL