AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue
arXiv:2602.17443v2 Announce Type: replace Abstract: Multi-turn LLM evaluation is typically reported as a single win-rate scalar, conflating distinct capabilities. We introduce AIDG (Adversarial Information Deduction Game), formalizing multi-turn adversarial dialogue as a two-player partially observable stochastic game (POSG) and decomposing performance along Seeker (extraction) and Holder (containment) roles. The decomposition isolates three failure modes: cooperative-prior leakage, constraint-reasoning interference, and inefficient hypothesis-space traversal. Across 439 games over six frontie
The rapid advancement and deployment of multi-turn LLMs necessitate more robust and granular evaluation methods beyond single scalar metrics.
Improved evaluation frameworks for LLMs are critical for understanding their capabilities and limitations, especially as they become more complex and integrated into advanced applications like AI agents.
This formal decomposition provides a structured way to assess LLM performance in adversarial dialogue, moving beyond simplistic win-rates to identify specific failure modes in information extraction and containment.
- · LLM developers
- · AI safety researchers
- · Organizations deploying LLM agents
- · Undifferentiated LLMs
- · LLM evaluation benchmarks based on single metrics
More sophisticated and secure multi-turn LLM agent designs will emerge as developers can pinpoint and address specific vulnerabilities.
The ability to formally decompose LLM performance will accelerate progress in building robust and trustworthy autonomous AI systems.
This could lead to new standards for AI agent certification based on provable information-theoretic security and reasoning capabilities.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL