Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents
arXiv:2606.22528v2 Announce Type: replace Abstract: Modern LLM agents increasingly rely on context compaction, summarization, or eviction to keep long-running sessions within a token budget. We show that this context-management layer is a safety-critical failure surface: in-context governance constraints that agents reliably obey while visible can be silently removed by compaction, causing the same agent to perform prohibited tool actions later in the session. We call this failure mode Governance Decay. We introduce ConstraintRot, a benchmark of long-horizon agent scenarios with deterministic
The increasing reliance on long-running LLM agent sessions and context management techniques brings this critical security vulnerability to the forefront. This research highlights a systemic flaw in widely adopted LLM agent architectures as they scale in complexity and duration.
This research uncovers a fundamental flaw in LLM agent safety, revealing that essential governance constraints can be silently erased, leading to prohibited actions and significant security risks. It demands immediate re-evaluation of agent design and deployment for any organization utilizing or developing advanced AI agents.
The understanding of LLM agent robustness and security shifts, necessitating new approaches to context management, constraint enforcement, and auditing to prevent 'Governance Decay.' Developers must now actively design against silent constraint removal rather than assuming persistent safety policies.
- · AI safety researchers
- · Security auditing firms
- · Developers of new context management techniques
- · Developers of current LLM agent frameworks
- · Organizations deploying agents without robust safety mechanisms
There will be a push for more resilient and auditable context management layers within LLM agents.
New industry standards or best practices for ensuring persistent safety constraints in long-horizon AI agents may emerge.
Legal and regulatory frameworks might begin to address liability for autonomous agent actions resulting from 'Governance Decay.'
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI