"I Strongly Suspect This Website Is a Scam": Benchmarking PII Leakage and Detection without Defense in Autonomous Web Agents
arXiv:2606.00497v1 Announce Type: cross Abstract: Deceptive web content, widely instantiated across the internet and commonly known as \textit{social-engineering attacks}, manipulates autonomous web agents into submitting users' personally identifiable information (PII) to attacker-controlled endpoints. In this paper, we show that social-engineering attacks are highly effective at extracting critical-tier PII from frontier web agents, posing a severe risk to deployed agentic systems. To quantify this risk, we introduce \textbf{\textsc{Scammer4U}}, a pre-registered benchmark of 91 attacker-cont
The proliferation of advanced autonomous web agents, alongside increasingly sophisticated social engineering tactics, creates an urgent need to assess their vulnerability to PII leakage.
This research provides critical new data on a fundamental security risk for AI agents, indicating a severe vulnerability that could undermine trust and widespread deployment of agentic systems.
The understanding of AI agent security shifts from theoretical concerns to empirically validated vulnerabilities, necessitating immediate defense mechanisms and a re-evaluation of current deployment strategies.
- · Cybersecurity firms
- · AI safety researchers
- · Developers of defensive AI agents
- · Companies deploying frontier web agents
- · Users relying on agent privacy
- · General-purpose AI agents without robust defenses
Companies will accelerate efforts to integrate robust PII protection into their autonomous agent deployments.
New regulatory frameworks may emerge, specifically addressing the security and data privacy implications of AI agents.
A competitive market for AI agent security and auditing tools will rapidly develop, influencing the design and adoption of future agentic AI.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL