WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
arXiv:2606.09426v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114 tasks across 8 real-world work domains, grounded in real user requests and publicly verifiable artifacts. Each task requires agents to combine GUI obse
The proliferation of Computer-Use Agents necessitates more robust and real-world oriented benchmarks to accurately assess their capabilities across diverse, integrated environments.
This benchmark addresses a critical gap in evaluating autonomous agents' real-world utility by focusing on complex, multi-interface tasks, directly impacting the development and deployment of truly capable AI agents.
The introduction of WeaveBench provides a standardized, challenging framework for assessing Computer-Use Agents, shifting evaluation from isolated functionalities to integrated, long-horizon real-world performance.
- · AI agent developers
- · Companies adopting AI agents
- · Benchmark creators
- · Developers of narrow AI agents
- · Companies relying on simplistic agent evaluations
Improved evaluation leads to more capable and reliable AI agents for complex computer-based tasks.
Accelerated development of general-purpose AI agents capable of orchestrating across multiple software and interfaces.
Increased automation of white-collar workflows and a restructuring of professional services as agents handle more sophisticated tasks.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI