arXiv:2605.28721v1 Announce Type: new Abstract: Are LLM-based search agents genuinely searching, or using the web to verify what they already know? We study this question on BrowseComp with three diagnostics. Our analysis reveals Intrinsic Knowledge Dependence (IKD): even with tool access, agents often rely on intrinsic knowledge -- information encoded in the model before retrieval -- rather than on external evidence. Agents answer up to 44.5% of BrowseComp questions without tools, generate more than half of their search queries from internally produced hypotheses rather than retrieved leads,
The proliferation of LLM-based search agents necessitates understanding their true operational mechanisms to improve their effectiveness and reliability.
This study challenges the assumption that AI agents primarily rely on external web data, highlighting a significant dependence on internal knowledge, which impacts their utility and development.
The understanding of how AI agents perform search tasks shifts, indicating a need to rethink agent design, training data, and evaluation metrics.
- · AI model developers focused on knowledge grounding
- · Companies specializing in AI agent evaluation tools
- · Researchers developing advanced retrieval-augmented generation techniques
- · AI agent developers relying solely on external search for intelligence
- · Users expecting agents to always leverage the most current external data
- · Platforms providing only basic web search APIs for agents
AI agent architectures will be redesigned to more explicitly manage the interaction between internal knowledge and external retrieval.
There will be increased investment in developing better mechanisms for agents to identify and mitigate their 'Intrinsic Knowledge Dependence' (IKD).
The perceived value and trustworthiness of AI agents for critical information retrieval tasks may be temporarily lowered until these issues are addressed.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI