arXiv:2606.29495v1 Announce Type: new Abstract: Social influence dialogue changes user behavior by altering internal cognitive states. The central evaluation question is whether the user's beliefs, desires, intentions, and emotions measurably change over the course of conversation, a process-oriented criterion that neither surface-level text metrics (BLEU/ROUGE) nor single-score LLM judgments can capture. We propose the \textbf{Cog}nitive \textbf{W}orld \textbf{M}odel \textbf{(CogWM)}, an LLM-based user model that reframes multi-turn dialogue evaluation from ``what did the user say'' to ``how
The proliferation of advanced LLMs necessitates more sophisticated evaluation metrics beyond superficial text analysis, prompting the development of models that assess deeper cognitive impacts.
Evaluating real social influence of AI systems by assessing changes in internal cognitive states provides a critical tool for safety, ethics, and effective human-AI interaction in critical applications.
The focus of dialogue evaluation shifts from surface-level linguistic metrics to a process-oriented understanding of how AI alters beliefs, desires, intentions, and emotions in users.
- · AI Safety Researchers
- · Social Science Researchers
- · LLM Developers focused on persuasion/influence
- · Ethical AI Frameworks
- · Developers relying solely on BLEU/ROUGE
- · Systems lacking internal user models
- · Unregulated persuasive AI
More accurate and nuanced assessment of AI's genuine impact on human cognitive states will become possible.
This improved understanding could lead to the development of AI designed to be more ethically compelling or more resistant to manipulation.
Regulation around AI's persuasive capabilities may evolve to mandate such cognitive influence evaluations, leading to new compliance requirements for AI systems.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI