Quality Without Usefulness: LLM-Generated XAI Narratives as Trust Heuristics Rather Than Decision Aids
arXiv:2605.26770v1 Announce Type: new Abstract: Prior work shows that Large Language Models (LLMs) can transform Explainable AI (XAI) outputs into Natural Language Explanations (NLEs) that score highly on quality metrics such as plausibility, coherence, and comprehensibility. But does explanation quality translate to practical usefulness? We investigate this question in a time-series energy forecasting domain through five controlled experiments (2,730 judgments across 60 test instances), each operationalising a distinct facet of usefulness studied in the XAI literature. Holding NLE quality con
The rapid advancement and deployment of Large Language Models (LLMs) are leading to a critical examination of their real-world utility, particularly in sensitive applications like Explainable AI (XAI).
This research highlights a crucial distinction between perceived quality and actual usefulness of AI-generated explanations, potentially impacting trust and adoption of AI systems in critical decision-making.
The focus for AI development and evaluation may shift from merely generating plausible explanations to demonstrating their practical utility as decision aids.
- · AI ethics researchers
- · Explainable AI (XAI) developers focused on utility
- · Auditors of AI systems
- · LLM developers solely focused on quality metrics
- · Users relying on superficial AI explanations
- · AI systems lacking practical explanation utility
AI developers will need to integrate more rigorous usefulness testing into their explanation generation processes.
Increased scrutiny on the actual decision-making impact of AI explanations could slow down the adoption of some AI applications where trust is paramount.
A potential bifurcation in AI explanation approaches, with some optimizing for human perception (trust heuristics) and others for verifiable decision improvement (decision aids).
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL