When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals
arXiv:2607.08065v1 Announce Type: new Abstract: LLM-as-judge (Zheng et al., 2023) is increasingly the default for evaluating AI systems in enterprise pipelines, often scaled to ensembles (Verga et al., 2024) or "mixture-of-experts" (Shazeer et al., 2017) panels of judges. These systems share a key assumption: that consistency -- agreement among judges, or among a model's own samples -- indicates correctness. We show this assumption is unreliable. Agreement is not accuracy: a model can agree with itself, and different models can agree with each other, out of shared bias, a memorized heuristic,
The increasing reliance on LLM-as-judge systems in AI development necessitates a rigorous examination of their evaluation methodologies and underlying assumptions.
This research challenges a fundamental assumption in AI evaluation, suggesting that agreement among models or within a single model does not guarantee correctness, which has significant implications for AI safety and reliability.
The default evaluation practices for AI systems, particularly those relying on consistency signals, will need to be re-evaluated and potentially overhauled to ensure true accuracy and reduce inherent biases.
- · AI auditing firms
- · Developers focused on explainable AI
- · Researchers in AI safety
- · Developers solely relying on consistency metrics
- · Companies with biased AI evaluation pipelines
- · AI models exhibiting shared biases
Enterprise AI pipelines using LLM-as-judge will need to incorporate more robust validation methods beyond simple agreement.
Increased demand for novel AI evaluation techniques that can detect and mitigate shared biases and memorized heuristics.
Potential for a new generation of AI models designed for verifiable correctness rather than just statistical consistency.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI