arXiv:2605.26878v1 Announce Type: new Abstract: Multi-stakeholder tasks require one output to satisfy users with conflicting preferences. Holistic LLM judges conflate utility estimation and utility aggregation, yielding unstable implicit weights. We show empirically and theoretically that this aggregation-specific \emph{weighting noise} can create large score shifts when stakeholder satisfaction is dispersed; in our experiments, these weight-induced shifts also increase with stakeholder count. We propose \textsc{DecompR}: counterfactual-calibrated weights are fixed from query structure before
The proliferation of powerful LLMs and their deployment in multi-stakeholder environments necessitates robust alignment mechanisms, with current approaches proving insufficient for complex preference landscapes.
This research offers a methodical approach to address a fundamental challenge in LLM development: how to consistently satisfy diverse user groups without introducing bias through ad-hoc weighting.
The proposed 'DecompR' method separates utility estimation from aggregation, potentially leading to more stable and transparent LLM alignment, especially in scenarios with conflicting user preferences.
- · AI developers
- · Organizations deploying multi-stakeholder AI systems
- · Users of complex AI applications
- · Developers relying on ad-hoc LLM evaluation methods
- · Systems with implicit and unstable weighting mechanisms
More reliable and fairer outcomes from LLM-powered decision-making in diverse user groups.
Increased trust and adoption of AI systems in critical applications requiring multi-stakeholder consensus.
New frameworks for AI governance and regulatory compliance emerge, built on transparent and decomposable alignment methodologies.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI