arXiv:2607.02672v1 Announce Type: new Abstract: Local pairwise comparisons are a standard tool for learning how people want decision rules to work, e.g., in participatory design or alignment. However, their use builds in two strong assumptions: that local comparisons are sufficient evidence about how a person wants an automated decision rule to behave, and that people can always answer those comparisons decisively. We investigate how these assumptions may be compromised under internal pluralism: the idea that an individual evaluates decision rules according to multiple authoritative priorities
The increasing deployment of AI in complex decision-making systems highlights the need to understand how human preferences are incorporated and interpreted, particularly as AI systems become more autonomous.
This research directly addresses a fundamental challenge in AI alignment and participatory design, questioning the reliability of current methods for eliciting human values and preferences for AI systems.
Understanding the limitations of pairwise comparisons due to 'internal pluralism' will lead to the development of more nuanced and robust methods for AI alignment and value learning, potentially slowing current development models.
- · AI ethics researchers
- · Human-computer interaction specialists
- · Developers of advanced AI alignment techniques
- · Designers relying solely on simple pairwise comparisons
- · AI systems developed without robust human preference models
- · Rapid deployment of AI in sensitive areas
AI development methodologies will need to incorporate more sophisticated techniques for understanding complex human preferences beyond simple binary choices.
This could lead to slower AI development cycles initially, as more rigorous alignment processes are implemented to account for internal pluralism.
Long-term, this could result in more trustworthy and adaptable AI systems that better reflect diverse human values, potentially increasing public acceptance and reducing unintended consequences.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI