The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment
arXiv:2607.05552v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly issue judgments read as binary verdicts, and a growing literature reports such judgments shifting under logically irrelevant changes of wording - among them an amplified yes-no bias on moral dilemmas, absent in humans. A single framing cannot say what such a shift is: in a yes/no question the word "no" is at once logical verdict, lexical token, and last-printed option. We introduce a psychometric battery that separates these: crossed symmetrization - every logically irrelevant factor flipped in balanced
The proliferation of LLMs in decision-making contexts and growing public scrutiny over their biases necessitates deeper understanding of their underlying mechanisms.
Understanding LLM biases, particularly how superficial factors influence their 'judgments,' is crucial for developing reliable and ethically sound AI systems, impacting their deployment in critical applications.
This research reframes the 'yes-no bias' in LLMs from a moral judgment issue to one driven by answer order and wording, requiring a re-evaluation of how such biases are diagnosed and mitigated.
- · AI ethicists
- · LLM developers
- · AI testing platforms
- · Organizations deploying unverified LLMs for binary decisions
- · Simplistic interpretations of LLM moral judgment
Immediate adjustments in prompt engineering and fine-tuning strategies for LLMs used in sensitive decision-making.
Development of standardized psychometric batteries for LLMs to rigorously test and debias their responses across various linguistic and structural variations.
Increased regulatory focus on requiring comprehensive bias testing and transparency for LLMs before commercial deployment in high-stakes domains.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI