arXiv:2605.27025v1 Announce Type: new Abstract: Hate speech annotation is costly, subjective, and prone to annotator disagreement, making large-scale dataset construction challenging. We systematically analyze how well large language models (LLMs) align with human judgments across ten theoretically grounded subjective attributes, such as dehumanization, violence, and sentiment, evaluating both small and large variants of Llama 3.1 and Qwen 2.5. Our analysis reveals a consistent split across all models: behaviorally explicit dimensions (insult, humiliate, attack-defend) correlate strongly with
This research addresses the growing imperative to ensure Large Language Models (LLMs) align with human values as their deployment accelerates, particularly in sensitive areas like content moderation.
Understanding LLM alignment with subjective human judgments, especially regarding harmful content like hate speech, is critical for safe and ethical AI development and widespread adoption.
This research provides a more granular diagnostic tool for evaluating LLM alignment beyond simple classification, identifying specific attributes where models succeed or fail to mimic human judgment.
- · AI ethicists and safety researchers
- · Companies developing content moderation tools
- · Developers of foundational LLMs
- · Platforms with weak content moderation
- · LLMs lacking robust alignment mechanisms
Improved methodologies for assessing and re-training LLMs to better align with human values on subjective topics.
Development of more nuanced and explainable content moderation systems that can articulate why certain content is flagged.
Increased public and regulatory trust in AI systems that demonstrably align with societal norms and ethical standards.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL