Patterns vs. Patients: Evaluating LLMs against Mental Health Professionals on Personality Disorder Diagnosis through First-Person Narratives
arXiv:2512.20298v4 Announce Type: replace-cross Abstract: Growing reliance on LLMs for psychiatric self-assessment raises questions about their ability to interpret qualitative patient narratives. This depth over breadth case study directly compares state-of-the-art LLMs and mental health professionals in assessing Borderline (BPD) and Narcissistic (NPD) Personality Disorders based on Polish-language first-person autobiographical accounts. Within our sample, the overall diagnostic scores of the top-performing Gemini Pro models (65.48%) were 21.91 percentage points higher than the average score
The proliferation of advanced LLMs and their growing public access makes evaluating their capabilities in sensitive applications like mental health diagnosis crucial and timely.
This research provides direct evidence of LLMs' surprising performance in complex subjective evaluations, challenging assumptions about human-AI comparative advantages in psychological assessment.
The perceived boundary of AI's diagnostic capabilities is expanded, suggesting LLMs could play a significant, potentially primary, role in initial mental health assessments.
- · AI developers
- · Healthcare technology companies
- · Patients seeking accessible self-assessment tools
- · Traditional psychiatric diagnostic models
- · Mental health professionals focusing on initial screening
LLMs begin to be integrated into mental health screening platforms for initial patient intake and analysis.
Ethical and regulatory frameworks for AI-driven mental health diagnosis accelerate development to address accuracy, bias, and patient safety concerns.
The role of human mental health professionals shifts towards complex cases, therapeutic interventions, and oversight of AI systems rather than initial diagnostic assessment.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI