From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models
arXiv:2606.20152v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have substantially transformed Automated Essay Scoring (AES), yet the internal mechanisms underlying LLM-based scoring remain poorly understood. In this work, we systematically analyze the hidden representations of eight LLMs across two English essay datasets (ASAP++, CSEE) and one Portuguese dataset (ENEM). Using linear probing, cross-prompt generalization, dimensionality reduction, and neuron-level analyses, we find consistent evidence that essay quality information is encoded in a linearly access
The rapid advancement and widespread adoption of Large Language Models necessitate a deeper understanding of their internal mechanisms for responsible and effective application.
Understanding how LLMs encode essay quality is crucial for improving automated essay scoring systems, ensuring fairness, and guiding future AI development in educational and evaluative contexts.
This research moves automated essay scoring beyond black-box output, providing insights into the cognitive processes within LLMs when evaluating text quality.
- · Educational technology companies
- · AI researchers in interpretability
- · Developers of automated assessment tools
- · Companies relying on opaque AI evaluation systems
- · Traditional essay grading services (potentially, long-term)
Improved, more transparent, and trustworthy Automated Essay Scoring (AES) systems will emerge.
The interpretability methods developed could be applied to other LLM applications, leading to broader advancements in explainable AI.
Enhanced AI understanding of text quality could fundamentally alter how large-scale content creation and evaluation are performed, impacting industries from publishing to legal documentation.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL