From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment
arXiv:2606.05180v1 Announce Type: new Abstract: Automated scoring models are increasingly used to assign rubric-based quality ratings to complex language performances, including classroom transcripts, yet they typically provide little insight into why a particular score is produced. We propose a general framework for sentence-level interpretability of rubric-based scoring that combines model-agnostic Shapley-value attributions with rationales generated by large language models (LLMs). Instantiated on the Quality of Feedback dimension of the CLASS framework using the NCTE corpus, the framework
The increasing sophistication and deployment of AI models in complex assessment tasks necessitates robust methods for explaining their outputs.
This work addresses a critical transparency and explainability gap in automated AI scoring, particularly for critical applications like educational assessment.
The ability to generate clear, sentence-level explanations for AI-driven assessments will increase trust, enable better model debugging, and facilitate human oversight of automated systems.
- · AI developers
- · Educators
- · Students
- · Assessment platforms
- · Opaque black-box AI systems
Automated scoring models will become more transparent and trustworthy, allowing for wider adoption in sensitive domains.
The integration of explainable AI (XAI) will become a standard requirement for AI systems deployed in high-stakes assessment and decision-making.
This could lead to a shift in how educational feedback is generated and consumed, moving towards AI-assisted personalized explanations.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL