arXiv:2606.05402v1 Announce Type: new Abstract: Large reasoning models (LRMs) produce reasoning traces with non-linear structures, such as backtracking and self-correction, that complicate the evaluation and monitoring of the reasoning process. We introduce ReasoningFlow, a framework that captures the discourse structures of LRM reasoning traces into fine-grained directed acyclic graphs (DAGs). We develop and validate our annotation schema through careful manual annotation of 31 traces (2.1k steps), achieving high inter-annotator agreement, then scale to automatic annotation of 1,260 traces (2
The increasing complexity of LLM outputs, especially in reasoning tasks, necessitates tools for better understanding and evaluation, driving innovation in tracing and analysis.
This framework offers a method to systematically analyze how LLMs arrive at conclusions, which is crucial for improving their reliability, trustworthiness, and debugging capabilities in complex applications.
The ability to capture and analyze the 'discourse structures' of LLM reasoning transforms how developers and researchers can understand, debug, and optimize large reasoning models.
- · AI researchers
- · LLM developers
- · AI safety and ethics organizations
- · Companies deploying complex AI systems
- · Developers relying solely on black-box LLM evaluation
- · Inefficient LLM development methodologies
Improved debugging and interpretability of large language models, leading to more robust AI applications.
Accelerated development of even more complex and autonomous AI systems as their internal processes become more transparent.
Potential for new regulatory frameworks for AI that require demonstrable understanding of model reasoning paths.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL