arXiv:2509.20784v3 Announce Type: replace-cross Abstract: The fundamental representational units (FRUs) of large language models (LLMs) remain undefined, limiting further understanding of their underlying mechanisms. In this paper, we introduce Atom Theory to systematically define, evaluate, and identify such FRUs, which we term atoms. Building on the atomic inner product (AIP), a non-Euclidean metric that captures the underlying geometry of LLM representations, we formally define atoms and propose two key criteria for ideal atoms: faithfulness ($R^2$) and stability ($q^*$). We further prove t
The rapid development and widespread adoption of LLMs necessitate deeper understanding of their internal workings to improve performance, safety, and efficiency.
Defining fundamental representational units could unlock new architectures, training methods, and explainability for LLMs, accelerating AI progress.
This research introduces a novel theoretical framework and methodology ('Atom Theory') for dissecting LLM representations, potentially moving beyond black-box analysis.
- · AI researchers
- · Model developers
- · Explainable AI sector
- · AI development relying solely on empirical scaling
- · Companies with opaque proprietary models
Systematic understanding of LLM 'atoms' leads to more predictable and controllable model behavior.
New architectural paradigms emerge, potentially allowing for more efficient, smaller, or specialized LLMs.
Deeper insight into AI representations could inform the development of more general artificial intelligence or novel forms of computation.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI