Trajectory Dynamics in Language Model Hidden States Predict Human Processing Costs Beyond Surprisal
arXiv:2606.05346v1 Announce Type: new Abstract: Human language comprehension unfolds sequentially: each word is processed in the context of those that came before, and the interpretation builds incrementally over time. Surprisal, the negative log probability of a word given its context, has been the dominant predictor of incremental processing cost. But surprisal reduces rich sequential representations to a single scalar at each word, discarding information about the direction in which the interpretation has been evolving. Dynamical-systems approaches suggest that the trajectory of the evolvin
The proliferation of advanced language models makes understanding their internal workings and aligning them with human cognitive processes critically important for further development and application.
This research provides a more nuanced understanding of how language models process information, potentially leading to more efficient, human-like, and robust AI systems.
The framework for evaluating and developing language models now has an alternative to surprisal, focusing on the dynamic trajectory of hidden states.
- · AI Researchers
- · NLP Developers
- · Cognitive Scientists
- · Generative AI Platforms
- · Models reliant solely on surprisal metrics
Increased interpretability and explainability of large language models through trajectory analysis.
Development of new language model architectures explicitly designed to optimize for trajectory dynamics rather than just surprisal.
More sophisticated human-computer interaction based on AI's ability to better anticipate and adapt to human cognitive processing paths.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.CL