arXiv:2606.03093v1 Announce Type: new Abstract: Prompting steers large language models (LLMs) and vision-language models (VLMs) without weight updates, but it remains unclear how instruction changes reshape internal representations to produce behavior. We introduce a nested geometric decomposition framework that treats prompting as a transformation of the representational geometry of the content following the prompt. For each prompt pair, we align representations of the same stimuli under two prompts using increasingly expressive stimulus-invariant maps: translation, rigid transformation with
The rapid advancement and widespread adoption of LLMs and VLMs necessitate deeper understanding of their internal mechanisms, especially regarding prompt engineering's influence on behavior.
Understanding how prompting fundamentally reshapes internal representations in LLMs and VLMs is crucial for controlling their behavior and unlocking advanced capabilities, directly impacting AI development and deployment.
This research provides a novel geometric framework to systematically analyze how prompts steer AI models, moving beyond empirical trial-and-error to a more principled understanding of model control.
- · AI researchers
- · Prompt engineers
- · Companies developing LLMs/VLMs
- · Developers of AI agents
- · Empirical prompting approaches without theoretical grounding
Improved prompt design and more robust, predictable AI behaviors are immediate outcomes.
This foundational understanding could lead to new methods for fine-tuning or even dynamically adapting model representations without weight updates.
A deeper grasp of representational transformation might enable more secure and interpretable AI systems, fostering greater trust and wider adoption in sensitive applications.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI