arXiv:2605.29850v1 Announce Type: new Abstract: Recent progress in task-optimized neural networks has established encoding models as a powerful tool for predicting brain responses to naturalistic stimuli, yet most existing approaches rely on unimodal representations. The emergence of omni-modal foundation models and rich multimodal neural datasets enables encoding models that jointly integrate visual, auditory, and linguistic information across subjects. We introduce MIRAGE, a brain encoding framework for predicting whole-brain fMRI responses to naturalistic audiovisual stimuli. MIRAGE achieve
The proliferation of advanced neural networks, particularly omni-modal foundation models, and increasingly rich multimodal neural datasets facilitates more sophisticated brain encoding approaches, enabling this development now.
This research represents a significant advancement in understanding and predicting brain responses to complex naturalistic stimuli, moving beyond unimodal approaches to integrate various sensory inputs.
The ability to predict whole-brain fMRI responses using multimodal models will lead to more nuanced brain-computer interfaces and potentially new diagnostic methods, shifting how we study and interact with the brain.
- · Neuroscience researchers
- · AI/ML developers
- · Brain-computer interface companies
- · Cognitive science
- · Unimodal brain encoding model developers
Improved accuracy and resolution in neural decoding and encoding tasks.
Accelerated development of assistive technologies that directly interpret complex brain states.
Ethical and privacy debates intensify regarding the interpretation and potential manipulation of detailed brain activity.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG