arXiv:2606.09705v1 Announce Type: new Abstract: Scientific generative modeling often requires size transfer, where models trained on small systems are evaluated on larger ones. While translation-invariant architectures enable this evaluation, we show that architectural locality alone does not guarantee stable size extrapolation. Instead, stable extrapolation is governed by the quasi-locality of the Gaussian-smoothed score. Through Tweedie's formula, far-away perturbations can influence local score components via posterior covariance, meaning a local model succeeds only if its receptive field c
This paper addresses fundamental limitations in AI model generalization, an escalating concern as models become larger and more complex, and highlights a critical gap in current architectural assumptions.
It provides a theoretical framework and diagnostic tool for understanding and improving the extrapolation capabilities of AI models, which is crucial for applications requiring deployment across varying scales.
Our understanding of what makes AI models reliably scale to unseen data sizes shifts from simple architectural locality to a more nuanced concept of Gaussian-smoothed score quasi-locality.
- · AI researchers focused on foundational models
- · Developers of large-scale scientific generative models
- · Industries requiring AI deployment across diverse system sizes
- · Developers relying solely on architectural locality for generalization
- · AI projects with insufficient diagnostic testing for scale extrapolation
It will likely lead to new architectural designs and training methodologies for more robust size extrapolation in AI models.
Improved model extrapolation could accelerate progress in scientific discovery, enabling AI to predict properties across vastly different scales in domains like materials science or drug discovery.
More reliable scaling could broaden the practical applicability of AI agents in dynamic, real-world environments where system sizes are highly variable, impacting sectors like robotics or autonomous systems.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG