arXiv:2607.05238v1 Announce Type: new Abstract: JEPA world models predict the next latent state with a single deterministic predictor trained by latent regression. We show that this fails structurally when the environment is stochastic: at a branching transition, the regression-optimal predictor outputs the conditional mean of the successor embeddings, a point between the true next states that corresponds to no state at all. We prove this collapse for deterministic and gated mixture-of-experts predictors, and prove that MoP-JEPA's hard-assigned predictors converge instead to a quantizer of the
The paper addresses a fundamental limitation in current JEPA (Joint Embedding Predictive Architecture) world models, offering a solution that improves their ability to handle stochastic and complex environments.
Improving the predictive capabilities of world models, especially in stochastic environments, is crucial for advancing AI systems towards more robust and generalizable intelligence, impacting future AI development.
This paper proposes a method (MoP-JEPA) that allows JEPA models to more accurately predict future states in uncertain environments, moving beyond the limitation of averaging potential outcomes.
- · AI researchers
- · Developers of predictive AI models
- · Industries relying on advanced AI simulation
- · Prior deterministic JEPA prediction methods
More accurate and robust AI world models emerge, particularly for tasks in unpredictable settings.
This advancement could accelerate the development of more capable AI agents that can plan and act effectively in complex, real-world scenarios.
Improved world models might unlock new applications for AI in fields like robotics, autonomous systems, and scientific discovery where understanding stochasticity is vital.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI