Bregman meets L\'evy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time
arXiv:2606.03769v1 Announce Type: cross Abstract: We study the robustness of stochastic mirror descent (SMD) under heavy-tailed noise, focusing on whether the method retains its convergence guarantees when run with infinite-variance stochastic gradient input. To address this question in a principled manner, we begin by introducing a continuous-time model of SMD as a stochastic differential equation (SDE) driven by a centered L\'evy noise process with finite $p$-th order moments, $1 < p \leq 2$. This scheme -- which we call the L\'evy mirror flow (LMF) -- arises naturally as the scaling limit o
The paper addresses a fundamental robustness question in stochastic optimization, a key component of modern AI systems, driven by increasing real-world deployments where noise might be heavy-tailed.
Improving the theoretical understanding and robustness of stochastic mirror descent, particularly with heavy-tailed noise, is critical for developing more reliable and performance-guaranteed AI algorithms in uncertain environments.
This research provides a principled and theoretically grounded approach to handling infinite-variance noise in AI optimization, potentially leading to more stable and efficient AI models in real-world scenarios.
- · AI researchers
- · Machine learning developers
- · Sectors using AI in noisy environments
- · Optimisation software providers
- · AI systems vulnerable to noisy data
Increased theoretical understanding of AI algorithm robustness under challenging noise conditions.
Development of new, more resilient stochastic optimization algorithms adopted in various AI applications.
These more robust algorithms could enable AI deployment in even more unpredictable and data-scarce environments.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG