arXiv:2509.24882v2 Announce Type: replace Abstract: Neural scaling laws underlie many of the recent advances in deep learning, yet their theoretical understanding remains largely confined to linear models. In this work, we present a systematic analysis of scaling laws for quadratic and diagonal neural networks in the feature learning regime. Leveraging connections with matrix compressed sensing and LASSO, we derive a detailed phase diagram for the scaling exponents of the excess risk as a function of sample complexity and weight decay. This analysis uncovers crossovers between distinct scaling
This research is published as the AI field continues to push the boundaries of model scale, making theoretical understanding of scaling laws critically important for future development.
Understanding the theoretical underpinnings of neural network scaling laws allows for more efficient and predictable development of large AI models, reducing trial-and-error and improving performance.
This work potentially transitions the understanding of neural network scaling from empirical observation to more rigorous theoretical frameworks, particularly for specific network architectures.
- · AI researchers
- · Deep learning practitioners
- · Companies investing in AI model development
- · Teams using purely empirical scaling methods
- · AI development relying on inefficient resource allocation
Improved understanding and more efficient design of future neural networks.
Faster progress in developing larger, more capable AI models due to better theoretical guidance.
Reduced computational costs for achieving desired performance levels in AI, potentially broadening access to advanced AI development.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG