arXiv:2605.29152v1 Announce Type: new Abstract: Randomly initialized neural networks induce a prior over functions, but the predictor used in practice is produced only after training. We ask how much of this initial bias survives the training pipeline. To make the question measurable, we introduce initialization memory: the dependence of the validation-selected predictor on the scale of the random initialization. We perform controlled CIFAR-10 experiments on ResNets where initialization memory already sharply separates training regimes. Low-learning-rate SGD can interpolate while still remembe
This research is emerging as deep learning models become increasingly complex and foundational to AI development, making the understanding of their inductive biases during training critical.
Understanding how initial biases persist or are forgotten during training helps in developing more robust, predictable, and controllable AI systems, impacting their reliability and safety.
The methods for evaluating and understanding the impact of initialization on the final performance and behavior of deep neural networks are becoming more refined and measurable.
- · AI researchers
- · ML framework developers
- · AI safety researchers
- · Companies deploying critical AI models
- · Developers relying solely on 'black box' deep learning
- · Those overly optimistic about generalization without understanding bias
Improved understanding of deep learning training dynamics leads to better model design and optimization strategies.
More reliable AI systems reduce deployment risks and expand AI application into sensitive domains.
The ability to predictably control inductive bias could enable more efficient and specialized AI hardware development.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.LG