Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer

概要

arXiv:2605.07870v1 Announce Type: cross Abstract: We study the evolution of hidden-weight spectra in wide neural networks trained by (stochastic) gradient descent. We develop a two-level dynamical mean-field theory (DMFT) that jointly tracks bulk and outlier spectral dynamics for spiked ensembles w…