Learning Rate Engineering: From Coarse Single Parameter to Layered Evolution

arXiv:2604.27295v1 Announce Type: new Abstract: Learning rate scheduling has evolved from the single global fixed rate of early SGD to sophisticated layer-wise adaptive strategies. We systematize this evolution into five generations: (Gen1) global fixed learning rates, (Gen2) global scheduling,...