Activation Function Design Sustains Plasticity in Continual Learning

arXiv:2509.22562v4 Announce Type: replace-cross Abstract: In independent, identically distributed (i.i.d.) training regimes, activation functions have been benchmarked extensively, and their differences often shrink once model size and optimization are tuned. In continual learning, however, the...