Natural gradient descent with momentum

arXiv:2604.15554v1 Announce Type: cross Abstract: We consider the problem of approximating a function by an element of a nonlinear manifold which admits a differentiable parametrization, typical examples being neural networks with differentiable activation functions or tensor networks. Natural...