Geometric Monomial (GEM): a family of rational 2N-differentiable activation functions

arXiv:2604.21677v1 Announce Type: cross Abstract: The choice of activation function plays a crucial role in the optimization and performance of deep neural networks. While the Rectified Linear Unit (ReLU) remains the dominant choice due to its simplicity and effectiveness, its lack of smoothness...