Haraux Type Activation Functions in Neural Network Theory
Nasser-eddine Tatar *
Department of Mathematics and Statistics, King Fahd University of Petroleum and Minerals, Dhahran, 31261, Saudi Arabia.
*Author to whom correspondence should be addressed.
Abstract
A general system of ordinary differential equations that appear in neural network theory is studied. We consider nonlinear activation functions which have not been treated in the literature so far. Namely, we assume that the nonlinear activation functions are continuous functions but not necessarily Lipschitz continuous.
Keywords: Neural network, activation function, Lipschitz continuous, exponential convergence, global existence.