We show that neural networks with an absolute value activation function and with network path norm, network sizes and network weights having logarithmic dependence on $1/\epsilon$ can $\epsilon$-approximate functions that are analytic on certain regions of ${\mathbb{C}}^d$.

In this paper, it is shown that $$C_\beta $$C β -smooth functions can be approximated by deep neural networks with ReLU activation function and with parameters $$\{0,\pm \frac{1}{2}, \pm 1, 2\}$${ 0 , ± 1 2 , ± 1 , 2 } . The $$l_0$$l 0 and $$l_1$$l 1 parameter norms of considered networks are thus equivalent. The depth, the width and the number of active parameters of the constructed networks have, up to a logarithmic factor, the same dependence on the approximation error as the networks with parameters in $$[-1,1]$$[ - 1 , 1 ] . In particular, this implies that the nonparametric regression estimation with constructed networks achieves, up to logarithmic factors, the same minimax convergence rates as with sparse networks with parameters in $$[-1,1]$$[ - 1 , 1 ] .

Recent studies observed a surprising concept on model test error called the double descent phenomenon where the increasing model complexity decreases the test error first and then the error increases and decreases again. To observe this, we work on a two-layer neural network model with a ReLU activation function designed for binary classification under supervised learning. Our aim is to observe and investigate the mathematical theory behind the double descent behavior of model test error for varying model sizes. We quantify the model size by the ration of number of training samples to the dimension of the model. Due to the complexity of the empirical risk minimization procedure, we use the Convex Gaussian MinMax Theorem to find a suitable candidate for the global training loss.

In this paper we introduce the notions of an analytic curve and equivalent points on the Bohr-Riemann surfaces. By means of constructive and algebraic methods we prove that the points of the Bohr-Riemann surfaces locally have the same number of equivalent points.