This article describes a novel sufficient condition concerning approximations with reservoir computing (RC). Recently, RC using a physical system as the reservoir has attracted attention. Because many physical systems are modeled as state-space systems, it is necessary to guarantee the approximations given by reservoirs represented as nonlinear state-space systems. There are two problems with existing approaches: a reservoir must have a property called fading memory and must be represented as a set of maps between input and output signals on the bi-infinite-time (BIT) interval. These two conditions are too strict for reservoirs represented as nonlinear state-space systems as they require the reservoir to have a unique equilibrium state for the zero input. This article proposes an approach that employs operators from right-infinite-time (RIT) inputs to RIT outputs. Furthermore, we develop a novel extension of the Stone–Weierstrass theorem to handle discontinuous functions. To apply the extended theorem, we define functionals corresponding to operators and introduce a metric on the domain of the functionals. The resulting sufficient condition does not require the reservoir to have fading memory or continuity with respect to inputs and time. Therefore, our result guarantees the approximations with very common reservoirs and provides a rationale for physical RC. We present an example of a physical reservoir without fading memory. With the example reservoir, the RC model successfully approximates NARMA10, a benchmark task for time series predictions.
We discuss necessary and sufficient conditions for universal approximation using reservoir computing. Reservoir computing is a machine learning method used to train a dynamical system model by tuning only the static part of the model. The universality is the ability of the model to approximate any dynamical system with any precision. In the previous studies, we provided two sufficient conditions for the universality. We employed the universality definition that has been discussed since the earliest studies on reservoir computing. In this present paper, we prove that these two conditions and the universality are equivalent to one another. Using this equivalence, we show that a universal model must have a “pathological” property that can only be achieved or approached by chaotic reservoirs.