Delay-based reservoir computing has gained a lot of attention due to the relative simplicity with which this concept can be implemented in hardware. However, unnecessary constraints are commonly placed on the relationship between the delay-time and the input clock-cycle, which can have a detrimental effect on the performance. We review the existing literature on this subject and introduce the concept of delay-based reservoir computing in a manner that demonstrates that no predefined relationship between the delay-time and the input clock-cycle is required for this computing concept to work. Choosing the delay-times independent of the input clock-cycle, one gains an important degree of freedom. Consequently, we discuss ways to improve the computing performance of a reservoir formed by delay-coupled oscillators and show the impact of delay-time tuning in such systems.
In the reservoir computing literature, the information processing capacity is frequently used to characterize the computing capabilities of a reservoir. However, it remains unclear how the information processing capacity connects to the performance on specific tasks. We demonstrate on a set of standard benchmark tasks that the total information processing capacity correlates poorly with task specific performance. Further, we derive an expression for the normalized mean square error of a task as a weighted function of the individual information processing capacities. Mathematically, the derivation requires the task to have the same input distribution as used to calculate the information processing capacities. We test our method on a range of tasks that violate this requirement and find good qualitative agreement between the predicted and the actual errors as long as the task input sequences do not have long autocorrelation times. Our method offers deeper insight into the principles governing reservoir computing performance. It also increases the utility of the evaluation of information processing capacities, which are typically defined on i.i.d. input, even if specific tasks deliver inputs stemming from different distributions. Moreover, it offers the possibility of reducing the experimental cost of optimizing physical reservoirs, such as those implemented in photonic systems.
18 November 2024 in Computers & Chemical Engineering
Multiscale simulations utilizing high-fidelity, microscopic Monte Carlo models to provide the nonlinear response for continuum models can easily become computationally intractable. Surrogate models for the high-fidelity Monte Carlo models can overcome this but come with some challenges. One such challenges arise by the sampling noise in the underlying Monte Carlo data, which leads to uncontrolled errors possibly corrupting the surrogate even though it would be highly accurate in the case of noise-free data. Another challenge arises by the 'curse of dimensionality' when the response depends on many macro-variables. These points are addressed by a novel noise-balanced sparse grids interpolation approach which, in a quasi-optimal fashion, controls the amount of Monte Carlo sampling for each data point. The approach is complemented by a multilevel on-the-fly construction during the multiscale simulation. Besides its efficiency, a particularly appealing feature is the ease of use of the approach with only a single hyperparameter controlling the whole surrogate construction - from the surrogate's accuracy with guaranteed convergence to which data needs to be created with which accuracy. The approach is demonstrated on challenging examples from heterogeneous catalysis, coupling microscopic kinetic Monte Carlo models into macroscopic reactor simulations.