Delay-based reservoir computing has gained a lot of attention due to the relative simplicity with which this concept can be implemented in hardware. However, unnecessary constraints are commonly placed on the relationship between the delay-time and the input clock-cycle, which can have a detrimental effect on the performance. We review the existing literature on this subject and introduce the concept of delay-based reservoir computing in a manner that demonstrates that no predefined relationship between the delay-time and the input clock-cycle is required for this computing concept to work. Choosing the delay-times independent of the input clock-cycle, one gains an important degree of freedom. Consequently, we discuss ways to improve the computing performance of a reservoir formed by delay-coupled oscillators and show the impact of delay-time tuning in such systems.
22 January 2024 in Neuromorphic Computing and Engineering
Task specific hyperparameter tuning in reservoir computing is an open issue, and is of particular relevance for hardware implemented reservoirs. We investigate the influence of directly including externally controllable task specific timescales on the performance and hyperparameter sensitivity of reservoir computing approaches. We show that the need for hyperparameter optimisation can be reduced if timescales of the reservoir are tailored to the specific task. Our results are mainly relevant for temporal tasks requiring memory of past inputs, for example chaotic timeseries prediction. We consider various methods of including task specific timescales in the reservoir computing approach and demonstrate the universality of our message by looking at both time-multiplexed and spatially-multiplexed reservoir computing.
In the reservoir computing literature, the information processing capacity is frequently used to characterize the computing capabilities of a reservoir. However, it remains unclear how the information processing capacity connects to the performance on specific tasks. We demonstrate on a set of standard benchmark tasks that the total information processing capacity correlates poorly with task specific performance. Further, we derive an expression for the normalized mean square error of a task as a weighted function of the individual information processing capacities. Mathematically, the derivation requires the task to have the same input distribution as used to calculate the information processing capacities. We test our method on a range of tasks that violate this requirement and find good qualitative agreement between the predicted and the actual errors as long as the task input sequences do not have long autocorrelation times. Our method offers deeper insight into the principles governing reservoir computing performance. It also increases the utility of the evaluation of information processing capacities, which are typically defined on i.i.d. input, even if specific tasks deliver inputs stemming from different distributions. Moreover, it offers the possibility of reducing the experimental cost of optimizing physical reservoirs, such as those implemented in photonic systems.
1 July 2023 in Chaos: An Interdisciplinary Journal of Nonlinear Science
We propose a new approach to dynamical system forecasting called data-informed-reservoir computing (DI-RC) that, while solely being based on data, yields increased accuracy, reduced computational cost, and mitigates tedious hyper-parameter optimization of the reservoir computer (RC). Our DI-RC approach is based on the recently proposed hybrid setup where a knowledge-based model is combined with a machine learning prediction system, but it replaces the knowledge-based component by a data-driven model discovery technique. As a result, our approach can be chosen when a suitable knowledge-based model is not available. We demonstrate our approach using a delay-based RC as the machine learning component in conjunction with sparse identification of nonlinear dynamical systems for the data-driven model component. We test the performance on two example systems: the Lorenz system and the Kuramoto–Sivashinsky system. Our results indicate that our proposed technique can yield an improvement in the time-series forecasting capabilities compared with both approaches applied individually, while remaining computationally cheap. The benefit of our proposed approach, compared with pure RC, is most pronounced when the reservoir parameters are not optimized, thereby reducing the need for hyperparameter optimization.
We propose a scheme that can enhance the performance and reduce the computational cost of quantum reservoir computing. Quantum reservoir computing is a computing approach which aims at utilizing the complexity and high dimensionality of small quantum systems, together with the fast trainability of reservoir computing, in order to solve complex tasks. The suitability of quantum reservoir computing for solving temporal tasks is hindered by the collapse of the quantum system when measurements are made. This leads to the erasure of the memory of the reservoir. Hence, for every output, the entire input signal is needed to reinitialize the reservoir, leading to quadratic time complexity. Another critical issue for the hardware implementation of quantum reservoir computing is the need for an experimentally accessible means of tuning the nonlinearity of the quantum reservoir. We present an approach which addresses both of these issues. We propose artificially restricting the memory of the quantum reservoir by only using a small number inputs to reinitialize the reservoir after measurements are performed. This strongly influences the nonlinearity of the reservoir response due to the influence of the initial reservoir state, while also substantially reducing the number of quantum operations needed to perform time-series prediction tasks due to the linear rather than quadratic time complexity. The reinitialization length therefore provides an experimental accessible means of tuning the nonlinearity of the response of the reservoir, which can lead to significant task-specific performance improvement. We numerically study the linear and quadratic algorithms for a fully connected transverse Ising model and a quantum processor model. Published by the American Physical Society 2024
27 August 2024 in Machine Learning: Science and Technology
Forecasting timeseries based upon measured data is needed in a wide range of applications and has been the subject of extensive research. A particularly challenging task is the forecasting of timeseries generated by chaotic dynamics. In recent years reservoir computing has been shown to be an effective method of forecasting chaotic dynamics and reconstructing chaotic attractors from data. In this work strides are made toward smaller and lower complexity reservoirs with the goal of improved hardware implementability and more reliable production of adequate surrogate models. We show that a reservoir of uncoupled nodes more reliably produces long term timeseries predictions than more complex reservoir topologies. We then link the improved attractor reconstruction of the uncoupled reservoir with smaller spectral radii of the resulting surrogate systems. These results indicate that, the node degree plays an important role in determining whether the desired dynamics will be stable in the autonomous surrogate system which is attained via closed-loop operation of the trained reservoir. In terms of hardware implementability, uncoupled nodes would allow for greater freedom in the hardware architecture because no complex coupling setups are needed and because, for uncoupled nodes, the system response is equivalent for space and time multiplexing.