1 July 2019 in Neural Networks
Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for mapping inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.
17 January 2018 in IEEE Access
We propose a reservoir computing device utilizing spin waves that propagate in a garnet film equipped with multiple input/output electrodes. In recent years, reservoir computing has been expected to realize energy-efficient and/or high-speed machine learning. Our proposed device enhances such significant merits in a hardware approach. It utilizes the nonlinear interference of history-dependent asymmetrically propagating spin waves excited by the magneto-electric effect. First, we investigate a feasible device structure with practical physical parameters in micromagnetic numerical analysis, and show the detailed characteristics of the forward volume magnetostatic spin waves. Then, we demonstrate high generalization ability in the estimation of input-signal parameters performed by the spin-wave-based reservoir computing. We find that the hysteresis characteristics of the spin waves propagating asymmetrically with respect to excitation points, as well as the nonlinear interference, works advantageously to realize high diversity in the time-sequential signals in high-dimensional information space, which has the highest significance for effective learning in reservoir computing. The spin wave device is highly promising for next-generation machine-learning electronics.
28 July 2009 in IEEE Transactions on Neural Networks
A widely used complex-valued activation function for complex-valued multistate Hopfield networks is revealed to be essentially based on a multilevel step function. By replacing the multilevel step function with other multilevel characteristics, we present two alternative complex-valued activation functions. One is based on a multilevel sigmoid function, while the other on a characteristic of a multistate bifurcating neuron. Numerical experiments show that both modifications to the complex-valued activation function bring about improvements in network performance for a multistate associative memory. The advantage of the proposed networks over the complex-valued Hopfield networks with the multilevel step function is more outstanding when a complex-valued neuron represents a larger number of multivalued states. Further, the performance of the proposed networks in reconstructing noisy 256 gray-level images is demonstrated in comparison with other recent associative memories to clarify their advantages and disadvantages.
19 October 2008 in Journal of Nonlinear Science
25 January 2012 in Scientific Reports
Many social, biological, and technological networks consist of a small number of highly connected components (hubs) and a very large number of loosely connected components (low-degree nodes). It has been commonly recognized that such heterogeneously connected networks are extremely vulnerable to the failure of hubs in terms of structural robustness of complex networks. However, little is known about dynamical robustness, which refers to the ability of a network to maintain its dynamical activity against local perturbations. Here we demonstrate that, in contrast to the structural fragility, the nonlinear dynamics of heterogeneously connected networks can be highly vulnerable to the failure of low-degree nodes. The crucial role of low-degree nodes results from dynamical processes where normal (active) units compensate for the failure of neighboring (inactive) units at the expense of a reduction in their own activity. Our finding highlights the significant difference between structural and dynamical robustness in complex networks.
13 November 2010 in Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
Hormone therapy in the form of androgen deprivation is a major treatment for advanced prostate cancer. However, if such therapy is overly prolonged, tumour cells may become resistant to this treatment and result in recurrent fatal disease. Long-term hormone deprivation also is associated with side effects poorly tolerated by patients. In contrast, intermittent hormone therapy with alternating on- and off-treatment periods is a possible clinical strategy to delay progression to hormone-refractory disease with the advantage of reduced side effects during the off-treatment periods. In this paper, we first overview previous studies on mathematical modelling of prostate tumour growth under intermittent hormone therapy. The model is categorized into a hybrid dynamical system because switching between on-treatment and off-treatment intervals is treated in addition to continuous dynamics of tumour growth. Next, we present an extended model of stochastic differential equations and examine how well the model is able to capture the characteristics of authentic serum prostate-specific antigen (PSA) data. We also highlight recent advances in time-series analysis and prediction of changes in serum PSA concentrations. Finally, we discuss practical issues to be considered towards establishment of mathematical model-based tailor-made medicine, which defines how to realize personalized hormone therapy for individual patients based on monitored serum PSA levels.