Limitations of the Recall Capabilities in Delay-Based Reservoir Computing Systems
- PDF / 1,119,528 Bytes
- 8 Pages / 595.224 x 790.955 pts Page_size
- 14 Downloads / 166 Views
Limitations of the Recall Capabilities in Delay-Based Reservoir Computing Systems 1 ¨ Felix Koster
1 ¨ · Dominik Ehlert1 · Kathy Ludge
Received: 28 February 2020 / Accepted: 14 May 2020 © The Author(s) 2020
Abstract We analyse the memory capacity of a delay-based reservoir computer with a Hopf normal form as nonlinearity and numerically compute the linear as well as the higher order recall capabilities. A possible physical realization could be a laser with external cavity, for which the information is fed via electrical injection. A task-independent quantification of the computational capability of the reservoir system is done via a complete orthonormal set of basis functions. Our results suggest that even for constant readout dimension the total memory capacity is dependent on the ratio between the information input period, also called the clock cycle, and the time delay in the system. Optimal performance is found for a time delay about 1.6 times the clock cycle. Keywords Lasers · Reservoir computing · Nonlinear dynamics
Introduction Reservoir computing is a machine learning paradigm [1] inspired by the human brain [2], which utilizes the natural computational capabilities of dynamical systems. As a subset of recurrent neural networks it was developed to predict time-dependent tasks with the advantage of a very fast training procedure. Generally the training of recurrent neural networks is connected with high computational cost resulting e.g. from connections that are correlated in time. Therefore, problems like the vanishing gradient in time arise [3]. Reservoir computing avoids this problem by training just a linear output layer, leaving the rest of the system (the
This article belongs to the Topical Collection: Trends in Reservoir Computing Guest Editors: Claudio Gallicchio, Alessio Micheli, Simone Scardapane, Miguel C. Soriano Felix K¨oster
[email protected] Dominik Ehlert [email protected] Kathy L¨udge [email protected] 1
Institut f¨ur Theoretische Physik, Technische Universit¨at Berlin, Straße des 17. Juni 135, 10623 Berlin, Germany
reservoir) as it is. Thus, the inherent computing capabilities can be exploited. One can divide a reservoir into three distinct subsystems, the input layer, which corresponds to the projection of the input information into the system, the dynamical system itself that processes the information, and the output layer, which is a linear combination of the system’s states trained to predict an often time-dependent task. Many different realizations have been presented in the last years, ranging from a bucket of water [4] over field programmable gate arrays (FPGAs) [5] to dissociated neural cell cultures [6], being used for satellite communications [7], real-time audio processing [8, 9], bit-error correction for optical data transmission [10], amplitude of chaotic laser pulse prediction [11] and cross-predicting the dynamics of an injected laser [12]. Especially opto-electronic [13, 14] and optical setups [15–19] were frequently studied because their high s
Data Loading...
