Existence of n -cycles and border-collision bifurcations in piecewise-linear continuous maps with applications to recurr
- PDF / 4,660,074 Bytes
- 16 Pages / 547.087 x 737.008 pts Page_size
- 55 Downloads / 176 Views
ORIGINAL PAPER
Existence of n-cycles and border-collision bifurcations in piecewise-linear continuous maps with applications to recurrent neural networks Z. Monfared · D. Durstewitz
Published online: 11 August 2020 © The Author(s) 2020
Abstract Piecewise linear recurrent neural networks (PLRNNs) form the basis of many successful machine learning applications for time series prediction and dynamical systems identification, but rigorous mathematical analysis of their dynamics and properties is lagging behind. Here, we contribute to this topic by investigating the existence of n-cycles (n ≥ 3) and bordercollision bifurcations in a class of m-dimensional piecewise linear continuous maps which have the general form of a PLRNN. This is particularly important as for one-dimensional maps the existence of 3-cycles implies chaos. It is shown that these n-cycles collide with the switching boundary in a border-collision bifurcation, and parametric regions for the existence of both stable and unstable n-cycles and border-collision bifurcations will be derived theoretically. We then discuss how our results can be extended and applied to PLRNNs. Finally, numerical simulations demonstrate the implementation of our results and are found to be in good agreement with the theoretical derivations. Our findings thus provide a basis for understanding periodic
Z. Monfared (B) · D. Durstewitz Department of Theoretical Neuroscience, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany e-mail: [email protected] D. Durstewitz Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany e-mail: [email protected]
behavior in PLRNNs, how it emerges in bifurcations, and how it may lead into chaos. Keywords Piecewise linear continuous maps · n-cycles · Stability · Border-collision bifurcations · Chaos · Recurrent neural networks · Machine learning
1 Introduction A piecewise smooth discrete-time dynamical system is a discrete-time map whose state space is split into two or more components (subregions) by some discontinuity borders or switching manifolds, such that in each subregion there is a different functional form of the map [2,3,7,26]. Piecewise smooth (PWS) maps have received growing attention in recent years, as they have a wide range of applications in various areas such as neural dynamics, switching circuits, or impacting mechanical systems [23]. One important type of PWS map is a piecewise linear continuous map, which is continuous but has some discontinuities in its Jacobian matrix across the switching boundaries. Piecewise linear recurrent neural networks (PLRNNs), which build on so-called ‘rectified linear units (ReLU)’, φ(z) = max(z, 0), as the network’s nonlinear activation function, are one example of such maps. In general, RNNs are the standard these days in machine learning for processing sequential, time-series information, due to their success in domains rich in temporal structure like natural language processing [15,27], predicti
Data Loading...
