Nonlinear System Identification Using Neural Networks Trained with Natural Gradient Descent
- PDF / 720,383 Bytes
- 9 Pages / 600 x 792 pts Page_size
- 15 Downloads / 190 Views
Nonlinear System Identification Using Neural Networks Trained with Natural Gradient Descent Mohamed Ibnkahla Electrical and Computer Engineering Department, Queen’s University, Kingston, Ontario, Canada K7L 3N6 Email: [email protected] Received 13 December 2002 and in revised form 17 May 2003 We use natural gradient (NG) learning neural networks (NNs) for modeling and identifying nonlinear systems with memory. The nonlinear system is comprised of a discrete-time linear filter H followed by a zero-memory nonlinearity g(·). The NN model is composed of a linear adaptive filter Q followed by a two-layer memoryless nonlinear NN. A Kalman filter-based technique and a search-and-converge method have been employed for the NG algorithm. It is shown that the NG descent learning significantly outperforms the ordinary gradient descent and the Levenberg-Marquardt (LM) procedure in terms of convergence speed and mean squared error (MSE) performance. Keywords and phrases: satellite communications, system identification, adaptive signal processing, neural networks.
1.
INTRODUCTION
Most techniques that have been proposed for nonlinear system identification are based on parametrized nonlinear models such as Wiener and Hammerstein models [1, 2, 3, 4], Volterra series [5], wavelet networks [3], neural networks (NNs) [6, 7], and so forth. The estimation of the parameters is performed either using nonadaptive techniques such as least squares methods and higher-order statistics-based methods [4, 8, 9, 10, 11], or adaptive techniques such as the backpropagation (BP) algorithm [12, 13, 14] and online learning [3, 15]. NN approaches for modeling and identifying nonlinear dynamical systems have shown excellent performance compared to classical techniques [1, 6, 9, 13, 16]. NNs trained with the BP algorithm [14, 16] have, however, two major drawbacks: first, their convergence is slow, which can be inadequate for online training; second, the NN parameters may be trapped in a nonoptimal local minimum, leading to suboptimal approximation of the system [6]. Natural gradient (NG) learning [17, 18] on the other hand, has been shown to have better convergence capabilities than the classical BP algorithm because it takes into account the geometry of the manifold in which the NN weights evolve. Therefore, NG learning can better avoid the plateau phenomena, which characterize the BP learning curves. The unknown nonlinear system studied in this paper (Figure 1) is a nonlinear Wiener system composed of a lin ear filter H(z) = Nk=h −0 1 hk z−k followed by a zero-memory
nonlinearity g(·). This nonlinear system structure has been used in many applications, for example, in satellite communications where the uplink channel is composed of a linear filter followed by a traveling wave tube (TWT) amplifier [5, 19, 20], in microwave amplifier design when modeling solid-state power amplifiers (SSPAs) [13], in adaptive control of nonlinear systems [9], and in biomedical applications when modeling the relationships between cardiovascular signals [1,
Data Loading...
