Deep Boltzmann machine for nonlinear system modelling
- PDF / 2,819,560 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 7 Downloads / 177 Views
ORIGINAL ARTICLE
Deep Boltzmann machine for nonlinear system modelling Wen Yu1 · Erick de la Rosa1 Received: 8 December 2017 / Accepted: 8 June 2018 © Springer-Verlag GmbH Germany, part of Springer Nature 2018
Abstract Deep Boltzmann machine (DBM) has been successfully applied in classification, regression and time series modeling. For nonlinear system modelling, DBM should also have many advantages over the other neural networks, such as input features extraction and noise tolerance. In this paper, we use DBM to model nonlinear systems by calculating the probability distributions of the input and output. Two novel weight updating algorithms are proposed to obtain these distributions. We use binary encoding and conditional probability transformation methods. The proposed methods are validated with two benchmark nonlinear systems. Keywords Deep learning · Boltzmann machine · System identification
1 Introduction Recent years, deep learning techniques have achieved impressive good results in many difficult tasks [1, 2]. There are some popular deep learning models, such as deep belief networks (DBN) [3], convolutional neural networks (CNN) [4], and deep Boltzmann machine (DBM) [5]. By using deep structure and feature extraction, these models successfully solve many machine learning problems. Both DBN and DBM use restricted Boltzmann machines (RBM). The dual-direction training of RBM can extract features from ambiguous and complex data effectively [6]. DBMs are energy based generative models. They learn the probability distribution of the input data through the latent or hidden variables. The latent variables capture features of the data, which help DBMs to obtain better representation of the empirical distribution. The DBM has been successfully applied to extract features [7]. It is also used as pre-training tool to set initial parameters for discriminative models [9, 10]. These properties of DBMs are widely used for solving classification problems for the image and text data [11]. DBMs are also applied for data regression and time series modeling [12]. [15] uses denoising autoencoders to pre-train * Wen Yu [email protected] 1
Departamento de Control Automatico, CINVESTAVIPN (National Polytechnic Institute), Av. IPN 2508, Mexico City 07360, Mexico
the model. The prediction results are better than no pre-training learning methods. To obtain the desired accuracy, [8] shows that hidden node number of the RBM should be the same as the total data number. So the denoising autoencoders may not improve prediction results if the time series is not sufficiently large [13]. Since the hidden and visible units of the DBM are binary, the prediction results for continuous values are not satisfied [14]. Time series modeling with deep learning methods is that they do not take into account the continuous nature of the variables of physical systems. [18] has tried continuous values in image classification, and showed that the classification accuracy of DBM is acceptable. In the sense of probability theory, the objective of s
Data Loading...
