New prior distribution for Bayesian neural network and learning via Hamiltonian Monte Carlo
- PDF / 1,761,534 Bytes
- 11 Pages / 595.276 x 790.866 pts Page_size
- 26 Downloads / 183 Views
ORIGINAL PAPER
New prior distribution for Bayesian neural network and learning via Hamiltonian Monte Carlo Hassan Ramchoun1 · Mohamed Ettaouil1 Received: 7 September 2018 / Accepted: 11 May 2019 © Springer-Verlag GmbH Germany, part of Springer Nature 2019
Abstract A prior distribution of weights for Multilayer feedforward neural network in Bayesian point of view plays a central role toward generalization. In this context, we propose a new prior law for weights parameters which motivate the network regularization more than l1 and l2 early proposed. To train the network, we have based on Hamiltonian Monte Carlo, it is used to simulate the prior and the posterior distribution. The generated samples are used to approximate the gradient of the evidence which allows to re-estimate the hyperparameters that balance a trade off between the likelihood term and regularized term, on the other hand we use the obtained posterior samples to estimate the network output. The case problem studied in this paper includes a regression and classification tasks. The obtained results illustrate the advantages of our approach in term of error rate compared to old approach, unfortunately our method consomme time before convergence. Keywords Bayesian multilayer feedforward neural network · Prior · Hamiltonian Monte Carlo · Evidence framework · Hyper-parameter · Regularization
1 Introduction Artificial neural network is one of the most maching learning method used for several real application (Ekonomou et al. 2016). Multilayer perceptron (MLP) is the most class of Multilayer feedforward neural network (FNN) used in various tasks such as regression for real target and classification for discrete one, therefore it has been used for prediction in application related to economics and finance (Kocadağlı and Aşıkgil 2014) etc. With standard FNNs the main difficulty is how to control the complexity, furthermore, there are a lack of tools for analyzing the results (confidence intervals, error bars) and for complex data, the problem of overfitting arise. Recent works that involve the regularization of hybrid models Fuzzy Neural Networks that also use Lasso to regularize the model. With the difference, they use the regularization techniques through bootstrap replications. The method is called a bolasso. There are two examples where the former is used to * Hassan Ramchoun [email protected] 1
Department of Mathematics, Modeling and Scientific Computing Laboratory, Faculty of Sciences and Technics, University of Sidi Mohamed Ben Abdellah, Fez, Morocco
prune neurons less relevant to the model that has a generation of neurons in an exponential problem between the number of dimensions and the number of membership functions used in the fuzzification process (de Campos Souza et al. 2018, 2019; Angelov and Kasabov 2005). Bayesian methods are a viable alternative to the older error minimization approaches (Bishop 1995; Neal 2012), it based on prior information of parameters (Angelov 2014). This learning perspectives for MLP supposes that all n
Data Loading...
