Performance analysis of No-Propagation and ELM algorithms in classification
- PDF / 724,344 Bytes
- 11 Pages / 595.276 x 790.866 pts Page_size
- 5 Downloads / 219 Views
(0123456789().,-volV)(0123456789().,-volV)
S.I. : IWINAC 2015
Performance analysis of No-Propagation and ELM algorithms in classification Juan-Antonio Martı´nez-Garcı´a1
•
Jose´-Luis Sancho-Go´mez1
Received: 14 August 2017 / Accepted: 8 January 2018 Ó The Natural Computing Applications Forum 2018
Abstract The growing volume and complexity of data has led to the development of the so-called linear algorithms for neural networks like ELM, which maintain the precision of classic algorithms but with higher training speed. This speed increase is due to a simpler architecture, the random fixing of the input weights without being trained and the analytical calculation of the output weights instead of the slowly classical iterative gradient methods as Backpropagation. However, the random fixing of the input weights increases the sensibility to input perturbations like noise. Recently, No-Propagation (No-Prop) algorithm has been introduced as another linear algorithm, which shares with ELM the architecture and the random input weights (hidden weights) initialization. In this paper, an exhaustive comparison of both algorithms and its regularized versions are presented. The simulations results suggest that No-Prop is a competitive alternative to the ELM algorithm. Keywords Neural networks Machine learning Binary classification ELM No-Prop Noise Filtering Overfitting Mathematics Subject Classification 62M45 62L15 68T05 68T20 92B20 93E24
1 Introduction Extreme Learning Machines (ELM) and No-Propagation (No-Prop) are algorithms for training feedforward neural networks that have obtained increasing interest in the last recent years [4, 15]. This is because they outperform the speed of classic algorithms like Backpropagation or Support Vector Machines (SVM), without loosing their good precision performance. The increase of speed is partially due to the randomly fixing of the input weights, avoiding training them. As a result, the sensibility to input noise is increased.
& Juan-Antonio Martı´nez-Garcı´a [email protected] Jose´-Luis Sancho-Go´mez [email protected] 1
Grupo de Teorı´a y Tratamiento de la Sen˜al, Departamento de Tecnologı´as de la Informacio´n y las Comunicaciones, Universidad Polite´cnica de Cartagena, Plaza del Hospital, 1, Edificio Cuartel de Antigones, 30202 Cartagena, Murcia, Spain
Many papers introduce a solution for noise reduction with ELM algorithm. In [9], a FIR-ELM (Finite Input Response—ELM) scheme is introduced where the hidden layer works like a frequency filter training the input weights and using delays in data. A DFT-ELM (Discrete Fourier Transform—ELM) proposed is presented in [8]. In that work, the input features are turned into a time sequence and sampled at the hidden layer output by the DFT with as many equally spaced frequencies as the number of hidden neurons. The use of the hidden layer to filter in frequency the data input noise is not better than filtering them in a preprocessing stage. It is assumed that before training a neural network, data are preproces
Data Loading...
