Deep Representation Based on Multilayer Extreme Learning Machine
Here, we propose a fast deep learning architecture for feature representation. The target of deep learning in our model is to capture the relevant higher-level abstraction from disentangling input features, which is possible due to the speed of the extrem
- PDF / 138,896 Bytes
- 6 Pages / 439.37 x 666.142 pts Page_size
- 53 Downloads / 223 Views
Abstract Here, we propose a fast deep learning architecture for feature representation. The target of deep learning in our model is to capture the relevant higher-level abstraction from disentangling input features, which is possible due to the speed of the extreme learning machine (ELM). We use ELM auto encoder (ELM-AE) to add a regularization term into ELM for improving generalization performance. To demonstrate our model with a high performance for deep representation, we conduct experiments on the MNIST database and compare the proposed method with state-of-the-art deep representation methods. Experimental results show the proposed method is competitive for deep representation and reduces amount of time needed for training. Keywords Deep networks Auto encoder
Extreme learning machine Deep representation
1 Introduction The essence of deep representation is transitioning from simpler elementary features to more abstract advanced features based on deep learning models. Restricted Boltzmann machine (RBM) [1] and auto-encoders [2] can be used to train multiple-layer neural networks, or deep networks. For the RBM, one is the deep belief network (DBN) [1], the another is deep Boltzmann machine (DBM) [3]. For the auto-encoder, one is the stacked auto-encoder (SAE) [2], there is another type auto-encoder model called the stacked de-noising auto-encoder (SDAE) [3]. Deep networks sometimes perform better than single-layer feed-forward neural networks Y.-L. Qi (&) Y.-L. Li Department of Computer Science, Beijing Institute of Graphic Communication, Beijing, China e-mail: [email protected] Y.-L. Li e-mail: [email protected] © Springer Science+Business Media Singapore 2016 A. Hussain (ed.), Electronics, Communications and Networks V, Lecture Notes in Electrical Engineering 382, DOI 10.1007/978-981-10-0740-8_17
147
148
Y.-L. Qi and Y.-L. Li
(SLFNs) and traditional multilayer neural networks for big data, but they usually have slow learning speed [4]. Training the deep networks requires fine-tuning of the system weights and bias in reverse, which consumes a lot of time, thus reducing the learning speed [5–7]. To solve this problem, we proposed a method based on extreme learning machine (ELM). Huang proposed ELM, which has a fast learning speed and good generalization [8]. Lekamalage et al. [4] propose a multilayer extreme learning machine that performs layer-by-layer unsupervised learning. It is similar to deep networks but has significantly faster speed. Yu et al. propose a deep representation ELM (DrELM) [9]. It utilizes ELM as stacking elements to construct a stacked framework. In the work presented here, we propose an ELM-AE multilayer model for classification. The remainder of this paper is organized as following: Sect. 2 proposes the multilayer model based on ELM-AE; the experimental results are presented in Sects. 3 and 4 concludes this study and mentions the directions for future works.
2 Multilayer Model Based on ELM-AE 2.1
ELM-AE
The hidden nodes of the ELM can be randomly generated between its input and o
Data Loading...
