Research on denoising sparse autoencoder
- PDF / 3,166,435 Bytes
- 11 Pages / 595.276 x 790.866 pts Page_size
- 91 Downloads / 225 Views
ORIGINAL ARTICLE
Research on denoising sparse autoencoder Lingheng Meng1,2 • Shifei Ding1,2 • Yu Xue3
Received: 16 September 2015 / Accepted: 18 May 2016 Ó Springer-Verlag Berlin Heidelberg 2016
Abstract Autoencoder can learn the structure of data adaptively and represent data efficiently. These properties make autoencoder not only suit huge volume and variety of data well but also overcome expensive designing cost and poor generalization. Moreover, using autoencoder in deep learning to implement feature extraction could draw better classification accuracy. However, there exist poor robustness and overfitting problems when utilizing autoencoder. In order to extract useful features, meanwhile improve robustness and overcome overfitting, we studied denoising sparse autoencoder through adding corrupting operation and sparsity constraint to traditional autoencoder. The results suggest that different autoencoders mentioned in this paper have some close relation and the model we researched can extract interesting features which can reconstruct original data well. In addition, all results show a promising approach to utilizing the proposed autoencoder to build deep models. Keywords Autoencoder Feature extraction Unsupervised learning Sparse coding Deep networks
& Shifei Ding [email protected] 1
School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China
2
Jiangsu Key Laboratory of Mine Mechanical and Electrical Equipment, China University of Mining and Technology, Xuzhou 221116, China
3
College of Computer and Software, Nanjing University of Information Science and Technology, Nanjing 210044, China
1 Introduction The cost of manually designed feature extractors is extremely expensive and the generalization of these extractors is poor as well. Specifically, one sort of feature might be useful for one particular type of image, but for other kinds of images, it could does not work. How to adaptively learn feature extractors which are applicable to one kind of data and then exploiting these extractors to solve learning task relating to this data has become a key point of unsupervised learning. Furthermore, with the increasing in data dimensions, it is more and more important for machine learning and big data related researches to represent data efficiently. A manner proposed by Hinton et al. [1] uses lower dimensional data for efficiently representing high dimensional data by means of neural networks. Bengio et al. [2] got better results by using autoencoder to pre-train deep networks. All of this means autoencoder does have the ability to extract the structure of data and can be applied to machine learning task. Autoencoder can adaptively extract images’ inherent features [3–7]. From this perspective, autoencoder can be viewed as an encoder. The motivation of research on autoencoder is to improve features’ generalization and to make autoencoder has ability to encode and decode any kind of data. Autoencoder has not only got the attention of theorists, but also
Data Loading...
