Deep Robust Encoder Through Locality Preserving Low-Rank Dictionary
Deep learning has attracted increasing attentions recently due to its appealing performance in various tasks. As a principal way of deep feature learning, deep auto-encoder has been widely discussed in such problems as dimensionality reduction and model p
- PDF / 891,300 Bytes
- 16 Pages / 439.37 x 666.142 pts Page_size
- 81 Downloads / 158 Views
Department of Electrical and Computer Engineering, Northeastern University, Boston, USA {allanding,mingshao}@ece.neu.edu 2 College of Computer and Information Science, Northeastern University, Boston, USA [email protected]
Abstract. Deep learning has attracted increasing attentions recently due to its appealing performance in various tasks. As a principal way of deep feature learning, deep auto-encoder has been widely discussed in such problems as dimensionality reduction and model pre-training. Conventional auto-encoder and its variants usually involve additive noises (e.g., Gaussian, masking) for training data to learn robust features, which, however, did not consider the already corrupted data. In this paper, we propose a novel Deep Robust Encoder (DRE) through locality preserving low-rank dictionary to extract robust and discriminative features from corrupted data, where a low-rank dictionary and a regularized deep auto-encoder are jointly optimized. First, we propose a novel loss function in the output layer with a learned low-rank clean dictionary and corresponding weights with locality information, which ensures that the reconstruction is noise free. Second, discriminant graph regularizers that preserve the local geometric structure for the data are developed to guide the deep feature learning in each encoding layer. Experimental results on several benchmarks including object and face images verify the effectiveness of our algorithm by comparing with the state-of-the-art approaches.
Keywords: Auto-encoder
1
· Low-rank dictionary · Graph regularizer
Introduction
In the recent years, deep learning has attracted considerable interests in computer vision field, as it has achieved promising performance in various tasks, e.g., image classification [1], object detection [2] and face recognition [3]. Generally, deep structure learning tends to extract hierarchical feature representations directly from raw data. Recent representative research works include: deep convolutional neural networks [4], deep neural networks [5], deep auto-encoder [6], and deeply-supervised nets [7]. c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part VI, LNCS 9910, pp. 567–582, 2016. DOI: 10.1007/978-3-319-46466-4 34
568
Z. Ding et al.
Fig. 1. Illustration of our proposed algorithm. Corrupted data xi , xj are the inputs of the deep AE. After encoding and decoding process, the reconstructed xi , xj are encouraged to be close to Dzi , Dzj on the top, where D is the learned clean low-rank dictionary and zi , zj are corresponding coefficients. In addition, graph regularizers are added to the encoder layers to pass on the locality information.
Among different deep structures, auto-encoder (AE) [8] has been treated as robust feature extractors or pre-training scheme in various tasks [9–14]. Conventional AE was proposed to encourage similar or identical input-output pairs where the reconstruction loss is minimized after decoding [8]. Follow-up work with various additive noises in the input layer is able to pr
Data Loading...
