Multimodal feature fusion for CNN-based gait recognition: an empirical comparison
- PDF / 1,969,392 Bytes
- 21 Pages / 595.276 x 790.866 pts Page_size
- 2 Downloads / 193 Views
(0123456789().,-volV)(0123456789().,-volV)
ORIGINAL ARTICLE
Multimodal feature fusion for CNN-based gait recognition: an empirical comparison Francisco M. Castro1 • Manuel J. Marı´n-Jime´nez2 • Nicola´s Guil3 • Nicola´s Pe´rez de la Blanca4 Received: 6 March 2019 / Accepted: 22 February 2020 Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract People identification in video based on the way they walk (i.e., gait) is a relevant task in computer vision using a noninvasive approach. Standard and current approaches typically derive gait signatures from sequences of binary energy maps of subjects extracted from images, but this process introduces a large amount of non-stationary noise, thus conditioning their efficacy. In contrast, in this paper we focus on the raw pixels, or simple functions derived from them, letting advanced learning techniques to extract relevant features. Therefore, we present a comparative study of different convolutional neural network (CNN) architectures by using three different modalities (i.e., gray pixels, optical flow channels and depth maps) on two widely adopted and challenging datasets: TUM-GAID and CASIA-B. In addition, we perform a comparative study between different early and late fusion methods used to combine the information obtained from each kind of modalities. Our experimental results suggest that (1) the raw pixel values represent a competitive input modality, compared to the traditional state-of-the-art silhouette-based features (e.g., GEI), since equivalent or better results are obtained; (2) the fusion of the raw pixel information with information from optical flow and depth maps allows to obtain state-of-the-art results on the gait recognition task with an image resolution several times smaller than the previously reported results; and (3) the selection and the design of the CNN architecture are critical points that can make a difference between state-of-the-art results or poor ones. Keywords Gait signature Convolutional neural networks Multimodal fusion
1 Introduction & Francisco M. Castro [email protected] Manuel J. Marı´n-Jime´nez [email protected] Nicola´s Guil [email protected] Nicola´s Pe´rez de la Blanca [email protected] 1
Department of Computer Architecture, University of Malaga, Bulevar Louis Pasteur, 35, Office 2.3.8a, 29071 Malaga, Spain
2
Department of Computing and Numerical Analysis, University of Cordoba, Cordoba, Spain
3
Department of Computer Architecture, University of Malaga, Malaga, Spain
4
Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain
The goal of gait-based people identification or simply gait recognition is to identify people by the way they walk. This type of biometric approach is considered noninvasive, since it is performed at a distance, and does not require the cooperation of the subject that has to be identified, in contrast to other methods as iris- or fingerprint-based approaches [1, 2]. Gait recognition has multiple applications in the context of video surveillance, ranging f
Data Loading...
