CR-Net: A Deep Classification-Regression Network for Multimodal Apparent Personality Analysis
- PDF / 4,789,179 Bytes
- 18 Pages / 595.276 x 790.866 pts Page_size
- 88 Downloads / 229 Views
CR-Net: A Deep Classification-Regression Network for Multimodal Apparent Personality Analysis Yunan Li1,2 · Jun Wan3,4 · Qiguang Miao1,2 · Sergio Escalera5 · Huijuan Fang1,2 · Huizhou Chen1,2 · Xiangda Qi1,2 · Guodong Guo6,7 Received: 14 May 2019 / Accepted: 13 February 2020 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract First impressions strongly influence social interactions, having a high impact in the personal and professional life. In this paper, we present a deep Classification-Regression Network (CR-Net) for analyzing the Big Five personality problem and further assisting on job interview recommendation in a first impressions setup. The setup is based on the ChaLearn First Impressions dataset, including multimodal data with video, audio, and text converted from the corresponding audio data, where each person is talking in front of a camera. In order to give a comprehensive prediction, we analyze the videos from both the entire scene (including the person’s motions and background) and the face of the person. Our CR-Net first performs personality trait classification and applies a regression later, which can obtain accurate predictions for both personality traits and interview recommendation. Furthermore, we present a new loss function called Bell Loss to address inaccurate predictions caused by the regression-to-the-mean problem. Extensive experiments on the First Impressions dataset show the effectiveness of our proposed network, outperforming the state-of-the-art. Keywords Personality traits · Multimodal data · Convolutional neural networks · Classification-regression network · Bell Loss function
1 Introduction The analysis of human affective behavior is an active research in computer vision nowadays, which can be widely used in a variety of applications, such as social relation analysis (Xia et al. 2017), analysis of depression (Klein et al. 2011) and job candidate screening (Naim et al. 2015; PonceLópez et al. 2016; Escalante et al. 2016), among others.
Unconscious behaviors may produce facial expressions or words of a person that can reflect some traits of personality, influencing other people’s impression about him/her. Evidence with psychological support has been shown in the case of job interviews (Barrick and Mount 1991). However, in real-world situations, estimating one’s personality is still an open problem in psychology, linguistics and physiology (Wei et al. 2018). The advances in computer vision are providing support to advance the study of personality computing, ben-
Communicated by Wenjun Zeng.
B
1
Jun Wan [email protected]
2
Xi’an Key Laboratory of Big Data and Intelligent Vision, Xi’an, China
Yunan Li [email protected]
3
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
Qiguang Miao [email protected]
4
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
Sergio Escalera [email protected]
5
Universitat de Barcelona and Computer Vi
Data Loading...
