EEG-based emotion recognition using 4D convolutional recurrent neural network
- PDF / 2,293,522 Bytes
- 14 Pages / 595.276 x 790.866 pts Page_size
- 11 Downloads / 386 Views
(0123456789().,-volV)(0123456789(). ,- volV)
RESEARCH ARTICLE
EEG-based emotion recognition using 4D convolutional recurrent neural network Fangyao Shen1 • Guojun Dai1,2 • Guang Lin1 • Jianhai Zhang1,2 • Wanzeng Kong1,2 • Hong Zeng1,2 Received: 30 April 2020 / Revised: 25 August 2020 / Accepted: 4 September 2020 Ó Springer Nature B.V. 2020
Abstract In this paper, we present a novel method, called four-dimensional convolutional recurrent neural network, which integrating frequency, spatial and temporal information of multichannel EEG signals explicitly to improve EEG-based emotion recognition accuracy. First, to maintain these three kinds of information of EEG, we transform the differential entropy features from different channels into 4D structures to train the deep model. Then, we introduce CRNN model, which is combined by convolutional neural network (CNN) and recurrent neural network with long short term memory (LSTM) cell. CNN is used to learn frequency and spatial information from each temporal slice of 4D inputs, and LSTM is used to extract temporal dependence from CNN outputs. The output of the last node of LSTM performs classification. Our model achieves state-of-the-art performance both on SEED and DEAP datasets under intra-subject splitting. The experimental results demonstrate the effectiveness of integrating frequency, spatial and temporal information of EEG for emotion recognition. Keywords EEG Emotion recognition 4D data Convolutional recurrent neural network
Introduction Emotion recognition has received increasing attention in the field of affective computing in recent years, due to its potential applications in human–machine interaction (HMI) (Fiorini et al. 2020; Cowie et al. 2001), diseases evaluation (Figueiredo et al. 2019; Bamdad et al. 2015; Vansteensel and Jarosiewicz 2020) and driving fatigue detection (Kong et al. 2017; Zeng et al. 2018, 2019b), and mental workload estimation (Blankertz et al. 2016; Arico` et al. 2019; Cartocci et al. 2015). Emotion recognition methods could be categorized into two major classes, one is based on non-physiological signals [e.g., facial expression and speech (Yan et al. 2016; Zhang et al. 2019)] and another is based on physiological signals [e.g., electroencephalography (EEG) and electrocardiography (ECG) & Hong Zeng [email protected] 1
School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
2
Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou, China
(Chen et al. 2015; Zheng et al. 2017; Hsu et al. 2017)]. EEG is characterized by noninvasive, portability, reliability, and small cost. It has been widely used in the field of brain–computer interfaces (BCIs) (Pfurtscheller et al. 2010; Arico` et al. 2018, 2020), which establishing a direct communication channel between human beings and computers. Recently, enhancing BCI by taking advantage of the information of the user’s emotional states from EEG has gained more and more attention, which termed as affective brain–computer interfa
Data Loading...
