Machine Learning for Identifying Emotional Expression in Text: Improving the Accuracy of Established Methods
- PDF / 303,273 Bytes
- 7 Pages / 595.276 x 790.866 pts Page_size
- 40 Downloads / 191 Views
Machine Learning for Identifying Emotional Expression in Text: Improving the Accuracy of Established Methods Erin O’Carroll Bantum 1 & Noémie Elhadad 2 & Jason E. Owen 3 & Shaodian Zhang 2 & Mitch Golant 4 & Joanne Buzaglo 4 & Joanne Stephen 5 & Janine Giese-Davis 6
# Springer International Publishing 2017
Abstract Expression of emotion has been linked to numerous critical and beneficial aspects of human functioning. Accurately capturing emotional expression in text grows in relevance as people continue to spend more time in an online environment. The Linguistic Inquiry and Word Count (LIWC) is a commonly used program for the identification of many constructs, including emotional expression. In an earlier study by Bantum and Owen (Psychol. Assess. 21:79–88, 2009), LIWC was demonstrated to have good sensitivity yet poor positive predictive value. The goal of the current study was to create an automated machine learning technique to mimic manual coding. The sample included online support groups, cancer discussion boards, and transcripts from an expressive writing study, which resulted in 39,367 sentence-level coding decisions. In examining the entire sample, the machine learning approach outperformed LIWC, in all categories outside of sensitivity for negative emotion (LIWC sensitivity = 0.85; machine learning sensitivity = 0.41), although LIWC does not take into consideration prosocial emotion, such as affection, interest, and validation. LIWC performed significantly better
* Erin O’Carroll Bantum [email protected]
1
Cancer Prevention & Control Program, University of Hawaii Cancer Center, Honolulu, HI, USA
2
Biomedical Informatics, Columbia University, New York, NY, USA
3
Dissemination & Training Division, VA Palo Alto Health Care System, Livermore, CA, USA
4
Cancer Support Community, Washington, DC, USA
5
Alberta Health Services, Calgary, Alberta, Canada
6
Cumming School of Medicine, Department of Oncology, University of Calgary, Calgary, Alberta, Canada
than the machine learning approach when removing the prosocial emotions (p = 0.6), the definitions of the rules were no longer changed and each new transcript was assigned to two coders.
Method
Coding Rules If emotion was identified in a given sentence, raters were asked to identify the specific emotions expressed. Twenty-four specific emotions were originally coded, which included interest, validation, affection, gratitude, contentment/ peacefulness, excitement, pride, humor, tense humor, tension, fear, sadness, frustration, direct anger, contempt, domineering, belligerence, defensiveness, disgust, stonewalling, whining, shame, guilt, and embarrassment. Following the theory behind the SPAFF-C coding system (Giese-Davis et al., 2005), some of the codes are meant to capture a more direct expression of a given negative emotion, with other codes capturing more defensive or indirect types of emotional expression (e.g., defensiveness, constrained anger). There was a process of shaping up coding accuracy, where, for 4 months, coders spent approximate
Data Loading...
