Regularising LSTM classifier by transfer learning for detecting misogynistic tweets with small training set
- PDF / 1,366,187 Bytes
- 26 Pages / 439.37 x 666.142 pts Page_size
- 12 Downloads / 160 Views
Regularising LSTM classifier by transfer learning for detecting misogynistic tweets with small training set Md Abul Bashar1 · Richi Nayak1 · Nicolas Suzor2 Received: 4 August 2019 / Accepted: 6 June 2020 © Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract Supervised machine learning methods depend highly on the quality of the training dataset and the underlying model. In particular, neural network models, that have shown great success in dealing with natural language problems, require a large dataset to learn a vast number of parameters. However, it is not always easy to build a large (labelled) dataset. For example, due to the complex nature of tweets and the manual labour involved, it is hard to create a large Twitter data set with the misogynistic label. In this paper, we propose to regularise a long short-term memory (LSTM) classifier using a pretrained LSTM-based language model (LM) to build an accurate classification model with a small training set. We explain transfer learning (TL) with a Bayesian interpretation and show that TL can be viewed as an uncertainty regularisation technique in Bayesian inference. We show that a LM pre-trained on a sequence of general to task-specific domain datasets can be used to regularise a LSTM classifier effectively when a small training dataset is available. Empirical analysis with two small Twitter datasets reveals that an LSTM model trained in this way can outperform the state-of-the-art classification models. Keywords Misogynistic tweet · Transfer learning · LSTM · Small dataset · Overfitting
1 Introduction Incidents of abuse, hate and harassment have grown with the proliferated use of social media platforms (e.g. Twitter, Facebook, Instagram) [14]. Online abuse targeting women (i.e. name-calling, offensive language, threats of harm or sexual violence, intimidation, shaming, and silencing) has become common [2]. An automated method to identify
* Md Abul Bashar [email protected] Richi Nayak [email protected] Nicolas Suzor [email protected] 1
School of Computer Science, Queensland University of Technology, Brisbane, Australia
2
School of Law, Queensland University of Technology, Brisbane, Australia
13
Vol.:(0123456789)
M. A. Bashar et al.
misogynistic tweets can help in ongoing efforts to develop effective remedies [53]. A key challenge in misogynistic tweet detection is understanding the context of a tweet. The complex and noisy nature of tweets makes it difficult. Classification of a tweet based on the presence of typical misogynistic keywords using a lexical detection approach results in poor accuracy [11, 60]. Simple methods such as bagof-words, part-of-speech (POS) and belief propagation perform poorly due to the noise present in the data [60]. An algorithm should identify patterns, sequences, and other complex features that can be correlated with misogynistic tweets despite the noise. In the supervised machine learning setting, traditional algorithms such as Random Forest [32] and Support Vector Machines [21] rely on manual pro
Data Loading...
