ALBERT-based fine-tuning model for cyberbullying analysis
- PDF / 837,247 Bytes
- 9 Pages / 595.276 x 790.866 pts Page_size
- 35 Downloads / 190 Views
SPECIAL ISSUE PAPER
ALBERT‑based fine‑tuning model for cyberbullying analysis Jatin Karthik Tripathy1 · S. Sibi Chakkaravarthy1 · Suresh Chandra Satapathy2 · Madhulika Sahoo3 · V. Vaidehi4
© Springer-Verlag GmbH Germany, part of Springer Nature 2020
Abstract With the world’s interaction moving more and more toward using online social media platforms, the advent of cyberbullying has also raised its head. Multiple forms of cyberbullying exist from the more common text based to images or even videos, and this paper will explore the context of textual comments. Even in the niche area of considering only text-based data, several approaches have already been worked upon such as n-grams, recurrent units, convolutional neural networks (CNNs), gated recurrent unit (GRU) and even a combination of the mentioned architectures. While all of these produce workable results, the main point of contention is that true contextual understanding is quite a complex concept. These methods fail due to two simple reasons: (i) lack of large datasets to properly utilize these architectures and (ii) the fact that understanding context requires some mechanism of remembering history that is only present in the recurrent units. This paper explores some of the recent approaches to the difficulties of contextual understanding and proposes an ALBERT-based fine-tuned model that achieves state-of-the-art results. ALBERT is a transformer-based architecture and thus even at its untrained form provides better contextual understanding than other recurrent units. This coupled with the fact that ALBERT is pre-trained on a large corpus allowing the flexibility to use a smaller dataset for fine-tuning as the pre-trained model already has deep understanding of the complexities of the human language. ALBERT showcases high scores in multiple benchmarks such as the GLUE and SQuAD showing that high levels of contextual understanding are inherently present and thus fine-tuning for the specific case of cyberbullying allows to use this to our advantage. With this approach, we have achieved an F1 score of 95% which surpasses current approaches such as the CNN + wordVec, CNN + GRU and BERT implementations. Keywords ALBERT · Fine tuning · Deep learning · wordVec · Gated recurrent unit · GRU · CNN
1 Introduction * S. Sibi Chakkaravarthy [email protected] Jatin Karthik Tripathy [email protected] Suresh Chandra Satapathy [email protected] Madhulika Sahoo [email protected] V. Vaidehi [email protected] 1
School of Computer Science and Engineering, VIT-AP University, Amaravati, Andhra Pradesh, India
2
School of Computer Engineering, KIIT University, Bhubaneshwar, India
3
VIT‑AP Business School, VIT-AP University, Amaravati, Andhra Pradesh, India
4
Mother Teresa Women’s University, Kodaikanal, Tamilnadu, India
The world is evolving as a result of technological progress. The present pandemic situation has put the entire human race into digital beings such as virtual and embodied agents, although not a part of the natura
Data Loading...
