Word-character attention model for Chinese text classification
- PDF / 3,864,394 Bytes
- 17 Pages / 595.276 x 790.866 pts Page_size
- 40 Downloads / 237 Views
ORIGINAL ARTICLE
Word-character attention model for Chinese text classification Xue Qiao1 · Chen Peng1 · Zhen Liu1 · Yanfeng Hu1 Received: 12 September 2018 / Accepted: 18 February 2019 © Springer-Verlag GmbH Germany, part of Springer Nature 2019
Abstract Recent progress in applying neural networks to image classification has motivated the exploration of their applications to text classification tasks. Unlike the majority of these researches devoting to English corpus, in this paper, we focus on Chinese text, which is more intricate in semantic representations. As the basic unit of Chinese words, character plays a vital role in Chinese linguistic. However, most existing Chinese text classification methods typically regard word features as the basic unit of text representation but ignore the beneficial performance of character features. Besides, existing approaches compress the entire word features into a semantic representation, without considering attention mechanism which allows for capturing salient features. To tackle these issues, we propose the word-character attention model (WCAM) for Chinese text classification. This WCAM approach integrates two levels of attention models: word-level attention model captures salient words which have closer semantic relationship to the text meaning, and character-level attention model selects discriminative characters of text. Both are jointly employed to learn representation of texts. Meanwhile, the word-character constraint model and character alignment are introduced in our proposed approach to ensure the highly representative of selected characters as well as enhance their discrimination. Both are jointly employed to exploit the subtle and local differences for distinguishing the text classes. Extensive experiments on two benchmark datasets demonstrate that our WCAM approach achieves comparable or even better performance than the state-of-the-art methods for Chinese text classification. Keywords Chinese text classification · Attention mechanism · Word-character attention · Word-character constraint
1 Introduction Previously, main approaches of text classification focus on text representations and classification methods. A number of traditional methods are applied to text classification, including Naive Bayes model [1], k-nearest neighbors algorithm [2], expectation maximization algorithm [3], support vector machine (SVM) model [4], and back-propagation neural networks [5]. However, the difficulty of feature engineering [6, 7] is regarded as a challenge in traditional text classification. In recent years, thanks to the rapid development of deep learning methods and artificial intelligence, a lot of remarkable results have been achieved in Chinese text classification [8]. Different from traditional Chinese text classification approaches, deep learning methods are proposed to learn word embeddings [9] by deep neural network models and * Xue Qiao [email protected] 1
Institute of Electronics, Chinese Academy of Sciences, Suzhou, Suzhou 215123, China
to perform co
Data Loading...
