Malay Word Stemmer to Stem Standard and Slang Word Patterns on Social Media
Word stemmer is a text preprocessing tool used in many artificial intelligence applications such as text mining, text categorization and information retrieval. It is used to stem derived words into their respective root words. Many researchers have propos
- PDF / 514,514 Bytes
- 10 Pages / 439.37 x 666.142 pts Page_size
- 25 Downloads / 145 Views
Strategic Research, CyberSecurity Malaysia, The Mines Resort City, 43300 Seri Kembangan, Malaysia {nizam,amirudin}@cybersecurity.my 2 Faculty of Computing, Universiti Teknologi Malaysia, 81310 Skudai, Johor, Malaysia {aizaini,anazida}@utm.my
Abstract. Word stemmer is a text preprocessing tool used in many artificial intelligence applications such as text mining, text categorization and information retrieval. It is used to stem derived words into their respective root words. Many researchers have proposed word stemmers for Malay language using various stemming approaches. Since the proliferation of social media, there are various word patterns have been used by social media users in which the existing word stemmers do not support in their stemming rules. These word patterns are slang words or informal conversation words which are used in daily conversation. Therefore, this paper proposes the new word stemmer for Malay language that able to stem standard and slang words. This paper also examines the differences between standard words and slang words. The experimental results show that the proposed word stemmer able to stem standard affixation and reduplication words and also stem slang affixation and reduplication words with better stemming accuracy. Keywords: Malay word stemmer word Slang word Social media
Word stemming algorithm
Standard
1 Introduction The advent of Web 2.0 technology creates social media platforms such as social networking, microblogging and video sharing that allow the internet users to create, publish and share their user-generated contents to other internet users without any editorial approval [13]. This scenario has led to information overloaded due to the amount of user generated contents that have been created on social media platform. In the context of Malay language, there are two types of written texts on social media i.e. formal and informal written text documents. Formal text documents are written based on morphological structures of Malay language (standard words) whereas informal text documents are written based on colloquial speech communication (slang words). For instance, the standard affixation word of ‘punyalah’ (belonging) in a formal text document could be written in the © Springer International Publishing Switzerland 2016 Y. Tan and Y. Shi (Eds.): DMBD 2016, LNCS 9714, pp. 391–400, 2016. DOI: 10.1007/978-3-319-40973-3_39
392
M.N. Kassim et al.
informal text documents such as ‘punyelah’, ‘punyalaa’ and ‘punyela’. As a result, the existence of digital Malay text documents that may comprise of derived words (affixation and reduplication words) in the form of colloquial speech conversations on social media. These slang derived words are word pattern variations of affixation and reduplication words such as mngambil (slang affixation word for mengambil), makanannye (slang affixation word for makanannya), kepunyeannye (slang affixation word for kepunyaannya) and burung2 (slang reduplication word for burung-burung). There are also slang derived words with special characters s
Data Loading...
