Empirical evaluation and study of text stemming algorithms
- PDF / 1,126,084 Bytes
- 30 Pages / 439.37 x 666.142 pts Page_size
- 91 Downloads / 215 Views
Empirical evaluation and study of text stemming algorithms Abdul Jabbar1 · Sajid Iqbal2 · Manzoor Ilahi Tamimy1 · Shafiq Hussain3 · Adnan Akhunzada1
© Springer Nature B.V. 2020
Abstract Text stemming is one of the basic preprocessing step for Natural Language Processing applications which is used to transform different word forms into a standard root form. For Arabic script based languages, adequate analysis of text by stemmers is a challenging task due to large number of ambigious structures of the language. In literature, multiple performance evaluation metrics exist for stemmers, each describing the performance from particular aspect. In this work, we review and analyze the text stemming evaluation methods in order to devise criteria for better measurement of stemmer performance. Role of different aspects of stemmer performance measurement like main features, merits and shortcomings are discussed using a resource scarce language i.e. Urdu. Through our experiments we conclude that the current evaluation metrics can only measure an average conflation of words regardless of the correctness of the stem. Moreover, some evaluation metrics favor some type of languages only. None of the existing evaluation metrics can perfectly measure the stemmer performance for all kind of languages. This study will help researchers to evaluate their stemmer using right methods. Keywords Natural language processing · Information retrieval · Text mining · Stemming algorithms · Stemmer evaluation methods · Urdu stemming
* Sajid Iqbal [email protected] Abdul Jabbar [email protected] Shafiq Hussain [email protected] Adnan Akhunzada [email protected] 1
Department of Computer Science, COMSATS University Islamabad (CUI), Main campus, Park Road, Tarlai Kalan, Islamabad 45550, Pakistan
2
Department of Computer Science, Bahauddin Zakariya University Multan, Multan, Punjab, Pakistan
3
Bahauddin Zakariya University Multan (Sahiwal Sub-Campus), Multan, Punjab, Pakistan
13
Vol.:(0123456789)
A. Jabbar et al.
1 Introduction Performance evaluation is the primary method to find the effectiveness of algorithms and methods developed to solve different scientific problems. Efficient evaluation methods can boost the applicability of the solution. Performance evaluation methods describe and determine the extent to which the solution can achieve its intended goals. It is an open challenge for computational linguistics researchers to assess the performance of NLP applications (Cambria and White 2014). Existing evaluation methods can be divided between two main categories: intrinsic and extrinsic methods (Gaidhane et al. 2015). In the intrinsic evaluation, performance measure of NLP applications and methods are compared with some gold standard results that are calculated using manual methods. For instance, a stem produced by a stemmer is compared with relevant dictionary stem developed by human experts. Whereas, in the extrinsic evaluation method, the performance is measured directly in any realistic scenario. For ex
Data Loading...
