Assigning Different Weights to Feature Values in Naive Bayes
Assigning weights in features has been an important topic in some classification learning algorithms. While the current weighting methods assign a weight to each feature, in this paper, we assign a different weight to the values of each feature. The perfo
- PDF / 154,884 Bytes
- 9 Pages / 439.37 x 666.142 pts Page_size
- 62 Downloads / 337 Views
Abstract. Assigning weights in features has been an important topic in some classification learning algorithms. While the current weighting methods assign a weight to each feature, in this paper, we assign a different weight to the values of each feature. The performance of naive Bayes learning with value-based weighting method is compared with that of some other traditional methods for a number of datasets. Keywords: Feature weighting Kullback-Leibler
1
·
Feature selection
·
Naive Bayes
·
Introduction
In some classifiers, the algorithms operate under the implicit assumption that all features are of equal value as far as the classification problem is concerned. However, when irrelevant and noisy features influence the learning task to the same degree as highly relevant features, the accuracy of the model is likely to deteriorate. Since the assumption that all features are equally important hardly holds true in real world application, there have been some attempts to relax this assumption in classification. Zheng and Webb [1] provide a comprehensive overview of work in this area. The first approach for relaxing this assumption is to combine feature subset selection with classification learning. It is to combine a learning method with a preprocessing step that eliminates redundant features from the data. Feature selection methods usually adopt a heuristic search in the space of feature subsets. Since the number of distinct feature subsets grows exponentially, it is not reasonable to do an exhaustive search to find optimal feature subsets. Another major way to help mitigate this weakness, feature independence assumption, is to assign weights to important features in classification. Since features do not play the same role in many real world applications, some of them are more important than others. Therefore, a natural way to extend classification learning is to assign each feature different weight to relax the conditional independence assumption. Feature weighting is a technique used to approximate the optimal degree of influence of individual features using a training set. While feature selection methods assign 0/1 values as the weights of features, feature weighting is more flexible than feature subset selection by assigning continuous weights. c Springer Nature Singapore Pte Ltd. 2016 M.W. Berry et al. (Eds.): SCDS 2016, CCIS 652, pp. 171–179, 2016. DOI: 10.1007/978-981-10-2777-2 15
172
C.-H. Lee
When successfully applied, important features are attributed a high weight value, whereas unimportant features are given a weight value close to zero. There have been many feature weighting methods proposed in the machine learning literature, mostly in the domain of nearest neighbor algorithms [2]. They have significantly improved the performance of classification algorithms. In this paper, we propose a new paradigm of weighting method, called value weighting method. While the current weighting methods assign a weight to each feature, we assign a weight to each feature value. Therefore, the value weighting method is a more fin
Data Loading...
