Ensemble Classification
This chapter is concerned with ensemble classification, i.e. using a set of classifiers to classify unseen data rather than just a single one. The classifiers in the ensemble all predict the correct classification of each unseen instance and their predict
- PDF / 324,269 Bytes
- 12 Pages / 441 x 666 pts Page_size
- 61 Downloads / 290 Views
14.1 Introduction The idea of ensemble classification is to learn not just one classifier but a set of classifiers, called an ensemble of classifiers, and then to combine their predictions for the classification of unseen instances using some form of voting. This is illustrated in Figure 14.1 below. It is hoped that the ensemble will collectively have a higher level of predictive accuracy than any one of the individual classifiers, but that is not guaranteed. The term ensemble learning is often used to mean the same as ensemble classification, but the former is a more general technique where a set of models is learnt that collectively can be applied to solving a problem of potentially any kind, not just classification. The individual classifiers in an ensemble are known as base classifiers. If the base classifiers are all of the same kind (e.g. decision trees) the ensemble is known as homogeneous. Otherwise it is known as heterogeneous. A simple form of ensemble classification algorithm is: 1. Generate N classifiers for a given dataset 2. For an unseen instance X a) Compute the predicted classification of X for each of the N classifiers b) Select the classification that is most frequently predicted. This is a majority voting model where each time a classifier predicts a particular classification for an unseen instance it counts as one ‘vote’ for that © Springer-Verlag London Ltd. 2016 M. Bramer, Principles of Data Mining, Undergraduate Topics in Computer Science, DOI 10.1007/978-1-4471-7307-6 14
209
210
Principles of Data Mining
classification. With N classifiers in the ensemble there will be a total of N votes and the classification with most votes wins, i.e. is deemed to be the ensemble’s prediction of the correct classification.
Figure 14.1 Ensemble Classification The obvious objection to an ensemble classifier approach is that generating N classifiers takes much longer than only one and this additional effort is only justified if the performance of the ensemble is substantially better than that of just a single classifier. There is no guarantee that this will be the case for a given set of test data and far less so for an individual unseen instance, but intuitively it seems reasonable to believe that N classifiers ‘working together’ have the potential to give better predictive accuracy than one on its own. In practice this is likely to depend on how the classifiers are generated and how their predictions are combined (majority voting or otherwise). In this chapter we will restrict our attention to the homogeneous case, where all the classifiers are of the same kind, say decision trees. There are several ways in which an ensemble can be formed, for example: – N trees generated using the same tree generation algorithm, with different parameter settings, all using the same training data. – N trees generated using the same tree generation algorithm, all with different training data and either with the same or with different parameter settings. – N trees generated using a variety of different tree generation algorithms, either with the same or wi
Data Loading...
