A study of model and hyper-parameter selection strategies for classifier ensembles: a robust analysis on different optim
- PDF / 651,617 Bytes
- 15 Pages / 595.276 x 790.866 pts Page_size
- 22 Downloads / 155 Views
(0123456789().,-volV)(0123456789(). ,- volV)
A study of model and hyper-parameter selection strategies for classifier ensembles: a robust analysis on different optimization algorithms and extended results Antonino A. Feitosa-Neto1 • Joa˜o C. Xavier-Ju´nior1
•
Anne M. P. Canuto1
•
Alexandre C. M. Oliveira2
Accepted: 16 October 2020 Ó Springer Nature B.V. 2020
Abstract It is well known that machine learning (ML) techniques have been playing an important role in several real world applications. However, one of the main challenges is the selection of the most accurate technique to be used in a specific application. In the classification context, for instance, two main approaches can be applied, model selection and hyperparameter selection. In the first approach, the best classification algorithm is selected for a given input dataset, by doing a heuristic search in a large space of candidate classification algorithms and their corresponding hyper-parameter settings. As the main focus of this approach is the selection of the classification algorithms, it is referred to as model selection and they are also called automated machine learning (Auto-ML). The second approach defines one classification system and performs an extensive search to select the best hyper-parameters for this model. In this paper, we perform a wide and robust comparative analysis of both approaches for Classifier Ensembles. In this analysis, two methods of the first approach (Auto-WEKA and H2 O) are compared to four methods of the second approach (Genetic Algorithm, Particle Swarm Optimization, Tabu Search and GRASP). The main aim is to determine which of these techniques generate more accurate Classifier Ensembles, given a time constraint. Additionally, an empirical analysis will be conducted with 21 classification datasets for evaluating the performance of the aforementioned techniques. Our findings indicate that the use of a hyperparameter selection method provides the most accurate classifier ensembles, but this improvement was not detected by the statistical test. Keywords Hyper-parameter selection Model selection Classifier ensembles
1 Introduction Machine Learning (ML) techniques have been widely applied in several domain applications. Nonetheless, there is no Machine Learning algorithm better than another when & Joa˜o C. Xavier-Ju´nior [email protected] Antonino A. Feitosa-Neto [email protected] Anne M. P. Canuto [email protected] Alexandre C. M. Oliveira [email protected] 1
Federal University of Rio Grande do Norte, Natal, RN, Brazil
2
Federal University of Maranha˜o, Sa˜o Luiz, MA, Brazil
considering the entire domain of possible applications. This problem is stated in the No Free Lunch theorem (Wolpert 1996). In the Classification context, this problem has been tackled using two main approaches, model selection and hyper-parameter selection. In the first approach, a selection technique can be applied to ML algorithms and their hyperparameter settings aiming to maximize performance in classification task
Data Loading...
