The Big Data razor
- PDF / 397,385 Bytes
- 20 Pages / 439.642 x 666.49 pts Page_size
- 25 Downloads / 209 Views
(2020) 10:22
PAPER IN PHILOSOPHY OF TECHNOLOGY AND INFORMATION
The Big Data razor 1,2 ´ Ezequiel Lopez-Rubio
Received: 29 July 2019 / Accepted: 24 March 2020 / © Springer Nature B.V. 2020
Abstract Classic conceptions of model simplicity for machine learning are mainly based on the analysis of the structure of the model. Bayesian, Frequentist, information theoretic and expressive power concepts are the best known of them, which are reviewed in this work, along with their underlying assumptions and weaknesses. These approaches were developed before the advent of the Big Data deluge, which has overturned the importance of structural simplicity. The computational simplicity concept is presented, and it is argued that it is more encompassing and closer to actual machine learning practices than the classic ones. In order to process the huge datasets which are commonplace nowadays, the computational complexity of the learning algorithm is the decisive factor to assess the viability of a machine learning strategy, while the classic accounts of simplicity play a surrogate role. Some of the desirable features of computational simplicity derive from its reliance on the learning system concept, which integrates key aspects of machine learning that are ignored by the classic concepts. Moreover, computational simplicity is directly associated with energy efficiency. In particular, the question of whether the maximum possibly achievable predictive accuracy should be attained, no matter the economic cost of the associated energy consumption pattern, is considered. Keywords Model simplicity · Machine learning · Bayesianism · Information theory · Energy efficiency
1 Introduction Automated model selection is one of the most relevant features of machine learning. It gives the scientist a powerful tool to quantitatively assess the merits of several Ezequiel L´opez-Rubio
[email protected] 1
Departamento de Lenguajes y Ciencias de la Computaci´on, Universidad de M´alaga (UMA), Bulevar Louis Pasteur 35, M´alaga, 29071, Spain
2
Departamento de L´ogica, Historia y Filosof´ıa de la Ciencia, Universidad Nacional de Educaci´on a Distancia (UNED), Paseo de Senda del Rey 7, Madrid, 28040, Spain
22
Page 2 of 20
European Journal for Philosophy of Science
(2020) 10:22
possible models in the light of their predictive performance when fitted to large volumes of data. In the current Big Data age, many scientific and engineering endeavors involve the execution of machine learning software to obtain a fitted model, with little or no human intervention. This calls for an analysis of the criteria which are employed in such software to choose one model over another. Simplicity is often employed to justify these selections, but the concept of simplicity has different meanings depending on the school of thought that a machine learning practitioner adheres to. Here we aim to explain such differences and their associated underlying assumptions about the goals of model selection. Furthermore, we describe and discuss the computational concept of
Data Loading...
