On robust randomized neural networks for regression: a comprehensive review and evaluation
- PDF / 1,513,337 Bytes
- 20 Pages / 595.276 x 790.866 pts Page_size
- 45 Downloads / 219 Views
(0123456789().,-volV)(0123456789(). ,- volV)
REVIEW
On robust randomized neural networks for regression: a comprehensive review and evaluation Ananda L. Freire1 • Ajalmar R. Rocha-Neto1 • Guilherme A. Barreto2 Received: 13 February 2019 / Accepted: 2 May 2020 Ó Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract Data from real-world regression problems are quite often contaminated with outliers. In order to efficiently handle such undesirable samples, robust parameter estimation methods have been incorporated into randomized neural network (RNN) models, usually replacing the ordinary least squares (OLS) method. Despite recent successful applications to outliercontaminated scenarios, significant issues remain unaddressed in the design of reliable outlier-robust RNN models for regression tasks. For example, the number of hidden neurons impacts directly on the norm of the estimated output weights, since the OLS method will rely on an ill-conditioned hidden-layer output matrix. Another design concern involves the high sensitivity of RNNs to the randomization of the hidden layer weights, an issue that can be suitably handled, e.g., by intrinsic plasticity techniques. Bearing these concerns in mind, we describe several ideas introduced in previous works concerning the design of RNN models that are both robust to outliers and numerically stable. A comprehensive evaluation of their performances is carried out across several benchmarking regression datasets taking into account accuracy, weight norms, and training time as figures of merit. Keywords Randomized neural networks Robustness Outliers Numerical stability Regularization
1 Introduction Modeling real-world problems from data are seldom errorfree. The data are usually contaminated with noise, reflecting either inaccuracy in measuring the observations or the intrinsic stochastic nature of the underlying process. It may even contain gross observation errors, generically referred to as outliers, occurring due to equipment malfunction, replacement of missing values with zeros, and wrong decimal point placement [9, 20]. Commonly, the state-of-the-art learning algorithms rely on the ordinary & Guilherme A. Barreto [email protected] Ananda L. Freire [email protected] Ajalmar R. Rocha-Neto [email protected] 1
Department of Computer Science, Federal Institute of Ceara´, Fortaleza, Brazil
2
Department of Teleinformatics Engineering, Federal University of Ceara´ (UFC), Fortaleza, Ceara´, Brazil
least squares (OLS) criterion for parameter estimation. This procedure makes strong assumptions on the statistical behavior of random errors, such as Gaussianity, stationarity, and statistical independence. As a consequence, the OLS criterion leads to algorithms whose performances are strongly influenced by outliers, significantly reducing the accuracy and, hence, the reliability of the learned model [34]. In order to handle outliers without resorting to automatic data cleaning procedures, many authors have been modifying supervised learning rul
Data Loading...
