Minimizers of Sparsity Regularized Huber Loss Function
- PDF / 656,988 Bytes
- 29 Pages / 439.37 x 666.142 pts Page_size
- 5 Downloads / 223 Views
Minimizers of Sparsity Regularized Huber Loss Function Deniz Akkaya1
· Mustafa Ç. Pınar1
Received: 27 January 2020 / Accepted: 2 September 2020 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract We investigate the structure of the local and global minimizers of the Huber loss function regularized with a sparsity inducing L0 norm term. We characterize local minimizers and establish conditions that are necessary and sufficient for a local minimizer to be strict. A necessary condition is established for global minimizers, as well as non-emptiness of the set of global minimizers. The sparsity of minimizers is also studied by giving bounds on a regularization parameter controlling sparsity. Results are illustrated in numerical examples. Keywords Sparse solution of linear systems · Regularization · local minimizer · Global minimizer · Huber loss function · L0-norm Mathematics Subject Classification 15A29 · 62J05 · 90C26 · 90C46
1 Introduction The search for an approximate and regularized solution (i.e., a solution with some desirable properties such as sparsity) to a possibly inconsistent system of linear equations is a ubiquitous problem in applied mathematics. It aims at finding a solution vector x, that minimizes both objectives (Ax − b, x) with respect to a norm or another appropriate error measure. Here, we use the Huber loss function and the 0 -norm, respectively. The Huber loss function has been used in several engineering applications since the 1970s (for a recent account, see, e.g., [1]) while the search for sparse solutions of
Communicated by Panos M. Pardalos.
B
Mustafa Ç. Pınar [email protected] Deniz Akkaya [email protected]
1
Bilkent University, Ankara, Turkey
123
Journal of Optimization Theory and Applications
linear systems has been a popular topic of the last decade. The purpose of the present paper is to initiate an investigation of a combination of the two. In the world of statistical data analysis, a common assumption is the normality of errors in the measurements. The normality assumption leads to methods yielding closed form solutions and thus is quite convenient. However, many real-world applications in engineering present the modeler with data deviating from the normality assumption. Robust statistics or robust methods in engineering aim at alleviating the effects of departure from normality by being largely immune to its negative ones. One of the proposals for robust statistical procedures was put forward in the 1970s and 1980s by Huber [2]. The so-called Huber loss function (a.k.a. Huber’s M-estimator) coincides with the quadratic error measure up to a range beyond which a linear error measure is adopted. Huber established that the resulting estimator corresponds to a maximum likelihood estimate for a perturbed normal law. The Huber loss function has numerous applications in statistics and engineering as documented among others in the recent monograph by Zoubir et al. [1]. The present paper studies a case where sparsity is incorporated
Data Loading...
