A Maximum Likelihood Approach to Least Absolute Deviation Regression
- PDF / 792,307 Bytes
- 8 Pages / 600 x 792 pts Page_size
- 5 Downloads / 235 Views
A Maximum Likelihood Approach to Least Absolute Deviation Regression Yinbo Li Department of Electrical and Computer Engineering, University of Delaware, Newark, DE 19716 3130, USA Email: [email protected]
Gonzalo R. Arce Department of Electrical and Computer Engineering, University of Delaware, Newark, DE 19716 3130, USA Email: [email protected] Received 7 October 2003; Revised 22 December 2003 Least absolute deviation (LAD) regression is an important tool used in numerous applications throughout science and engineering, mainly due to the intrinsic robust characteristics of LAD. In this paper, we show that the optimization needed to solve the LAD regression problem can be viewed as a sequence of maximum likelihood estimates (MLE) of location. The derived algorithm reduces to an iterative procedure where a simple coordinate transformation is applied during each iteration to direct the optimization procedure along edge lines of the cost surface, followed by an MLE of location which is executed by a weighted median operation. Requiring weighted medians only, the new algorithm can be easily modularized for hardware implementation, as opposed to most of the other existing LAD methods which require complicated operations such as matrix entry manipulations. One exception is Wesolowsky’s direct descent algorithm, which among the top algorithms is also based on weighted median operations. Simulation shows that the new algorithm is superior in speed to Wesolowsky’s algorithm, which is simple in structure as well. The new algorithm provides a better tradeoff solution between convergence speed and implementation complexity. Keywords and phrases: least absolute deviation, linear regression, maximum likelihood estimation, weighted median filters.
1. INTRODUCTION Linear regression has long been dominated by least squares (LS) techniques, mostly due to their elegant theoretical foundation and ease of implementation. The assumption in this method is that the model has normally distributed errors. In many applications, however, heavier-than-Gaussian tailed distributions may be encountered, where outliers in the measurements may easily ruin the estimates [1]. To address this problem, robust regression methods have been developed so as to mitigate the influence of outliers. Among all the approaches to robust regression, the least absolute deviations (LADs) method, or L1 -norm, is considered conceptually the simplest one since it does not require a “tuning” mechanism like most of other robust regression procedures. As a result, LAD regression has drawn significant attentions in statistics, finance, engineering, and other applied sciences as detailed in a series of studies on L1 -norm methods [2, 3, 4, 5]. LAD regression is based on the assumption that the model has Laplacian distributed errors. Unlike the LS approach though, LAD regression has no closed-form solution, hence numerical and iterative algorithms must be resorted to.
Surprisingly to many, the LAD regression method first suggested by Boscovich (1757) and studied by Laplac
Data Loading...
