Sparse classification with paired covariates
- PDF / 706,592 Bytes
- 18 Pages / 439.37 x 666.142 pts Page_size
- 67 Downloads / 246 Views
Sparse classification with paired covariates 1· Armin Rauschenberger1,2 · Iuliana Ciocanea-Teodorescu ˘ 3 1 · Renée X. Menezes · Mark A. van de Wiel1,4 Marianne A. Jonker
Received: 22 October 2018 / Revised: 12 July 2019 / Accepted: 12 October 2019 © The Author(s) 2019
Abstract This paper introduces the paired lasso: a generalisation of the lasso for paired covariate settings. Our aim is to predict a single response from two high-dimensional covariate sets. We assume a one-to-one correspondence between the covariate sets, with each covariate in one set forming a pair with a covariate in the other set. Paired covariates arise, for example, when two transformations of the same data are available. It is often unknown which of the two covariate sets leads to better predictions, or whether the two covariate sets complement each other. The paired lasso addresses this problem by weighting the covariates to improve the selection from the covariate sets and the covariate pairs. It thereby combines information from both covariate sets and accounts for the paired structure. We tested the paired lasso on more than 2000 classification problems with experimental genomics data, and found that for estimating sparse but predictive models, the paired lasso outperforms the standard and the adaptive lasso. The R package palasso is available from cran. Keywords Prediction · Sparsity · Lasso regression · Paired data Mathematics Subject Classification 62-04 · 62J12 · 62J07 · 62H30 · 62P10
Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11634019-00375-6) contains supplementary material, which is available to authorized users.
B
Mark A. van de Wiel [email protected]
1
Department of Epidemiology and Biostatistics, Amsterdam UMC, VU University Amsterdam, Amsterdam, The Netherlands
2
Luxembourg Centre for Systems Biomedicine, University of Luxembourg, Esch-sur-Alzette, Luxembourg
3
Department for Health Evidence, Radboud University Medical Center, Nijmegen, The Netherlands
4
MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
123
A. Rauschenberger et al.
1 Background Lasso regression has become a popular method for variable selection and prediction. Among other things, it extends generalised linear models to settings with more covariates than samples. The lasso shrinks the coefficients towards zero, setting some coefficients equal to zero. Compared to the standard lasso, the adaptive lasso shrinks large coefficients less. In high-dimensional spaces, most coefficients are set to zero, since the number of non-zero coefficients is bounded by the sample size (Zou and Hastie 2005). It is possible to decrease the maximum number of non-zero coefficients, and estimate the coefficients given this sparsity constraint. By including fewer covariates, the resulting model may be less predictive but more practical and interpretable. Given an efficient algorithm that produces the regularisation path, we can extract models of different sizes without increasing the computation
Data Loading...
