Forward variable selection for sparse ultra-high-dimensional generalized varying coefficient models
- PDF / 2,152,723 Bytes
- 29 Pages / 439.37 x 666.142 pts Page_size
- 16 Downloads / 180 Views
Forward variable selection for sparse ultra‑high‑dimensional generalized varying coefficient models Toshio Honda1 · Chien‑Tong Lin2 Received: 8 June 2020 / Accepted: 9 September 2020 © Japanese Federation of Statistical Science Associations 2020
Abstract In this paper, we propose forward variable selection procedures for feature screening in ultra-high-dimensional generalized varying coefficient models. We employ regression spline to approximate coefficient functions and then maximize the loglikelihood to select an additional relevant covariate sequentially. If we decide we do not significantly improve the log-likelihood any more by selecting any new covariates from our stopping rule, we terminate the forward procedures and give our estimates of relevant covariates. The effect of the size of the current model has been overlooked in stopping rules for sequential procedures for high-dimensional models. Our stopping rule takes into account the size of the current model suitably. Our forward procedures have screening consistency and some other desirable properties under regularity conditions. We also present the results of numerical studies to show their good finite sample performances. Keywords B-spline basis · Forward procedure · Maximum likelihood · Screening consistency · Stopping rule · Varying coefficient model
Honda’s research was supported in part by JSPS KAKENHI Grant Number JP 20K11705, Japan. Lin’s research was supported in part by the Science Vanguard Research Program of the Ministry of Science and Technology, Taiwan. Electronic supplementary material The online version of this article (https://doi.org/10.1007/s4208 1-020-00090-z) contains supplementary material, which is available to authorized users. * Toshio Honda [email protected]‑u.ac.jp Chien‑Tong Lin [email protected] 1
Graduate School of Economics, Hitotsubashi University, Tokyo 186‑8601, Japan
2
Institute of Statistics, National Tsing Hua University, Hsinchu 30013, Taiwan
13
Vol.:(0123456789)
Japanese Journal of Statistics and Data Science
1 Introduction Suppose we have n i.i.d. observations (Yi , Xi , Zi ), i = 1, … , n, where Yi is a real response variable and (Xi , Zi ) is a covariate vector, such that Xi = (Xi1 , … , Xip )T ∈ Rp , Xi1 ≡ 1 , and Zi is an index variable satisfying Zi ∈ [0, 1] . Here, we assume that Yi follows a high-dimensional sparse generalized varying coefficient model (GVCM): the conditional density on (Xi , Zi ) w.r.t. some known 𝜎-finite measure 𝜈 is: given by
f (y|x, z) = exp{yxT g∗ (z) − b(xT g∗ (z)) + c(y)},
(1)
where b(𝜃) and c(y) are known functions and:
g∗ (z) = (g∗1 (z), … , g∗p (z))T = (g∗j (z))j∈{1,…,p} is a vector of p unknown smooth functions. We consider the setup where p is extremely large compared to n, but g∗ (z) is sparse, i.e., most of g∗j (z) are irrelevant. We denote the set of relevant indecies by M and |M| is very small compared to p, where |S| is the number of the elements of S ⊂ {1, … , p}. In such high-dimensional settings, even if the dimension of Xi , p, is very large compare
Data Loading...
