An interpretable regression approach based on bi-sparse optimization
- PDF / 1,497,954 Bytes
- 26 Pages / 595.276 x 790.866 pts Page_size
- 62 Downloads / 243 Views
An interpretable regression approach based on bi-sparse optimization Zhiwang Zhang 1,2 & Guangxia Gao 3 & Tao Yao 1,2 & Jing He 4 & Yingjie Tian 5
# Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Given the increasing amounts of data and high feature dimensionalities in forecasting problems, it is challenging to build regression models that are both computationally efficient and highly accurate. Moreover, regression models commonly suffer from low interpretability when using a single kernel function or a composite of multi-kernel functions to address nonlinear fitting problems. In this paper, we propose a bi-sparse optimization-based regression (BSOR) model and corresponding algorithm with reconstructed row and column kernel matrices in the framework of support vector regression (SVR). The BSOR model can predict continuous output values for given input points while using the zero-norm regularization method to achieve sparse instance and feature sets. Experiments were run on 16 datasets to compare BSOR to SVR, linear programming SVR (LPSVR), least squares SVR (LSSVR), multi-kernel learning SVR (MKLSVR), least absolute shrinkage and selection operator regression (LASSOR), and relevance vector regression (RVR). BSOR significantly outperformed the other six regression models in predictive accuracy, identification of the fewest representative instances, selection of the fewest important features, and interpretability of results, apart from its slightly high runtime. Keywords Data mining . Multi-kernel learning . Sparse learning . Zero-norm regularization . Support vector regression
1 Introduction Regression is an important data mining technique that is known to fit input points from training data with high accuracy. Value prediction is a common application of regression that can be found in many domains, such as finance, telecommunication, economics and management, power and energy, web customer management, industrial production, and scientific computing [40, 59]. Regression predicts values by constructing functions that can estimate the relationship between
* Zhiwang Zhang [email protected] 1
School of Information and Electrical Engineering, Ludong University, Yantai 264025, China
2
Yantai Research Institute of New Generation Information Technology, Southwest Jiaotong University, Yantai 264000, China
3
Shandong Technology and Business University, Yantai 264005, China
4
Swinburne University of Technology, Melbourne, VIC 3122, Australia
5
Research Center on Fictitious Economy and Data Science, Chinese Academy of Sciences, Beijing 100190, China
independent and dependent variables. Many regression methods have been proposed for value prediction [3, 6]. These include linear regression [22], nonlinear regression [59], polynomial regression [13], ridge regression [28], stepwise regression [25], quantile regression [44], least angle regression [27], lasso regression [77], elastic net regression [89], neural networks, and support vector regression (SVR) [5, 68]. SVR is considered an effect
Data Loading...
