Timeliness online regularized extreme learning machine
- PDF / 2,216,899 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 52 Downloads / 195 Views
ORIGINAL ARTICLE
Timeliness online regularized extreme learning machine Xiong Luo1,2 • Xiaona Yang1,2 • Changwei Jiang1,2 • Xiaojuan Ban1,2
Received: 30 September 2015 / Accepted: 2 May 2016 Springer-Verlag Berlin Heidelberg 2016
Abstract To improve the learning performance, a novel online sequential extreme learning machine (ELM) algorithm for single-hidden layer feedforward networks is proposed with regularization mechanism in a unified framework. The proposed algorithm is called timeliness online regularized extreme learning machine (TORELM). Like the timeliness managing extreme learning machine which improves online sequential extreme learning machine by incorporating timeliness management scheme into ELM approach for the incremental training samples, TORELM also analyzes the training data one-byone or chunk-by-chunk (a block of data) with fixed or varied chunk size under the similar framework. Meanwhile, the newly incremental training data could be prior to the historical data by maximizing the contribution of the newly increasing training data, since in some cases it may be more feasible that the incremental data can contribute reasonable weights to represent the current system situation in accordance with the practical analysis. Furthermore, in consideration of the disproportion between empirical risk and structural risk in some traditional learning methods, we add regularization technique to the timeliness scheme of TORELM through the use of a weight factor to balance them to achieve better generalization performance. Hence, TORELM has its unique feature of higher generalization
& Xiong Luo [email protected] 1
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, People’s Republic of China
2
Beijing Key Laboratory of Knowledge Engineering for Materials Science, Beijing 100083, People’s Republic of China
capability in most cases with a small testing error while implementing online sequential learning. In addition, this algorithm is still competitive in training time compared with other schemes. Finally, the simulation results show that TORELM can achieve higher learning accuracy and better stability than other ELM-based machine learning methods. Keywords Incremental learning Extreme learning machine Timeliness online regularized extreme learning machine Regularization
1 Introduction Neural networks (NNs) have been widely used in machine learning due to their ability of solving those problems that classical techniques are not able to deal with. Among the available NN-based machine learning algorithms, extreme learning machine (ELM) for single-hidden layer feedforward network (SLFN) has attracted much attention because of its quickness and simplicity [1–6]. More recently, with the scale increase of data set in the Big Data era, there is a growing interest in the study of algorithms regarding the high-speed signal processing as well as fast and efficient feature learning [7]. ELM as one of the leading trends for fast learning and optimiza
Data Loading...
