Inverse partitioned matrix-based semi-random incremental ELM for regression
- PDF / 938,379 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 23 Downloads / 233 Views
(0123456789().,-volV)(0123456789(). ,- volV)
EXTREME LEARNING MACHINE AND DEEP LEARNING NETWORKS
Inverse partitioned matrix-based semi-random incremental ELM for regression Guoqiang Zeng1
•
Fenxi Yao1 • Baihai Zhang1
Received: 20 August 2018 / Accepted: 1 June 2019 Ó Springer-Verlag London Ltd., part of Springer Nature 2019
Abstract Incremental extreme learning machine has been verified that it has the universal approximation capability. However, there are two major issues lowering its efficiency: one is that some ‘‘random’’ hidden nodes are inefficient which decrease the convergence rate and increase the structural complexity, the other is that the final output weight vector is not the minimum norm least-squares solution which decreases the generalization capability. To settle these issues, this paper proposes a simple and efficient algorithm in which the parameters of even hidden nodes are calculated by fitting the residual error vector in the previous phase, and then, all existing output weights are recursively updated based on inverse partitioned matrix. The algorithm can reduce the inefficient hidden nodes and obtain a preferable output weight vector which is always the minimum norm least-squares solution. Theoretical analyses and experimental results show that the proposed algorithm has better performance on convergence rate, generalization capability and structural complexity than other incremental extreme learning machine algorithms. Keywords Extreme learning machine The minimum norm least-squares solution Inefficient nodes Convergence rate Inverse partitioned matrix
1 Introduction Feedforward neural networks (FNNs) have received much attention and been used in many fields due to its three advantages [1–3]: (i) the capability of approximation of linear and nonlinear mapping; (ii) the capability of learning complex input and output dynamically and (iii) the simplicity of learning rules for training. As a basic type of FNNs, the single hidden-layer feedforward networks (SLFNs) have occupied a large proportion and been investigated thoroughly [4–6]. SLFNs can fit any N random
& Guoqiang Zeng [email protected] Fenxi Yao [email protected] Baihai Zhang [email protected] 1
School of Automation, Beijing Institute of Technology, Beijing 100081, People’s Republic of China
samples with arbitrary small degree using at most N hidden nodes and almost any nonlinear activation function when they can freely tune parameters [4, 7]. However, there are some limitations in conventional learning methods for training FNNs including: slow learning speed, trivial human intervention, and complicated learning algorithms. To make algorithms get rid of tuning parameters, Huang et al. [8–10] have developed some new interpolation and universal approximation theories. Based on these theories, the extreme learning machine (ELM) which did not require tuning parameters was proposed by Huang et al. Thereafter, many researches have been done to verify in simulations and real applications (computer vision [11, 12], ima
Data Loading...
