AutoRWN: automatic construction and training of random weight networks using competitive swarm of agents
- PDF / 2,421,928 Bytes
- 18 Pages / 595.276 x 790.866 pts Page_size
- 92 Downloads / 152 Views
(0123456789().,-volV)(0123456789(). ,- volV)
ORIGINAL ARTICLE
AutoRWN: automatic construction and training of random weight networks using competitive swarm of agents Mohammed Eshtay1 • Hossam Faris2
•
Ali Asghar Heidari3,4 • Ala’ M. Al-Zoubi5 • Ibrahim Aljarah2
Received: 29 August 2019 / Accepted: 2 September 2020 Ó Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract Random Weight Networks have been extensively used in many applications in the last decade because it has many strong features such as fast learning and good generalization performance. Most of the traditional training techniques for Random Weight Networks randomly select the connection weights and hidden biases and thus suffer from local optima stagnation and degraded convergence. The literature shows that stochastic population-based optimization techniques are well regarded and reliable alternative for Random Weight Networks optimization because of high local optima avoidance and flexibility. In addition, many practitioners and non-expert users find it difficult to set the other parameters of the network like the number of hidden neurons, the activation function, and the regularization factor. In this paper, an approach for training Random Weight Networks is proposed based on a recent variant of particle swarm optimization called competitive swarm optimization. Unlike most of Random Weight Networks training techniques, which are used to optimize only the input weights and hidden biases, the proposed approach will automatically tune the weights, biases, the number of hidden neurons, and regularization factor as well as the embedded activation function in the network, simultaneously. The goal is to help users to effectively identify a proper structure and hyperparameter values to their applications while obtaining reasonable prediction results. Twenty benchmark classification datasets are used to compare the proposed approach with different types of basic and hybrid Random Weight Network-based models. The experimental results on the benchmark datasets show that the reasonable classification results can be obtained by automatically tuning the hyperparameters using the proposed approach. Keywords Classification Random weight networks Activation functions Competitive swarm optimizer
3
& Hossam Faris [email protected]
School of Surveying and Geospatial Engineering, College of Engineering, University of Tehran, Tehran, Iran
4
Mohammed Eshtay [email protected]
Department of Computer Science, School of Computing, National University of Singapore, Singapore, Singapore
5
School of Informatics and Telecommunications Engineering, University of Granada, Granada, Spain
Ali Asghar Heidari [email protected]; [email protected]; [email protected]; [email protected] Ala’ M. Al-Zoubi [email protected]; [email protected]; [email protected] Ibrahim Aljarah [email protected] 1
ASAC, Luminus Technical University College, Amman, Jordan
2
King Abdullah II School for Information Technology, The University of Jord
Data Loading...
