An improved optimization technique using Deep Neural Networks for digit recognition
- PDF / 1,250,405 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 66 Downloads / 214 Views
(0123456789().,-volV)(0123456789(). ,- volV)
METHODOLOGIES AND APPLICATION
An improved optimization technique using Deep Neural Networks for digit recognition T. Senthil1
•
C. Rajan2 • J. Deepika3
Springer-Verlag GmbH Germany, part of Springer Nature 2020
Abstract In the world of information retrieval, recognizing hand-written digits stands as an interesting application of machine learning (deep learning). Though this is already a matured field, a way to recognize digits using an effective optimization using soft computing technique is a challenging task. Training such a system with larger data often fails due to higher computation and storage. In this paper, a recurrent deep neural network with hybrid mini-batch and stochastic Hessian-free optimization (MBSHF) is for accurate and faster convergence of predictions as outputs. A second-order approximation is used for achieving better performance for solving quadratic equations which greatly depends on computation and storage. Also, the proposed technique uses an iterative minimization algorithm for faster convergence using a random initialization though huge additional parameters are involved. As a solution, a convex approximation of MBSHF optimization is formulated and its performance on experimenting with the standard MNIST dataset is discussed. A recurrent deep neural network till a depth of 20 layers is successfully trained using the proposed MBSHF optimization, resulting in a better quality performance in computation and storage. The results are compared with other standard optimization techniques like mini-batch stochastic gradient descent (MBSGD), stochastic gradient descent (SGD), stochastic Hessian-free optimization (SHF), Hessian-free optimization (HF), nonlinear conjugate gradient (NCG). The proposed technique produced higher recognition accuracy of 12.2% better than MBSGD, 27.2% better than SHF, 35.4% better than HF, 40.2% better than NCG and 32% better than SGD on an average when applied to 50,000 testing sample size. Keywords Recurrent deep neural networks Hybrid mini-batch and stochastic Hessian-free optimization Pattern recognition Predictive analysis Classification and validation
1 Introduction Communicated by V. Loia. & T. Senthil [email protected] C. Rajan [email protected] J. Deepika [email protected] 1
Department of Electronics and Communication Engineering, K.S.R Institute for Engineering and Technology, Tiruchengode, Namakkal(DT), Tamilnadu, India
2
Department of Information Technology, K.S.Rangasamy College of Technology, Tiruchengode, Namakkal(DT), Tamilnadu, India
3
Department of Information Technology, Bannari Amman Institute of Technology, Sathyamangalam, Erode(DT), Tamilnadu, India
A recurrent neural network (RNN) is one that works faster. An input vector is received, and periodic updations on high-dimensional hidden state are done in every iteration for accurate results. In machine learning, training a deep architecture model has always acquired a challenging focus (Deng et al. 2012; Deepika et al. 201
Data Loading...
