Pruning filters with L1-norm and capped L1-norm for CNN compression
- PDF / 1,088,578 Bytes
- 9 Pages / 595.276 x 790.866 pts Page_size
- 81 Downloads / 220 Views
Pruning filters with L1-norm and capped L1-norm for CNN compression Aakash Kumar 1
&
Ali Muhammad Shaikh 1 & Yun Li 1 & Hazrat Bilal 1 & Baoqun Yin 1
# Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract The blistering progress of convolutional neural networks (CNNs) in numerous applications of the real-world usually obstruct by a surge in network volume and computational cost. Recently, researchers concentrate on eliminating these issues by compressing the CNN models, such as pruning filters and weights. In comparison with the technique of pruning weights, the technique of pruning filters doesn’t effect in sparse connectivity patterns. In this article, we have proposed a fresh new technique to estimate the significance of filters. More precisely, we combined L1-norm with capped L1-norm to represent the amount of information extracted by the filter and control regularization. In the process of pruning, the insignificant filters remove directly without any loss in the test accuracy, providing much slimmer and compact models with comparable accuracy and this process is iterated a few times. To validate the effectiveness of our algorithm. We experimentally determine the usefulness of our approach with several advanced CNN models on numerous standard data sets. Particularly, data sets CIFAR-10 is used on VGG-16 and prunes 92.7% parameters with float-point-operations (FLOPs) reduction of 75.8% without loss of accuracy and has achieved advancement in state-of-art. Keywords Filter pruning . Capped L1-norm . VGGnet . CIFAR . Convolutional neural network . FLOPs
1 Introduction Convolutional Neural Networks (CNNs) are an attractive part of machine learning algorithms that has extensive effectiveness in several Artificial Intelligence (AI) practical applications, for instance, object detection, spam detection, speech recognition, personal assistants, image recognition [1–6]. But, the state-of-the-art CNNs such as ResNet56, ResNet164, and VGG16 are not memory-efficient with * Aakash Kumar [email protected] Ali Muhammad Shaikh [email protected] Yun Li [email protected] Hazrat Bilal [email protected] Baoqun Yin [email protected] 1
Department of Automation, University of Science and Technology of China, Hefei 230026, People’s Republic of China
billions of trainable parameters, and requirement of 62 M, 97.49B, and 15.3B FLOPS, respectively. These cumbersome architectures have considerable inference costs, particularly when implemented with mobile devices or embedded sensors where power consumption is the high and computational power may be limited. Therefore, it is important to reduce the size of the deep CNN architectures which have comparatively low power consumption and computational cost but high accuracy in real-world applications. Also, model compression has shown notable attention for both industry and research. The use of CNNs in practical applications is generally controlled by 1) Model size: most of the power of CNNs originates from their trainable parameters which are i
Data Loading...
