XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32\(\times \) memory saving. In XNOR-Netwo
- PDF / 1,295,251 Bytes
- 18 Pages / 439.37 x 666.142 pts Page_size
- 3 Downloads / 217 Views
2
Allen Institute for AI, Seattle, USA {mohammadr,vicenteor}@allenai.org University of Washington, Seattle, USA {pjreddie,ali}@cs.washington.edu
Abstract. We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNORNetworks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32× memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNORNetworks approximate convolutions using primarily binary operations. This results in 58× faster convolutional operations (in terms of number of the high precision operations) and 32× memory savings. XNORNets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a Binary-Weight-Network version of AlexNet is the same as the full-precision AlexNet. We compare our method with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than 16 % in top-1 accuracy. Our code is available at: http://allenai.org/plato/xnornet.
1
Introduction
Deep neural networks (DNN) have shown significant improvements in several application domains including computer vision and speech recognition. In computer vision, a particular type of DNN, known as Convolutional Neural Networks (CNN), have demonstrated state-of-the-art results in object recognition [1–4] and detection [5–7]. Convolutional neural networks show reliable results on object recognition and detection that are useful in real world applications. Concurrent to the recent progress in recognition, interesting advancements have been happening in virtual reality (VR by Oculus) [8], augmented reality (AR by HoloLens) [9], and smart wearable devices. Putting these two pieces together, we argue that it is the right time to equip smart portable devices with the power of state-of-the-art recognition systems. However, CNN-based recognition systems need large amounts of c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part IV, LNCS 9908, pp. 525–542, 2016. DOI: 10.1007/978-3-319-46493-0 32
526
M. Rastegari et al.
Fig. 1. We propose two efficient variations of convolutional neural networks. Binary-Weight-Networks, when the weight filters contains binary values. XNORNetworks, when both weigh and input have binary values. These networks are very efficient in terms of memory and computation, while being very accurate in natural image classification. This offers the possibility of using accurate vision techniques in portable devices with limited resources. (Color figure online)
memory and computational power. While they perform well on expensive, GPUbased machines, they are often unsuitable for smaller devices like cell phones and embedded electronics. For example, AlexNet [1] has 61 M parameters (249 MB of
Data Loading...
