Video Denoising by Combining Patch Search and CNNs
- PDF / 6,248,393 Bytes
- 16 Pages / 595.276 x 790.866 pts Page_size
- 14 Downloads / 259 Views
Video Denoising by Combining Patch Search and CNNs Axel Davy1
· Thibaud Ehret1 · Jean-Michel Morel1 · Pablo Arias1 · Gabriele Facciolo1
Received: 22 May 2020 / Accepted: 3 October 2020 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Non-local patch-based methods were until recently the state of the art for image denoising but are now outperformed by CNNs. In video denoising, however, they are still competitive with CNNs, as they can effectively exploit the video temporal redundancy, which is a key factor to attain high denoising performance. The problem is that CNN architectures are not compatible with the search for self-similarities. In this work, we propose a simple, yet efficient way to feed video selfsimilarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed method to image and video denoising. In the case of video, the patches are searched for in a 3D spatiotemporal volume. The proposed method achieves state-of-the-art results. Keywords Denoising · Video denoising · Non-local · Patch-based methods · CNN
1 Introduction Advances in image sensor hardware have steadily improved the acquisition quality of image and video cameras. However, a low signal-to-noise ratio is unavoidable in low lighting conditions if the exposure time is limited (for example to avoid motion blur). This results in high levels of noise, which negatively affects the visual quality of the video and hinders its use for many applications. As a consequence, denoising is a crucial component of any camera pipeline. Furthermore, by interpreting denoising algorithms as proximal operators, several inverse problems in image processing can be solved by iteratively applying a denoising algorithm [52]. Hence,
The authors gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research. Work partly financed by IDEX Paris-Saclay IDI 2016, ANR-11-IDEX-0003-02, ONR grant N00014-17-1-2552, CNES MISS project, DGA Astrid ANR-17-ASTR-0013-01, DGA ANR-16-DEFA-0004-01, MENRT. This work used HPC resources from the “Mésocentre” computing center of CentraleSup ’elec and ENS Paris-Saclay supported by CNRS and Région Île-de-France.
B 1
Axel Davy [email protected]
there is a need for video denoising algorithms with a low running time. Literature review on image denoising Image denoising has a vast literature where a variety of methods have been applied: PDEs and variational methods (including MRF models) [12,54,55], transform domain methods [23], non-local (or patch-based) methods [7,19], multiscale approaches [28], etc. See [36] for a review. In the last two or three years, CNNs have taken over the state of the art. In
Data Loading...
