High-resolution image de-raining using conditional GAN with sub-pixel upscaling
- PDF / 6,457,981 Bytes
- 20 Pages / 439.642 x 666.49 pts Page_size
- 112 Downloads / 168 Views
High-resolution image de-raining using conditional GAN with sub-pixel upscaling Prasen Kumar Sharma1
· Sathisha Basavaraju1 · Arijit Sur1
Received: 8 April 2020 / Revised: 3 August 2020 / Accepted: 13 August 2020 / © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract High-quality image de-raining is a challenging task that has been given considerable importance in recent times. To begin with, this problem is modeled as an image decomposition task where a rainy image is decomposed into the rain-free background and the associated rain streak map. Most of the existing methods have been successful in removing the rainstreaks but fails to restore the image quality, which is degraded due to noise removal. This paper proposes a novel architecture called High-Resolution Image De-Raining using Conditional Generative Adversarial Networks (HRID-GAN) to generate a de-rained image with minimal artifacts and better visual quality. Extensive experiments on publicly available synthetic as well as real-world datasets show a substantial improvement over the state-of-the-art methods SPANet (Wang et al. 2019) by ∼ 2.43% in PSNR and, DID-MDN (Zhang and Patel 2018) by ∼ 2.43%, ∼ 10.12% and ID-CGAN (Zhang et al. 2017) by ∼ 11.80%, ∼ 34.70% in SSIM and PSNR respectively. Keywords Image restoration · Deep learning · Conditional GAN
1 Introduction Rain-streaks, in the real-world scenario, may induce problems in many images and videobased applications such as UAV-based tracking, surveillance, autonomous car driving, etc. The rain streaks near camera lens appear to be raindrops and far from camera lens appears to be like haze veil, [39] thereby making it visually degraded. So far, the proposed solutions for the image de-raining, can be categorized based on (1) Supervised models with labelledconstrained, (2) Semi-supervision based learning paradigms, and (3) Unsupervised models with self-supervised constrained. Further, they span from traditional prior-based methods to deep learning-based models. However, a majority of the state-of-the-art (SOA) methods for image de-raining use supervised learning with labelled-constrained models, trained on synthetically generated datasets (Fig. 1). Prasen Kumar Sharma
[email protected] 1
Indian Institute of Technology Guwahati, Guwahati, India
Multimedia Tools and Applications
Supervised models with labelled-constrained require paired dataset when training, which is uncertain in the case of real-world rainy images. And, the manually generated rainy images may not cover a variety of rain information such as rain-streaks with various rain-density, direction, and scene illumination. Therefore, the supervised models, in general fail to remove the rain streaks from real-world samples. In last few years, a paradigm shift has been observed in the case of image de-raining from supervised to Semi-supervision based learning where the researchers have used both paired and unpaired training samples. The limitations of supervised learning can be reduced with the use of unpaired
Data Loading...
