RoCGAN: Robust Conditional GAN
- PDF / 2,112,361 Bytes
- 19 Pages / 595.276 x 790.866 pts Page_size
- 1 Downloads / 167 Views
RoCGAN: Robust Conditional GAN Grigorios G. Chrysos1
· Jean Kossaifi1,2 · Stefanos Zafeiriou1
Received: 15 May 2019 / Accepted: 16 June 2020 © The Author(s) 2020
Abstract Conditional image generation lies at the heart of computer vision and conditional generative adversarial networks (cGAN) have recently become the method of choice for this task, owing to their superior performance. The focus so far has largely been on performance improvement, with little effort in making cGANs more robust to noise. However, the regression (of the generator) might lead to arbitrarily large errors in the output, which makes cGANs unreliable for real-world applications. In this work, we introduce a novel conditional GAN model, called RoCGAN, which leverages structure in the target space of the model to address the issue. Specifically, we augment the generator with an unsupervised pathway, which promotes the outputs of the generator to span the target manifold, even in the presence of intense noise. We prove that RoCGAN share similar theoretical properties as GAN and establish with both synthetic and real data the merits of our model. We perform a thorough experimental validation on large scale datasets for natural scenes and faces and observe that our model outperforms existing cGAN architectures by a large margin. We also empirically demonstrate the performance of our approach in the face of two types of noise (adversarial and Bernoulli). Keywords Conditional GAN · Unsupervised learning · Autoencoder · Robust regression · Super-resolution · Adversarial attacks · Cross-noise experiments
1 Introduction Image-to-image translation and more generally conditional image generation lie at the heart of computer vision. Conditional generative adversarial networks (cGAN) (Mirza and Osindero 2014) have become a dominant approach in the
Communicated by Jun-Yan Zhu, Hongsheng Li, Eli Shechtman, MingYu Liu, Jan Kautz, Antonio Torralba. Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11263-020-01348-5) contains supplementary material, which is available to authorized users.
B
Grigorios G. Chrysos [email protected] Jean Kossaifi [email protected] Stefanos Zafeiriou [email protected]
field, e.g. in dense1 regression (Isola et al. 2017; Pathak et al. 2016; Ledig et al. 2017; Bousmalis et al. 2016; Liu et al. 2017; Miyato and Koyama 2018; Yu et al. 2018; Tulyakov et al. 2018). The major focus so far has been on improving the performance; we advocate instead that improving the generalization performance, e.g. as measured under intense noise and test-time perturbations, is a significant topic with a host of applications, e.g. facial analysis (Georgopoulos et al. 2018). If we aim to utilize cGAN or similar methods as a production technology, they need to have performance guarantees even under large amount of noise. To that end, we study the robustness of conditional GAN under noise. Conditional Generative Adversarial Networks consist of two modules, namely a generator and a discr
Data Loading...
