Learning color space adaptation from synthetic to real images of cirrus clouds
- PDF / 3,344,569 Bytes
- 13 Pages / 595.276 x 790.866 pts Page_size
- 32 Downloads / 152 Views
ORIGINAL ARTICLE
Learning color space adaptation from synthetic to real images of cirrus clouds Qing Lyu1 · Minghao Chen1 · Xiang Chen1 Accepted: 2 October 2020 © Springer-Verlag GmbH Germany, part of Springer Nature 2020
Abstract Cloud segmentation plays a crucial role in image analysis for climate modeling. Manually labeling the training data for cloud segmentation is time-consuming and error-prone. We explore to train segmentation networks with synthetic data due to the natural acquisition of pixel-level labels. Nevertheless, the domain gap between synthetic and real images significantly degrades the performance of the trained model. We propose a color space adaptation method to bridge the gap, by training a color-sensitive generator and discriminator to adapt synthetic data to real images in color space. Instead of transforming images by general convolutional kernels, we adopt a set of closed-form operations to make color-space adjustments while preserving the labels. We also construct a synthetic-to-real cirrus cloud dataset SynCloud and demonstrate the adaptation efficacy on the semantic segmentation task of cirrus clouds. With our adapted synthetic data for training the semantic segmentation, we achieve an improvement of 6.59% when applied to real images, superior to alternative methods. Keywords Color space · Synthetic-to-real · Domain adaptation · Cirrus clouds · Segmentation · Style transfer
1 Introduction Cloud images are widely used in climate modeling, weather prediction, renewable energy generation, and satellite communications [1–4]. Digital analysis of clouds and their features is necessary for these subjects. One of the first steps in cloud image analysis is cloud segmentation. Previous learning-based cloud segmentation methods are supervised and require a large number of training images with manually labeled ground-truth [5–8]. Since manual labeling is time-consuming and error-prone, we explore to train cloud segmentation networks with synthetic images. Training on synthetic images has become increasingly popular in vision tasks, such as object detection [9–12], viewpoints estimation [13–16], and semantic segmentation [17]. For example, photo-realistic rendering on 3D models can pro-
B
Xiang Chen [email protected] Qing Lyu [email protected] Minghao Chen [email protected]
1
The State Key Lab of CAD&CG, Zijingang Campus, Zhejiang University, Hangzhou 310058, Zhejiang, China
vide an accurate pixel-level annotation for each object in the scene, which eliminates the labeling cost for image segmentation learning. Under such circumstances, several synthetic datasets like Virtual KITTI [11] and SYNTHIA [18] have been generated for training and evaluating vision models. When directly applying the synthetic dataset for real-world image tasks, performance degradation arises from the inherent domain gap. Such gaps may result from many reasons, e.g., differences in geometric details, textures, backgrounds, and lightings. Some versatile domain transfer models are proposed to transfer synthetic i
Data Loading...
