Natural Image Matting Using Deep Convolutional Neural Networks
We propose a deep Convolutional Neural Networks (CNN) method for natural image matting. Our method takes results of the closed form matting, results of the KNN matting and normalized RGB color images as inputs, and directly learns an end-to-end mapping be
- PDF / 5,965,552 Bytes
- 18 Pages / 439.37 x 666.142 pts Page_size
- 54 Downloads / 244 Views
KAIST, Daejeon, South Korea [email protected], [email protected] 2 SenseTime Group Limited, Hong Kong, China [email protected] https://sites.google.com/site/cnnmatting/
Abstract. We propose a deep Convolutional Neural Networks (CNN) method for natural image matting. Our method takes results of the closed form matting, results of the KNN matting and normalized RGB color images as inputs, and directly learns an end-to-end mapping between the inputs, and reconstructed alpha mattes. We analyze pros and cons of the closed form matting, and the KNN matting in terms of local and nonlocal principle, and show that they are complementary to each other. A major benefit of our method is that it can “recognize” different local image structures, and then combine results of local (closed form matting), and nonlocal (KNN matting) matting effectively to achieve higher quality alpha mattes than both of its inputs. Extensive experiments demonstrate that our proposed deep CNN matting produces visually and quantitatively high-quality alpha mattes. In addition, our method has achieved the highest ranking in the public alpha matting evaluation dataset in terms of the sum of absolute differences, mean squared errors, and gradient errors. Keywords: Alpha matting
1
· Deep CNN · Local and nonlocal matting
Introduction and Related Work
Image matting aims to extract an alpha matte of foreground given a trimap of an image. This problem can be expressed as a linear combination of foreground and background colors as follows [1]: I = αF + (1 − α)B,
(1)
where I, F, B, and α denote the observed image (usually in RGB), foreground, background and mixing coefficients (alpha matte) respectively. Given an input I, finding F, B, and α simultaneously is a highly ill-posed problem. Previous works in image matting have shown that, if we make proper assumptions, e.g. the color line model, about F and B, we can solve α in a closed form [2]. Local affinity based methods [2,3] analyze statistical correlation among local pixels to propagate alpha values from known regions to unknown pixels. When their c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part II, LNCS 9906, pp. 626–643, 2016. DOI: 10.1007/978-3-319-46475-6 39
Natural Image Matting Using Deep Convolutional Neural Networks
627
assumptions about local color distribution were violated, unsatisfactory results can be obtained. Nonlocal affinity based approaches [4–9] and color sampling based methods [10–14] rely on the nonlocal principle. They try to relax the local color distribution assumption by searching nonlocal neighbors and color samples which provide a better description of the image matting equation (Eq. (1)). Moreover, some works utilize multiple frames such as video [9,15] and camera arrays [16,18] to get local and nonlocal information across the images for matting. Nonlocal methods, however, do not always outperform local methods. This is because these nonlocal methods were also built on top of some assumptions, e.g. nonlocal matting Laplacian [6], structure and
Data Loading...
