Convolutional networks for appearance-based recommendation and visualisation of mascara products
- PDF / 3,255,239 Bytes
- 13 Pages / 595.276 x 790.866 pts Page_size
- 33 Downloads / 188 Views
ORIGINAL PAPER
Convolutional networks for appearance‑based recommendation and visualisation of mascara products Christopher J. Holder1 · Stephen Ricketts2 · Boguslaw Obara1 Received: 31 May 2019 / Revised: 15 October 2019 / Accepted: 3 December 2019 / Published online: 21 January 2020 © The Author(s) 2020
Abstract In this work, we explore the problems of recommending and visualising makeup products based on images of customers. Focusing on mascara, we propose a two-stage approach that first recommends products to a new customer based on the preferences of other customers with similar visual appearance and then visualises how the recommended products might look on the customer. For the initial product recommendation, we train a Siamese convolutional neural network, using our own dataset of cropped eye regions from images of 91 female subjects, such that it learns to output feature vectors that place images of the same subject close together in high-dimensional space. We evaluate the trained network based on its ability to correctly identify existing subjects from unseen images, and then assess its capability to identify visually similar subjects when an image of a new subject is used as input. For product visualisation, we train per-product generative adversarial networks to map the appearance of a specific product onto an image of a customer with no makeup. We train models to generate images of two mascara formulations and assess their capability to generate realistic mascara lashes while changing as little as possible within non-lash image regions and simulating the different effects of the two products used. Keywords Deep learning · Generative adversarial networks · Siamese networks · Recommender systems · Cosmetics
1 Introduction In this paper, we describe, discuss and evaluate our work towards a cosmetic customer advisor system intended for use in retail environments. Focusing specifically on mascara products, our approach comprises two processes: product recommendation and product visualisation. Our recommendation approach uses a Siamese network [2] to identify This work is an extended version of our paper Visual Siamese Clustering for Cosmetic Product Recommendation [1] presented at the 1st International Workshop on Advanced Machine Vision. This paper expands on the discussion of the proposed approach for mascara recommendation and details additional related work that uses generative adversarial networks to visualise recommended products. * Boguslaw Obara [email protected] Christopher J. Holder [email protected] 1
Department of Computer Science, Durham University, Durham, UK
Walgreens Boots Alliance, Nottingham, UK
2
prior customers with similar visual features to the current customer so that products preferred by those prior customers can be recommended. Our visualisation approach uses a generative adversarial network (GAN) [3] to generate realistic images demonstrating how a customer might look with a specific product applied. We envision such a system being deployed within a
Data Loading...
