Convolutional neural networks for relevance feedback in content based image retrieval
- PDF / 2,813,671 Bytes
- 27 Pages / 439.642 x 666.49 pts Page_size
- 59 Downloads / 192 Views
Convolutional neural networks for relevance feedback in content based image retrieval A Content based image retrieval system that exploits convolutional neural networks both for feature extraction and for relevance feedback Lorenzo Putzu1
· Luca Piras1,2 · Giorgio Giacinto1,2
Received: 18 June 2019 / Revised: 14 June 2020 / Accepted: 29 June 2020 / © The Author(s) 2020
Abstract Given the great success of Convolutional Neural Network (CNN) for image representation and classification tasks, we argue that Content-Based Image Retrieval (CBIR) systems could also leverage on CNN capabilities, mainly when Relevance Feedback (RF) mechanisms are employed. On the one hand, to improve the performances of CBIRs, that are strictly related to the effectiveness of the descriptors used to represent an image, as they aim at providing the user with images similar to an initial query image. On the other hand, to reduce the semantic gap between the similarity perceived by the user and the similarity computed by the machine, by exploiting an RF mechanism where the user labels the returned images as being relevant or not concerning her interests. Consequently, in this work, we propose a CBIR system based on transfer learning from a CNN trained on a vast image database, thus exploiting the generic image representation that it has already learned. Then, the pretrained CNN is also fine-tuned exploiting the RF supplied by the user to reduce the semantic gap. In particular, after the user’s feedback, we propose to tune and then re-train the CNN according to the labelled set of relevant and non-relevant images. Then, we suggest different strategies to exploit the updated CNN for returning a novel set of images that are expected to be relevant to the user’s needs. Experimental results on different data sets show the effectiveness of the proposed mechanisms in improving the representation power of the CNN with respect to the user concept of image similarity. Moreover, the pros and cons of the different approaches can be clearly pointed out, thus providing clear guidelines for the implementation in production environments. Keywords Content based image retrieval · Convolutional neural network · Feature extraction · Similarity · Relevance feedback
Lorenzo Putzu
[email protected]
Extended author information available on the last page of the article.
Multimedia Tools and Applications
1 Introduction Description, recognition, and automatic classification of the structures in the images are the basis of a large number of applications that require the processing and transmission of visual information. These applications are based on image processing techniques aimed to extract information that is tailored to the task at hand. The information extracted from the images is then analysed to provide visual or logical patterns based on the characteristics of the images and their mutual relationship. Content-Based Image Retrieval (CBIR) is one of such applications that leverages on the description and representation of the information in images
Data Loading...
