A deep multimodal system for provenance filtering with universal forgery detection and localization
- PDF / 3,729,090 Bytes
- 20 Pages / 439.642 x 666.49 pts Page_size
- 0 Downloads / 187 Views
A deep multimodal system for provenance filtering with universal forgery detection and localization Saira Jabeen1 · Usman Ghani Khan1 · Razi Iqbal2 · Mithun Mukherjee3 · Jaime Lloret4,5 Received: 28 October 2019 / Revised: 24 July 2020 / Accepted: 13 August 2020 / © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Traditional multimedia forensics techniques inspect images to identify, localize forged regions and estimate forgery methods that have been applied. Provenance filtering is the research area that has been evolved recently to retrieve all the images that are involved in constructing a morphed image in order to analyze an image, completely forensically. This task can be performed in two stages: one is to detect and localize forgery in the query image, and the second integral part is to search potentially similar images from a large pool of images. We propose a multimodal system which covers both steps, forgery detection through deep neural networks(CNN) followed by part based image retrieval. Classification and localization of manipulated region are performed using a deep neural network. InceptionV3 is employed to extract key features of the entire image as well as for the manipulated Saira Jabeen
[email protected] Usman Ghani Khan [email protected] Razi Iqbal [email protected] Mithun Mukherjee [email protected] Jaime Lloret [email protected] 1
Department of Computer Science and Engineering, University of Engineering and Technology, Lahore, Pakistan
2
Al-Khawarizi Institute of Computer Science, University of Engineering and Technology, Lahore, Pakistan
3
Guangdong Provincial Key Laboratory of Petrochemical Equipment Fault Diagnosis, Guangdong University of Petrochemical Technology,Maoming 525000, China
4
Universitat Politecnica de Valencia, 46022 Valencia, Spain
5
School of Computing and Digital Technologies, Staffordshire University, Stoke, UK
Multimedia Tools and Applications
region. Potential donors and nearly duplicates are retrieved by using the Nearest Neighbour Algorithm. We take the CASIA-v2, CoMoFoD and NIST 2018 datasets to evaluate the proposed system. Experimental results show that deep features outperform low-level features previously used to perform provenance filtering with achieved Recall@50 of 92.8%. Keywords Provenance filtering · Convolutional neural networks · Forgery detection and localization · Manipulation detection
1 Introduction Social networks are expanding day by day through the electronic web. These networks have become a great source of communication and connection for people from all over the world. People around the globe upload their private and general information on these social platforms, such as Facebook, Twitter, YouTube, and Instagram. These digital information is not only uploaded but is also consumed with a considerable ratio on a daily basis. When this extensive amount of data is shared and viewed around the world, the authenticity of this multimedia becomes a major concern. A large number of software too
Data Loading...
