Dynamic character graph via online face clustering for movie analysis
- PDF / 2,122,870 Bytes
- 16 Pages / 439.642 x 666.49 pts Page_size
- 57 Downloads / 210 Views
Dynamic character graph via online face clustering for movie analysis Prakhar Kulshreshtha1 · Tanaya Guha2 Received: 27 September 2019 / Revised: 16 July 2020 / Accepted: 28 July 2020 / © The Author(s) 2020
Abstract An effective approach to automated movie content analysis involves building a network (graph) of its characters. Existing work usually builds a static character graph to summarize the content using metadata, scripts or manual annotations. We propose an unsupervised approach to building a dynamic character graph that captures the temporal evolution of character interaction. We refer to this as the character interaction graph (CIG). Our approach has two components: (i) an online face clustering algorithm that discovers the characters in the video stream as they appear, and (ii) simultaneous creation of a CIG using the temporal dynamics of the resulting clusters. We demonstrate the usefulness of the CIG for two movie analysis tasks: narrative structure (acts) segmentation and major character retrieval. Our evaluation on full-length movies containing more than 5000 face tracks shows that the proposed approach achieves superior performance for both the tasks. Keywords Online clustering · Face clustering · Media content analysis · Character graph · Narrative structure
1 Introduction Automated analysis of media content, such as movies has traditionally focused on extracting and using low level features from shots and scenes for analyzing narrative structures and key events [10, 11]. For humans, however, a movie is not just a collection of shots or scenes. It is the characters that usually play the most important role in storytelling [18]. More recently, character-centric representation of movies, such as character networks have emerged as
A part of this work was done when both the authors were at IIT Kanpur. Tanaya Guha
[email protected] Prakhar Kulshreshtha [email protected] 1
Carnegie Mellon University, Pittsburgh, USA
2
University of Warwick, Coventry, UK
Multimedia Tools and Applications
an effective approach towards media content analysis [15, 16, 22]. A character network usually has the major characters as its nodes where the edges summarize the relationship between character pairs. Such networks have been shown to facilitate a number of movie analysis tasks including character analysis [16], story segmentation [22] and major character identification [15]. The existing methods build a single, static character network for the entire movie. While static graphs offer a convenient summary of the overall interactions among characters, they can not capture the evolution of a movie’s dynamic narrative. In this paper, we present an unsupervised approach to building a dynamic character network via online face clustering. We refer to this network as the character interaction graph (CIG), where each movie character is represented as a node, and an edge represents pairwise interaction between characters. The dynamic aspect of the CIG offers an effective way to capture the variations in char
Data Loading...
