ASTS: attention based spatio-temporal sequential framework for movie trailer genre classification
- PDF / 1,296,323 Bytes
- 16 Pages / 439.642 x 666.49 pts Page_size
- 66 Downloads / 207 Views
ASTS: attention based spatio-temporal sequential framework for movie trailer genre classification Yitong Yu1 · Ziyu Lu2
· Yang Li1 · Delong Liu3
Received: 15 August 2019 / Revised: 31 August 2020 / Accepted: 19 October 2020 / © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Automatic movie trailer genre classification is a challenging task because trailers have more diverse content and high-level sequential semantic concepts within the movie storyline, which can help for multimedia search and personalized movie recommendation. Traditional methods generally extract the low-level features or consider the local sequential dependencies among trailer frames, ignoring the global high-level sequential semantic concepts. In this manuscript, we propose a novel and effective Attention based Spatio-temporal Sequential Framework (ASTS) for movie trailer genre classification. The proposed framework mainly consists of two modules, respectively the spatio-temporal descriptive module and the attention-based sequential module. The spatio-temporal descriptive module adopts some advanced convolution neural networks to extract the spatio-temporal features of key trailer frames, which can capture the local spatio-temporal semantic features. The attention-based sequential module is designed to process the extracted spatio-temporal feature representation sequence for capturing the global high-level sequential semantic concepts within the movie storyline. We crawl 14,415 labeled movie trailers from YouTube and integrate them into the public dataset MovieLens. Experiment results show that our proposed framework is superior to state-of-the-art methods. Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11042-020-10125-y) contains supplementary material, which is available to authorized users. Ziyu Lu
[email protected] Yitong Yu [email protected] Yang Li [email protected] Delong Liu [email protected] 1
School of Information, Central University of Finance and Economics, Beijing, China
2
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
3
China Institute of Water Resources and Hydropower Research, Beijing, China
Multimedia Tools and Applications
Keywords Movie genre classification · Self-attention · Spatio-temporal model · Sequential modeling
1 Introduction Movie genres have been important characteristics of movies as they can act as tags for specific search in multimedia platforms and help for personalized movie recommendation. In recent decades, the scale of movie genres has been increasingly larger, and the classifications of movie genres have become finer. For example, the action movie can be divided into the following finer categories-Kung fu, Wuxia, Police, etc. Therefore, constructing an automatic movie genre classifier to update the genre labels of existing movies is important for accurate personalized recommendation, and for can provide a fine-grained classification of movies in online multimedia platforms,
Data Loading...
