Summarizing Audiovisual Contents of a Video Program
- PDF / 1,036,421 Bytes
- 10 Pages / 595.2 x 792 pts Page_size
- 104 Downloads / 251 Views
Summarizing Audiovisual Contents of a Video Program Yihong Gong NEC Laboratories America, Inc., 10080 North Wolfe Road, SW3-350 Cupertino, CA 95014, USA Email: [email protected] Received 19 March 2002 and in revised form 22 October 2002 In this paper, we focus on video programs that are intended to disseminate information and knowledge such as news, documentaries, seminars, etc, and present an audiovisual summarization system that summarizes the audio and visual contents of the given video separately, and then integrating the two summaries with a partial alignment. The audio summary is created by selecting spoken sentences that best present the main content of the audio speech while the visual summary is created by eliminating duplicates/redundancies and preserving visually rich contents in the image stream. The alignment operation aims to synchronize each spoken sentence in the audio summary with its corresponding speaker’s face and to preserve the rich content in the visual summary. A Bipartite Graph-based audiovisual alignment algorithm is developed to efficiently find the best alignment solution that satisfies these alignment requirements. With the proposed system, we strive to produce a video summary that: (1) provides a natural visual and audio content overview, and (2) maximizes the coverage for both audio and visual contents of the original video without having to sacrifice either of them. Keywords and phrases: video summarization, audiovisual summarization, partial audiovisual alignment, bipartite graph, minimum spanning tree, maximum bipartite matching.
1.
INTRODUCTION
Video programs are voluminous, redundant, and are a timesequential medium whose overall contents cannot be captured at a glance. The voluminous and sequential nature of video programs not only creates congestions in computer systems and communication networks, but also causes bottlenecks in human information comprehension due to the limited human information processing speed. In the past decade, great efforts have been made to relieve the computer and communication congestion problems. However, in the whole video content creation, storage, processing, delivery, and utilization loop, the human bottleneck problem has long been neglected. Without technologies enabling fast and effective content overviews, browsing through video collections or finding desired video programs from a long list of search results will remain arduous and painful tasks. Automatic video content summarization is one of the promising solutions to the human bottleneck problem. A concise and informative video summary will enable the user to quickly figure out the general content of a video and help him/her to decide whether the whole video program is worth watching. On the Internet, a compact video summary can be used as a video thumbnail of the original video, which requires much less efforts to download and comprehend. For
most home users with very limited network bandwidths, this type of video thumbnails can well prevent them from spending minutes or tens of minutes do
Data Loading...