Multimodal analysis of user-generated multimedia content

Description

This book presents a study of semantics and sentics understanding derived from user-generated multimodal content (UGC). It enables researchers to learn about the ways multimodal analysis of UGC can augment semantics and sentics understanding and it helps in addressing several multimedia analytics problems from social media such as event detection and summarization, tag recommendation and ranking, soundtrack recommendation, lecture video segmentation, and news video uploading. Readers will discover how the derived knowledge structures from multimodal information are beneficial for efficient multimedia search, retrieval, and recommendation. However, real-world UGC is complex, and extracting the semantics and sentics from only multimedia content is very difficult because suitable concepts may be exhibited in different representations. Moreover, due to the increasing popularity of social media websites and advancements in technology, it is now possible to collect a significant amount of important contextual information (e.g., spatial, temporal, and preferential information). Thus, there is a need to analyze the information of UGC from multiple modalities to address these problems. A discussion of multimodal analysis is presented followed by studies on how multimodal information is exploited to address problems that have a significant impact on different areas of society (e.g., entertainment, education, and journalism). Specifically, the methods presented exploit the multimedia content (e.g., visual content) and associated contextual information (e.g., geo-, temporal, and other sensory data). The reader is introduced to several knowledge bases and fusion techniques to address these problems. This work includes future directions for several interesting multimedia analytics problems that have the potential to significantly impact society. The work is aimed at researchers in the multimedia field who would like to pursue research in the area of multimodal analysis of UGC Read more... Abstract: This book presents a summary of the multimodal analysis of user-generated multimedia content (UGC). Several multimedia systems and their proposed frameworks are also discussed. Subsequently, we present a personalized music video generation system for outdoor user-generated videos. Read more... Front Matter ....Pages i-xxii Introduction (Rajiv Shah, Roger Zimmermann)....Pages 1-30 Literature Review (Rajiv Shah, Roger Zimmermann)....Pages 31-57 Event Understanding (Rajiv Shah, Roger Zimmermann)....Pages 59-99 Tag Recommendation and Ranking (Rajiv Shah, Roger Zimmermann)....Pages 101-138 Soundtrack Recommendation for UGVs (Rajiv Shah, Roger Zimmermann)....Pages 139-171 Lecture Video Segmentation (Rajiv Shah, Roger Zimmermann)....Pages 173-203 Adaptive News Video Uploading (Rajiv Shah, Roger Zimmermann)....Pages 205-234 Conclusion and Future Work (Rajiv Shah, Roger Zimmermann)....Pages 235-260 Back Matter ....Pages 261-263