Automatic Generation of Video Summaries for Historical Films

A video summary is a sequence of video clips extracted from a longer video. Much shorter than the original, the summary preserves its essential messages. In the project ECHO (European Chronicles On-line) a system was developed to store and manage large collections of historical films for the preservation of cultural heritage. At the University of Mannheim we have developed the video summarization component of the ECHO system. In this paper we discuss the particular challenges the historical film material poses, and how we have designed new video processing algorithms and modified existing ones to cope with noisy black-and-white films. We also report empirical results from the use of our summarization tool at the four major European national video archives

Viznotes – Visual Summaries for videos

This project presents a method of visually summarizing TED-like videos called Viznotes. The Viznotes interface provides a structured yet organic summarization of the contents of the video. Derived from the concepts of sketchnoting, this interface provides segments of video represented as a sketch like image with summary of the segments and keywords arranged in a pre-defined template, with certain elements showing chronology and relations. Viznote also provides an interface for navigation of the videos. Further it also enables the user to customize and make a more personal visual summary. Tools like sketching, sketch components, screen image representation etc. help users to leverage additional functionality in note taking

Collaborative video searching on a tabletop

Almost all system and application design for multimedia systems is based around a single user working in isolation to perform some task yet much of the work for which we use computers to help us, is based on working collaboratively with colleagues. Groupware systems do support user collaboration but typically this is supported through software and users still physically work independently. Tabletop systems, such as the DiamondTouch from MERL, are interface devices which support direct user collaboration on a tabletop. When a tabletop is used as the interface for a multimedia system, such as a video search system, then this kind of direct collaboration raises many questions for system design. In this paper we present a tabletop system for supporting a pair of users in a video search task and we evaluate the system not only in terms of search performance but also in terms of user–user interaction and how different user personalities within each pair of searchers impacts search performance and user interaction. Incorporating the user into the system evaluation as we have done here reveals several interesting results and has important ramifications for the design of a multimedia search system

Artistic Expressions of Vegan Women with Disturbed Eating Behavior and Body Image Distress

This research explores the experience of women who are vegan, and have disturbed eating behaviors (DEB) and body image distress (BID). Four participants completed a series of three art-making sessions. Participants were invited to visually explore their experience as a vegan woman with DEB/BID. They made a mixed media collage with an emphasis on layering in each session. They engaged in discussion about their process, and the final art piece’s meaning. Between sessions, researcher response art pieces were created for each participant piece, with accompanying journal reflections to engage with the ideas they explored. All participant sessions were video and audio-recorded. Edited individual review videos were created for each participant. Participants attended a fourth session, during which they discussed the research process, their art, corresponding response art, and the video of their sessions. A final research summary video was created, and a final summary art piece was created. Qualitative analysis revealed Six Essential Ideas that characterized the women’s experience: re-claiming space, defining female, navigating food choices, vegan in context, identification and relationships with other animals and the environment, and disability as a vegan woman. A functional model of these six ideas, in relation to femaleness, veganism, and DEB/BID is presented to make meaning of the results. A set of theoretical models of the mechanisms between femaleness, veganism, and DEB/BID is proposed in response to the research question

Automatic summarization of narrative video

The amount of digital video content available to users is rapidly increasing. Developments in computer, digital network, and storage technologies all contribute to broaden the offer of digital video. Only users’ attention and time remain scarce resources. Users face the problem of choosing the right content to watch among hundreds of potentially interesting offers. Video and audio have a dynamic nature: they cannot be properly perceived without considering their temporal dimension. This property makes it difficult to get a good idea of what a video item is about without watching it. Video previews aim at solving this issue by providing compact representations of video items that can help users making choices in massive content collections. This thesis is concerned with solving the problem of automatic creation of video previews. To allow fast and convenient content selection, a video preview should take into consideration more than thirty requirements that we have collected by analyzing related literature on video summarization and film production. The list has been completed with additional requirements elicited by interviewing end-users, experts and practitioners in the field of video editing and multimedia. This list represents our collection of user needs with respect to video previews. The requirements, presented from the point of view of the end-users, can be divided into seven categories: duration, continuity, priority, uniqueness, exclusion, structural, and temporal order. Duration requirements deal with the durations of the preview and its subparts. Continuity requirements request video previews to be as continuous as possible. Priority requirements indicate which content should be included in the preview to convey as much information as possible in the shortest time. Uniqueness requirements aim at maximizing the efficiency of the preview by minimizing redundancy. Exclusion requirements indicate which content should not be included in the preview. Structural requirements are concerned with the structural properties of video, while temporal order requirements set the order of the sequences included in the preview. Based on these requirements, we have introduced a formal model of video summarization specialized for the generation of video previews. The basic idea is to translate the requirements into score functions. Each score function is defined to have a non-positive value if a requirement is not met, and to increase depending on the degree of fulfillment of the requirement. A global objective function is then defined that combines all the score functions and the problem of generating a preview is translated into the problem of finding the parts of the initial content that maximize the objective function. Our solution approach is based on two main steps: preparation and selection. In the preparation step, the raw audiovisual data is analyzed and segmented into basic elements that are suitable for being included in a preview. The segmentation of the raw data is based on a shot-cut detection algorithm. In the selection step various content analysis algorithms are used to perform scene segmentation, advertisements detection and to extract numerical descriptors of the content that, introduced in the objective function, allow to estimate the quality of a video preview. The core part of the selection step is the optimization step that consists in searching the set of segments that maximizes the objective function in the space of all possible previews. Instead of solving the optimization problem exactly, an approximate solution is found by means of a local search algorithm using simulated annealing. We have performed a numerical evaluation of the quality of the solutions generated by our algorithm with respect to previews generated randomly or by selecting segments uniformly in time. The results on thirty content items have shown that the local search approach outperforms the other methods. However, based on this evaluation, we cannot conclude that the degree of fulfillment of the requirements achieved by our method satisfies the end-user needs completely. To validate our approach and assess end-user satisfaction, we conducted a user evaluation study in which we compared six aspects of previews generated using our algorithm to human-made previews and to previews generated by subsampling. The results have shown that previews generated using our optimization-based approach are not as good as manually made previews, but have higher quality than previews created using subsample. The differences between the previews are statistically significant

The Rhetoric of Conspiracism in User-Centered Democracy

This dissertation examines social media-based conspiracy theories of the past five years (2016-2021) and considers what this recent conspiracy rhetoric suggests about the evolving relationship between people, platforms, and politics in the contemporary United States. I use the tools of rhetorical theory and criticism to analyze a small archive of conspiracist content across three case studies—Pizzagate, a conspiracy theory alleging a vast pedophilia ring run by political elites; conspiracy theories surrounding the mass shooting at Marjory Stoneman Douglas High School in Parkland, Florida; and Plandemic, a self-styled “documentary” about COVID-19 conspiracies that went viral in May 2020. In each case study, I show that the conspiracy rhetoric in question uses the unique affordances of social media platforms to amplify that conspiracy theory’s rhetorical efficacy. Ultimately, I argue that conspiracism has now become a durable form of social media content that threatens to wreak havoc on American political discourse.Doctor of Philosoph