Computational media aesthetics: Finding meaning beautiful

Innovative media management, annotation, delivery, and navigation services will enrich online shopping, help-desk services, and anytime-anywhere training over wireless devices. However, the semantic gap between the rich meaning that users want when they query and browse media and the shallowness of the content descriptions that one can actually compute is weakening today\u27s automatic content-annotation systems. To address such problems, an approach that markedly departs from existing methods based on detecting and annotating low-level audio-visual features is advocated

Computational Media Aesthetics for Media Synthesis

Ph.DDOCTOR OF PHILOSOPH

Scene extraction in motion pictures

This paper addresses the challenge of bridging the semantic gap between the rich meaning users desire when they query to locate and browse media and the shallowness of media descriptions that can be computed in today\u27s content management systems. To facilitate high-level semantics-based content annotation and interpretation, we tackle the problem of automatic decomposition of motion pictures into meaningful story units, namely scenes. Since a scene is a complicated and subjective concept, we first propose guidelines from fill production to determine when a scene change occurs. We then investigate different rules and conventions followed as part of Fill Grammar that would guide and shape an algorithmic solution for determining a scene. Two different techniques using intershot analysis are proposed as solutions in this paper. In addition, we present different refinement mechanisms, such as film-punctuation detection founded on Film Grammar, to further improve the results. These refinement techniques demonstrate significant improvements in overall performance. Furthermore, we analyze errors in the context of film-production techniques, which offer useful insights into the limitations of our method

"You Tube and I Find" - personalizing multimedia content access

Recent growth in broadband access and proliferation of small personal devices that capture images and videos has led to explosive growth of multimedia content available everywhereVfrom personal disks to the Web. While digital media capture and upload has become nearly universal with newer device technology, there is still a need for better tools and technologies to search large collections of multimedia data and to find and deliver the right content to a user according to her current needs and preferences. A renewed focus on the subjective dimension in the multimedia lifecycle, fromcreation, distribution, to delivery and consumption, is required to address this need beyond what is feasible today. Integration of the subjective aspects of the media itselfVits affective, perceptual, and physiological potential (both intended and achieved), together with those of the users themselves will allow for personalizing the content access, beyond today’s facility. This integration, transforming the traditional multimedia information retrieval (MIR) indexes to more effectively answer specific user needs, will allow a richer degree of personalization predicated on user intention and mode of interaction, relationship to the producer, content of the media, and their history and lifestyle. In this paper, we identify the challenges in achieving this integration, current approaches to interpreting content creation processes, to user modelling and profiling, and to personalized content selection, and we detail future directions. The structure of the paper is as follows: In Section I, we introduce the problem and present some definitions. In Section II, we present a review of the aspects of personalized content and current approaches for the same. Section III discusses the problem of obtaining metadata that is required for personalized media creation and present eMediate as a case study of an integrated media capture environment. Section IV presents the MAGIC system as a case study of capturing effective descriptive data and putting users first in distributed learning delivery. The aspects of modelling the user are presented as a case study in using user’s personality as a way to personalize summaries in Section V. Finally, Section VI concludes the paper with a discussion on the emerging challenges and the open problems

Injection, detection and repair of aesthetics in home movies

This paper details the design of an algorithm for automatically manipulating the important aesthetic element of video, visual tempo. Automatic injection, detection and repair of such aesthetic elements, it is argued, is vital to the next generation of amateur multimedia authoring tools. We evaluate the performance of the algorithm on a battery of synthetic data and demonstrate its ability to return the visual tempo of the final media a considerable degree closer to the target signal. The novelty of this work lies chiefly in the systematic manipulation of this high level aesthetic element of video

Indexing narrative structure and semantics in motion pictures with a probabilistic framework

This work constitutes the first attempt to extract an important narrative structure, the 3-Act story telling paradigm, in film. This narrative structure is prevalent in the domain of film as it forms the foundation and framework in which the film can be made to function as an effective tool for story telling, and its extraction is a vital step in automatic content management for film data. A novel act boundary likelihood function for Act 1 is derived using a Bayesian formulation under guidance from film grammar, tested under many configurations and the results are reported for experiments involving 25 full length movies. The formulation is shown to be a useful tool in both the automatic and semi-interactive setting for semantic analysis of film.<br /