Indexing of fictional video content for event detection and summarisation

This paper presents an approach to movie video indexing that utilises audiovisual analysis to detect important and meaningful temporal video segments, that we term events. We consider three event classes, corresponding to dialogues, action sequences, and montages, where the latter also includes musical sequences. These three event classes are intuitive for a viewer to understand and recognise whilst accounting for over 90% of the content of most movies. To detect events we leverage traditional filmmaking principles and map these to a set of computable low-level audiovisual features. Finite state machines (FSMs) are used to detect when temporal sequences of specific features occur. A set of heuristics, again inspired by filmmaking conventions, are then applied to the output of multiple FSMs to detect the required events. A movie search system, named MovieBrowser, built upon this approach is also described. The overall approach is evaluated against a ground truth of over twenty-three hours of movie content drawn from various genres and consistently obtains high precision and recall for all event classes. A user experiment designed to evaluate the usefulness of an event-based structure for both searching and browsing movie archives is also described and the results indicate the usefulness of the proposed approach

Indexing of fictional video content for event detection and summarisation

This paper presents an approach to movie video indexing that utilises audiovisual analysis to detect important and meaningful temporal video segments, that we term events. We consider three event classes, corresponding to dialogues, action sequences, and montages, where the latter also includes musical sequences. These three event classes are intuitive for a viewer to understand and recognise whilst accounting for over 90% of the content of most movies. To detect events we leverage traditional filmmaking principles and map these to a set of computable low-level audiovisual features. Finite state machines (FSMs) are used to detect when temporal sequences of specific features occur. A set of heuristics, again inspired by filmmaking conventions, are then applied to the output of multiple FSMs to detect the required events. A movie search system, named MovieBrowser, built upon this approach is also described. The overall approach is evaluated against a ground truth of over twenty-three hours of movie content drawn from various genres and consistently obtains high precision and recall for all event classes. A user experiment designed to evaluate the usefulness of an event-based structure for both searching and browsing movie archives is also described and the results indicate the usefulness of the proposed approach

Indexing of fictional video content for event detection and summarisation

This paper presents an approach to movie video indexing that utilises audiovisual analysis to detect important and meaningful temporal video segments, that we term events. We consider three event classes, corresponding to dialogues, action sequences, and montages, where the latter also includes musical sequences. These three event classes are intuitive for a viewer to understand and recognise whilst accounting for over 90% of the content of most movies. To detect events we leverage traditional filmmaking principles and map these to a set of computable low-level audiovisual features. Finite state machines (FSMs) are used to detect when temporal sequences of specific features occur. A set of heuristics, again inspired by filmmaking conventions, are then applied to the output of multiple FSMs to detect the required events. A movie search system, named MovieBrowser, built upon this approach is also described. The overall approach is evaluated against a ground truth of over twenty-three hours of movie content drawn from various genres and consistently obtains high precision and recall for all event classes. A user experiment designed to evaluate the usefulness of an event-based structure for both searching and browsing movie archives is also described and the results indicate the usefulness of the proposed approach

Associating characters with events in films

The work presented here combines the analysis of a film's audiovisual features with the analysis of an accompanying audio description. Specifically, we describe a technique for semantic-based indexing of feature films that associates character names with meaningful events. The technique fuses the results of event detection based on audiovisual features with the inferred on-screen presence of characters, based on an analysis of an audio description script. In an evaluation with 215 events from 11 films, the technique performed the character detection task with Precision = 93% and Recall = 71%. We then go on to show how novel access modes to film content are enabled by our analysis. The specific examples illustrated include video retrieval via a combination of event-type and character name and our first steps towards visualization of narrative and character interplay based on characters occurrence and co-occurrence in events

On the design of television as a service based on average TV watching

Ten households were interviewed about their TV watching to inform the design of TV services. Our participants were average TV viewers who had Internet access but were not technically advanced or frequent users of the Internet as a source for TV material. We found that the flow of programs that broadcast television brings to viewers was the most important motivation for our participants to turn to the TV on-demand possibilities they had access to. Examples of triggers were social cues from people talking about things seen on TV, or time-shifting issues such as missing all or part of programs in the broadcast flow. Special interests such as sports were also a strong motivation for on-demand behavior. For the viewers, linear and on-demand TV watching was intertwined. We conclude that on-demand services should be integrated with broadcast TV in the design of future TV services

Digital cinema--an environment for multi-threaded stories

Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1993.Includes bibliographical references (leaves 85-86).by Mark David Halliday.M.S

Design, deployment and assessment of a movie archive system in a film studies context

This thesis describe our work in developing a movie archive system for students of Film Studies in Dublin City University. In particular, our system uses several recent multimedia technologies to automatically process digital video content but at the same time we use the usability engineering process to relate to the real tasks of real users in their real environments. We investigate how real users take advantage of technologies in a movie browsing system. By designing, building, deploying and assessing the usage of a technology in a user-focused way, the overall impact of a movie browsing system can be determined holistically. The application domain we work in is film studies where students need to study movie contents and analyse movie sequences. Our work began by identification of user needs through observations, focus groups and usability testing, followed by sketching and prototyping a web-based system. We then deployed the system to film study classes over a semester, monitored usage and gathered quantitative as well as qualitative data. Focused experiments were carried out to assess students’ performance and satisfaction levels. Our findings show expected patterns of usage of a real-user setting outside the lab, but at the same time highlighted issues that need to be further investigated. In general, students found most of the provided features were beneficial for their studies. Findings from the experiment shows better performance in the essay assessments and higher satisfaction levels. An interesting finding shows students are more engaged with the newly-introduced software application and take longer time to complete the same task than without the advanced features of the application. This phenomenon was rationalized from established learning theory from the psychology domain. In a technologically-oriented multimedia field today, we attempted to bring in a user-centred approach of end-user interactions throughout a 3-year development process, and we identified benefits and challenges in trying to align the technical perspectives of novel multimedia features to real-world setting

When Large Language Model based Agent Meets User Behavior Analysis: A Novel User Simulation Paradigm

User behavior analysis is crucial in human-centered AI applications. In this field, the collection of sufficient and high-quality user behavior data has always been a fundamental yet challenging problem. An intuitive idea to address this problem is automatically simulating the user behaviors. However, due to the subjective and complex nature of human cognitive processes, reliably simulating the user behavior is difficult. Recently, large language models (LLM) have obtained remarkable successes, showing great potential to achieve human-like intelligence. We argue that these models present significant opportunities for reliable user simulation, and have the potential to revolutionize traditional study paradigms in user behavior analysis. In this paper, we take recommender system as an example to explore the potential of using LLM for user simulation. Specifically, we regard each user as an LLM-based autonomous agent, and let different agents freely communicate, behave and evolve in a virtual simulator called RecAgent. For comprehensively simulation, we not only consider the behaviors within the recommender system (\emph{e.g.}, item browsing and clicking), but also accounts for external influential factors, such as, friend chatting and social advertisement. Our simulator contains at most 1000 agents, and each agent is composed of a profiling module, a memory module and an action module, enabling it to behave consistently, reasonably and reliably. In addition, to more flexibly operate our simulator, we also design two global functions including real-human playing and system intervention. To evaluate the effectiveness of our simulator, we conduct extensive experiments from both agent and system perspectives. In order to advance this direction, we have released our project at {https://github.com/RUC-GSAI/YuLan-Rec}.Comment: 26 pages, 9 figure

Sex in the city: the rise of soft-erotic film culture in Cinema Leopold, Ghent, 1945-1954

Since the 1990s, film studies saw a disciplinary shift from approaches favoring a textual and ideological analysis of films to a broader understanding of the socio-cultural history of cinema under the banner of new cinema history. This turn not only allowed for ‘niche’ research domains to flourish such as film economics or cinema memory research, or for new empirical and critical methodologies to be applied to film and cinema history. This change in researching and writing film/cinema history also shed light on previously marginalized, neglected or uncharted film cultures and histories, burgeoning scholarship in for instance (s)exploitation cinema. This contribution examines a peculiar part of post-war local film culture in the Belgian city of Ghent, more precisely the one around the city-center soft-erotic cinema Cinema Leopold (1945-54). The research is based on a programming and box-office database compiled from archival sources and contextualized by other data (internal and external correspondence, posters,…) coming from the business archive of Octave Bonnevalle, Cinema Leopold’s founding pater familias (material kept in the State Archives of Belgium; RAB/B70/1928-1977). The database now contains information on 625 film titles shown between 1945 and 1954, out of which 233 were unidentified (due to lack of information). Although the database is at times crippled by source inconsistencies, it is extremely rich in documenting the everyday practices of a cinema that gradually turned into a soft-erotic movie theater. The database allows for some remarkable findings concerning shifts in the origin of films, their production years, genres, censorship and popularity. The key finding is that Cinema Leopold started out after the Second World War with a child-friendly, mainstream Hollywood-oriented film program, as did most cinemas in Ghent, but its profile slowly tilted towards more mature audiences and provocative film genres. These included French ‘risqué’ feature films containing some forms of nudity like Perfectionist/Un Grand Patron (Ciampi, 1951) and documentaries on venereal diseases like the successful Austrian Creeping Poison/Schleichendes Gift (Wallbrück, 1946), but also auteur movies such as Bergman’s Port of Call/Hamnstad (1948) were shown. It is interesting how Leopold walked a fine line between innovative, bold European art-house cinema, soft-erotic ‘didactic’ movies and flat-out commercial soft-porn. By 1954, Leopold had gathered a loyal crowd, which kept the cinema alive until 1981 despite the several law suits and trials. This micro-history offers a remarkable example of the post-war flourishing of alternative, yet profit-driven cinema circuits, riddled with media controversies and censorship