Enhancing Video Recommendation Using Multimedia Content

Video recordings are complex media types. When we watch a movie, we can effortlessly register a lot of details conveyed to us (by the author) through different multimedia channels, in particular, the audio and visual modalities. To date, majority of movie recommender systems use collaborative filtering (CF) models or content-based filtering (CBF) relying on metadata (e.g., editorial such as genre or wisdom of the crowd such as user-generated tags) at their core since they are human-generated and are assumed to cover the 'content semantics' of movies by a great degree. The information obtained from multimedia content and learning from muli-modal sources (e.g., audio, visual and metadata) on the other hand, offers the possibility of uncovering relationships between modalities and obtaining an in-depth understanding of natural phenomena occurring in a video. These discerning characteristics of heterogeneous feature sets meet users' differing information needs. In the context of this Ph.D. thesis [9], which is briefly summarized in the current extended abstract, approaches to automated extraction of multimedia information from videos and their integration with video recommender systems have been elaborated, implemented, and analyzed. Variety of tasks related to movie recommendation using multimedia content have been studied. The results of this thesis can motivate the fact that recommender system research can benefit from knowledge in multimedia signal processing and machine learning established over the last decades for solving various recommendation tasks

Video Recommendations Based on Visual Features Extracted with Deep Learning

Postponed access: the file will be accessible after 2022-06-01When a movie is uploaded to a movie Recommender System (e.g., YouTube), the system can exploit various forms of descriptive features (e.g., tags and genre) in order to generate personalized recommendation for users. However, there are situations where the descriptive features are missing or very limited and the system may fail to include such a movie in the recommendation list, known as Cold-start problem. This thesis investigates recommendation based on a novel form of content features, extracted from movies, in order to generate recommendation for users. Such features represent the visual aspects of movies, based on Deep Learning models, and hence, do not require any human annotation when extracted. The proposed technique has been evaluated in both offline and online evaluations using a large dataset of movies. The online evaluation has been carried out in a evaluation framework developed for this thesis. Results from the offline and online evaluation (N=150) show that automatically extracted visual features can mitigate the cold-start problem by generating recommendation with a superior quality compared to different baselines, including recommendation based on human-annotated features. The results also point to subtitles as a high-quality future source of automatically extracted features. The visual feature dataset, named DeepCineProp13K and the subtitle dataset, CineSub3K, as well as the proposed evaluation framework are all made openly available online in a designated Github repository.Masteroppgave i informasjonsvitenskapINFO390MASV-INF

Movie recommendation based on stylistic visual features.

Masteroppgave i informasjonsvitenskapINFO390MASV-INF

Video Abstracting at a Semantical Level

One the most common form of a video abstract is the movie trailer. Contemporary movie trailers share a common structure across genres which allows for an automatic generation and also reflects the corresponding moviea s composition. In this thesis a system for the automatic generation of trailers is presented. In addition to action trailers, the system is able to deal with further genres such as Horror and comedy trailers, which were first manually analyzed in order to identify their basic structures. To simplify the modeling of trailers and the abstract generation itself a new video abstracting application was developed. This application is capable of performing all steps of the abstract generation automatically and allows for previews and manual optimizations. Based on this system, new abstracting models for horror and comedy trailers were created and the corresponding trailers have been automatically generated using the new abstracting models. In an evaluation the automatic trailers were compared to the original Trailers and showed a similar structure. However, the automatically generated trailers still do not exhibit the full perfection of the Hollywood originals as they lack intentional storylines across shots

Japanese Cinema as Mass Art: An Export of Mass Cinema

The aim of the thesis is to examine the distribution and consumption of contemporary Japanese cinema in the UK. Through the study of aesthetics it seeks to identify the ways in which content and current distribution methods suit different markets. Noël Carroll’s definition of cinema and contemporary media as ‘mass art’ has informed the way in which the thesis notes variation in film content, and forms the core methodology. The selected texts range from the post-war period (1948) with films by directors such as Akira Kurosawa, to contemporary animation cinema such as the work of Mamoru Hosada (2009). The distribution methods of these films is of particular importance as the thesis links them to the export of Japanese identity. The films are available to a UK audience via a range of distribution methods such as officially distributed DVDs, online distribution, and art house cinema exhibitions. There is an emphasis on digital distribution throughout the thesis and this is reflected through the use of texts downloaded from the internet as well as digital sources such as university encyclopaedias and e-books. Each chapter is organised around a distinctive and specified market for Japanese cinema and includes detailed textual analysis of film examples. In the final chapter, on the popularity of Japanese animation (anime), the analysis is enhanced with empirical research into online consumption groups and internet communication, given these groups’ important role in the distribution of anime outside of Japan

James Ensor, Paul Haesaerts and Cinéma Critique in 'Masques et Visages de James Ensor' (1952) and 'Moi, Ensor' (1972)

Late nineteenth-century artist James Ensor proved to be a beloved subject for Belgian filmmakers such as Charles Dekeukeleire, Henri Storck, André Cauvin, and Luc de Heusch, who contributed to a veritable boom of mid-century high-quality experimental art documentaries. Highly aware of the power of modern media himself, Ensor showed a great interest in film as well, and he often got involved in the production of documentaries on his artworks. Notwithstanding the significance of the cinematic explorations of his oeuvre, hitherto the interconnections between Ensor’s artistry and cinema have never been substantially investigated. Aiming to remedy this overlooked intersection of art and film, this article focuses on the representation of Ensor in the oeuvre of leading art historian-cum-filmmaker Paul Haesaerts. Following his concept of cinéma critique, Haesaerts’s exceptional 1952 film Masques et visages de James Ensor is a brilliant piece of art analysis. It self-consciously touches upon the ontological boundaries between Ensor’s visual arts and cinema as it wavers between stillness and movement, and between pictorial surface and cinematic space. Drawing on press articles, original archival research, and textual and film analysis, this article investigates how Haesaerts creates a specific cinematic syntax to analyze the formal elements of Ensor’s paintings. In addition, it takes Masques et visages de James Ensor as a starting point for discussions on broader themes concerning the Ensor scholarship, such as Ensor’s affinities with Belgian cinema; the convergence of Ensor’s pictorial strategies and cinematography; and the impact of experimental art documentaries on the reception of Ensor’s work by both filmmakers and art critics

From social tagging to polyrepresentation: a study of expert annotating behavior of moving images

Mención Internacional en el título de doctorThis thesis investigates “nichesourcing” (De Boer, Hildebrand, et al., 2012), an emergent initiative of cultural heritage crowdsoucing in which niches of experts are involved in the annotating tasks. This initiative is studied in relation to moving image annotation, and in the context of audiovisual heritage, more specifically, within the sector of film archives. The work presents a case study of film and media scholars to investigate the types of annotations and attribute descriptions that they could eventually contribute, as well as the information needs, and seeking and searching behaviors of this group, in order to determine what the role of the different types of annotations in supporting their expert tasks would be. The study is composed of three independent but interconnected studies using a mixed methodology and an interpretive approach. It uses concepts from the information behavior discipline, and the "Integrated Information Seeking and Retrieval Framework" (IS&R) (Ingwersen and Järvelin, 2005) as guidance for the investigation. The findings show that there are several types of annotations that moving image experts could contribute to a nichesourcing initiative, of which time-based tags are only one of the possibilities. The findings also indicate that for the different foci in film and media research, in-depth indexing at the content level is only needed for supporting a specific research focus, for supporting research in other domains, or for engaging broader audiences. The main implications at the level of information infrastructure are the requirement for more varied annotating support, more interoperability among existing metadata standards and frameworks, and the need for guidelines about crowdsoucing and nichesourcing implementation in the audiovisual heritage sector. This research presents contributions to the studies of social tagging applied to moving images, to the discipline of information behavior, by proposing new concepts related to the area of use behavior, and to the concept of “polyrepresentation” (Ingwersen, 1992, 1996) applied to the humanities domain.Esta tesis investiga la iniciativa del nichesourcing (De Boer, Hildebrand, et al., 2012), como una forma de crowdsoucing en sector del patrimonio cultural, en la cuál grupos de expertos participan en las tareas de anotación de las colecciones. El ámbito de aplicación es la anotación de las imágenes en movimiento en el contexto del patrimonio audiovisual, más específicamente, en el caso de los archivos fílmicos. El trabajo presenta un estudio de caso aplicado a un dominio específico de expertos en el ámbito audiovisual: los académicos de cine y medios. El análisis se centra en dos aspectos específicos del problema: los tipos de anotaciones y atributos en las descripciones que podrían obtenerse de este nicho de expertos; y en las necesidades de información y el comportamiento informacional de dicho grupo, con el fin de determinar cuál es el rol de los diferentes tipos de anotaciones en sus tareas de investigación. La tesis se compone de tres estudios independientes e interconectados; se usa una metodología mixta e interpretativa. El marco teórico se compone de conceptos del área de estudios de comportamiento informacional (“information behavior”) y del “Marco integrado de búsqueda y recuperación de la información” ("Integrated Information Seeking and Retrieval Framework" (IS&R)) propuesto por Ingwersen y Järvelin (2005), que sirven de guía para la investigación. Los hallazgos indican que existen diversas formas de anotación de la imagen en movimiento que podrían generarse a partir de las contribuciones de expertos, de las cuáles las etiquetas a nivel de plano son sólo una de las posibilidades. Igualmente, se identificaron diversos focos de investigación en el área académica de cine y medios. La indexación detallada de contenidos sólo es requerida por uno de esos grupos y por investigadores de otras disciplinas, o como forma de involucrar audiencias más amplias. Las implicaciones más relevantes, a nivel de la infraestructura informacional, se refieren a los requisitos de soporte a formas más variadas de anotación, el requisito de mayor interoperabilidad de los estándares y marcos de metadatos, y la necesidad de publicación de guías de buenas prácticas sobre de cómo implementar iniciativas de crowdsoucing o nichesourcing en el sector del patrimonio audiovisual. Este trabajo presenta aportes a la investigación sobre el etiquetado social aplicado a las imágenes en movimiento, a la disciplina de estudios del comportamiento informacional, a la que se proponen nuevos conceptos relacionados con el área de uso de la información, y al concepto de “poli-representación” (Ingwersen, 1992, 1996) en las disciplinas humanísticas.Programa Oficial de Doctorado en Documentación: Archivos y Bibliotecas en el Entorno DigitalPresidente: Peter Emil Rerup Ingwersen.- Secretario: Antonio Hernández Pérez.- Vocal: Nils Phar

DE LA MODELISATION A L'EXPLOITATION DES DOCUMENTS A STRUCTURES MULTIPLES

With the recent development of new information and communication technologies, the paper documents are transformed to digital documents. Furthermore, it considers that the document is no longer seen as a whole, or as a monolithic bloc, but as organized entities. Exploiting these documents amount to identify and locate these entities. These entities are connected by relationships to give a "form" to document. Several types of relationships may occur, so that several "forms" of a document emerge. These different materializations of the same document are related to different uses of the same document and are essential for optimal management and shared of holdings. The work presented in this thesis aims to address the challenges of representing different materializations of a document through its representation of entities and their relationships. If those materializations are translated through structures, the issues are related to the representation of multistructured documents. Our work focuses mainly on the modeling, integration and exploitation of multistructured documents: (1) Proposal of multistructured document model. This model incorporates two levels of description: a specific level to describe each document through entities that compose and a generic level to identify document kinds through the grouping of similar structures. (2) Proposal of techniques for extracting structure (implicit or explicit) of a document (the specific level) and classification of this structure with respect to common structures (the generic level). The classification algorithm proposed includes a calculation of distance called "structural" (comparison of trees and graphs). This classification is associated with a process of verification of the "cohesion" of classes and possible reorganization of disrupted classes. (3) Proposal of document exploitation technical from their structures and their contents: (a) a document search that can reproduce documentary granules through criteria based on research of structures and / or content, (b) a multidimensional analysis that is to analyze and visualize the documentary information across multiple dimensions (of structures and / or content). In order to validate our proposals, we have developed a tool for integration and analysis of multistructured documents, called MDOCREP (Multistructured Document Repository). This tool provides on the one hand, the extraction and classification of document structures, and on the other hand, the querying and the multidimensional analysis of documents from their different structures.Avec l'évolution des nouvelles technologies de l'information et de la communication, les documents papier ont laissé la place aux documents numériques. On considère de plus que le document n'est plus vu comme un tout, ni comme un bloc monolithique, mais comme un ensemble organisé d'entités. Exploiter ces documents revient à identifier et retrouver ces entités. Ces dernières sont reliées par des relations permettant de donner une « forme » au document. Plusieurs types de relations peuvent apparaître, de sorte à ce que plusieurs « formes » d'un même document émergent. Ces différentes matérialisations d'un même document sont liées à des usages différents d'un même document et sont primordiales pour une gestion optimale et partagée des fonds documentaires. Les travaux présentés dans cette thèse visent à faire face aux défis de représentation des différentes matérialisations d'un document au travers de la représentation de ses entités et de leurs relations. Si ces matérialisations sont traduites par des structures, les enjeux concernent la représentation des documents à structures multiples. Nos travaux portent essentiellement sur la modélisation, l'intégration et l'exploitation des documents à structures multiples : (1) Proposition d'un modèle de documents multistructurés. Ce modèle intègre deux niveaux de description : un niveau spécifique permettant de décrire chaque document au travers des entités qui le composent et un niveau générique permettant de définir des typologies de documents au travers du regroupement de structures similaires. (2) Proposition des techniques d'extraction de structure (implicite ou explicite) d'un document (niveau spécifique) et de classification de cette structure par rapport à des structures communes (niveau générique). L'algorithme de classification proposé intègre un calcul d'une distance dite « structurelle » (comparaison d'arbres et de graphes). Cette démarche de classification est associée à une démarche de vérification de la « cohésion » des classes et de réorganisation éventuelle des classes perturbées. (3) Proposition de techniques d'exploitation des documents à partir de leurs structures et de leur contenu : (a) une recherche de documents qui permet de restituer des granules documentaires selon des critères de recherches basés sur la ou les structures et/ou le contenu ; (b) une analyse multidimensionnelle qui consiste à analyser et visualiser les informations documentaires selon plusieurs dimensions (de structures et/ou de contenu). Pour valider nos propositions, nous avons développé un outil d'aide à l'intégration et à l'analyse de documents à structures multiples, intitulé MDOCREP (Multistructured DOCument REPository). Cet outil assure d'une part, l'extraction et la classification des structures de documents, et d'autre part, l'interrogation et la restitution multidimensionnelle des documents à partir de leurs différentes structures

Video Vortex reader : responses to Youtube

The Video Vortex Reader is the first collection of critical texts to deal with the rapidly emerging world of online video – from its explosive rise in 2005 with YouTube, to its future as a significant form of personal media. After years of talk about digital convergence and crossmedia platforms we now witness the merger of the Internet and television at a pace no-one predicted. These contributions from scholars, artists and curators evolved from the first two Video Vortex conferences in Brussels and Amsterdam in 2007 which focused on responses to YouTube, and address key issues around independent production and distribution of online video content. What does this new distribution platform mean for artists and activists? What are the alternatives