Acesso a filmes com base em emoções

Tese de mestrado em Engenharia Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2011Os filmes constituem uma das fontes mais importantes de entretenimento, tanto em contextos individuais como sociais. Através da combinação de diversos sistemas de símbolos, tais como imagens, textos, áudio e narração para criar histórias, frequentemente envolvem o espectador perceptual, cognitiva e emocionalmente. Os avanços tecnológicos permitem o acesso a colecções enormes de vídeos e filmes pela Internet, nas redes sociais, ou através de Video on Demand, que se vem tornando uma actividade de entretenimento em crescendo na Internet e na iTV (televisão interactiva). O desenvolvimento de técnicas de análise e classificação do vídeo baseadas no conteúdo permitem o acesso a mais informações sobre ou contidas nos filmes, exigindo o desenvolvimento de novos mecanismos de pesquisa, navegação e visualização de vídeos e filmes neste cenário. Este trabalho apresenta diferentes mecanismos de acesso, sumarização, exploração e visualização de filmes baseados nas suas propriedades emocionais, expressas nos filmes ou sentidas pelos espectadores. Com esse intuito, foi feito um estudo sobre o impacto emocional dos filmes e foram concebidos dois protótipos: O iFelt, onde os mecanismos desenvolvidos se focam em emoções sentidas, para aceder e interagir com filmes e cenas dos filmes; e o movieClouds, que enquadra também emoções sentidas, mas foca-se em emoções expressas nos filmes e veiculadas nas diferentes perspectivas do seu conteúdo, especialmente no áudio e nas legendas, onde a maioria da semântica é expressa. Ambos os protótipos foram avaliados com utilizadores, para perceber se eram úteis, satisfatórios e fáceis de utilizar, avaliar a sua usabilidade e obter opiniões e comentários. Os resultados foram bastante positivos e animadores. Apesar de alguns conceitos novos nas primeiras interacções, os utilizadores apreciaram a ideia de utilizar as propriedades emocionais para interagir com filmes, desde as perspectivas mais globais ao nível do espaço de filmes, até aos filmes, as suas cenas, e a sua evolução ao longo do tempo em timelines, nas diferentes perspectivas.Movies are one of the main sources of entertainment, in individual and social contexts. By combining diverse symbol systems, such as images, texts, audio and narration to create stories, they often engage the viewer’s perceptually, cognitively and emotionally. Technological advances are enabling the access to enormous collections of videos and movies over the Internet, in social media, and through video on demand services, that is becoming a growing entertainment activity on the Internet and iTV (interactive television). The development of video content-based analysis and classification techniques is also allowing the access to more information about or contained in the movies, demanding for new ways to search, browse and view videos and movies in this scenario. This work presents different mechanisms to access, summarize, explore and visualize movies based on their emotional properties, either expressed in the movies or felt by the viewers. For this, were designed and developed two prototypes: iFelt, where the mechanisms are focused on felt emotions to interact with movies and movie scenes; and movieClouds, that also uses felt emotions, but has a stronger focus on expressed emotions on movies, conveyed in the different tracks or perspectives of its content, especially audio and subtitles, where most of the semantics is expressed. Both prototypes were evaluated with users to get insights and feedback about their usefulness, satisfaction and ease to use. The results were very positive and quite encouraging. Although some concepts were new in the first interactions, users loved the idea of taking advantage of emotional properties to interact with movies, to overview, watch and browse movies, from a global perspective in the movie space, down to the movies, its scenes, and the timelines, where specific emotions and events can be accessed along time, in the different perspectives

Multimedia interaction and access based on emotions:automating video elicited emotions recognition and visualization

Tese de doutoramento, Informática (Engenharia Informática), Universidade de Lisboa, Faculdade de Ciências, 2013Films are an excellent form of art that exploit our affective, perceptual and intellectual abilities. Technological developments and the trends for media convergence are turning video into a dominant and pervasive medium, and online video is becoming a growing entertainment activity on the web. Alongside, physiological measures are making it possible to study additional ways to identify and use emotions in human-machine interactions, multimedia retrieval and information visualization. The work described in this thesis has two main objectives: to develop an Emotions Recognition and Classification mechanism for video induced emotions; and to enable Emotional Movie Access and Exploration. Regarding the first objective, we explore recognition and classification mechanisms, in order to allow video classification based on emotions, and to identify each user’s emotional states providing different access mechanisms. We aim to provide video classification and indexing based on emotions, felt by the users while watching movies. In what concerns the second objective, we focus on emotional movie access and exploration mechanisms to find ways to access and visualize videos based on their emotional properties and users’ emotions and profiles. In this context, we designed a set of methods to access and watch the movies, both at the level of the whole movie collection, and at the individual movies level. The automatic recognition mechanism developed in this work allows for the detection of physiologic patterns, indeed providing valid individual information about users emotion while they were watching a specific movie; in addition, the user interface representations and exploration mechanisms proposed and evaluated in this thesis, show that more perceptive, satisfactory and useful visual representations influenced positively the exploration of emotional information in movies.Fundação para a Ciência e a Tecnologia (FCT, PROTEC SFRH/BD/49475/2009, LASIGE Multiannual Funding e VIRUS projecto (PTDC/EIAEIA/101012/2008