Interaction and Expressivity in Video Games: Harnessing the Rhetoric of Film

The film-maker uses the camera and editing creatively, not simply to present the action of the film but also to set up a particular relation between the action and the viewer. In 3D video games with action controlled by the player, the pseudo camera is usually less creatively controlled and has less effect on the player’s appreciation of and engagement with the game. This paper discusses methods of controlling games by easy and intuitive interfaces and use of an automated virtual camera to increase the appeal of games for users

Text-based Editing of Talking-head Video

Editing talking-head video to change the speech content or to remove filler words is challenging. We propose a novel method to edit talking-head video based on its transcript to produce a realistic output video in which the dialogue of the speaker has been modified, while maintaining a seamless audio-visual flow (i.e. no jump cuts). Our method automatically annotates an input talking-head video with phonemes, visemes, 3D face pose and geometry, reflectance, expression and scene illumination per frame. To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material. The annotated parameters corresponding to the selected segments are seamlessly stitched together and used to produce an intermediate video representation in which the lower half of the face is rendered with a parametric face model. Finally, a recurrent video generation network transforms this representation to a photorealistic video that matches the edited transcript. We demonstrate a large variety of edits, such as the addition, removal, and alteration of words, as well as convincing language translation and full sentence synthesis

GeoCamera: Telling Stories in Geographic Visualizations with Camera Movements

In geographic data videos, camera movements are frequently used and combined to present information from multiple perspectives. However, creating and editing camera movements requires significant time and professional skills. This work aims to lower the barrier of crafting diverse camera movements for geographic data videos. First, we analyze a corpus of 66 geographic data videos and derive a design space of camera movements with a dimension for geospatial targets and one for narrative purposes. Based on the design space, we propose a set of adaptive camera shots and further develop an interactive tool called GeoCamera. This interactive tool allows users to flexibly design camera movements for geographic visualizations. We verify the expressiveness of our tool through case studies and evaluate its usability with a user study. The participants find that the tool facilitates the design of camera movements.Comment: 15 pages. Published as a conference paper at the ACM Conference on Human Factors in Computing Systems (CHI) 202

Weather-driven interactive video

To build this weather-driven interactive video feature, one ingredient, was necessary to know the conditions of the environment of the user, to do so with greater accuracy, a simplified version of a weather station was made using an Arduino and some sensors to measure the environment, the weather of the user’s location. Another necessary element, was a tool that could use this collected data and convert it in a meaningful way to the user, contextualise the video content. A digital prototype was developed, combining the framework Processing and the data from the Arduino, that would present the user, the viewer with a video, story that would be contextualised to the user environment, current weather conditions. These videos were a set of 6 parallel stories, not simple weather effects but live video production, aiming to improve the connection with the users between the scene actions and their own environment. To evaluate the relationship between the video weather effects, its content and the user’s context, tests were made to measure the impact of this feature on the user experience, resulting in insights, that help to influence and enhance the user’s experience. Some suggest that, contextualising the video setting to the user’s current environment setting, might not be ideal, even detrimental while other might be beneficial to the user experience.Para construir este projeto de vídeo interativo orientado ao clima, era necessário saber as condições do ambiente em que o utilizador se encontrava e, para fazê-lo com maior precisão, foi construída uma versão simplificada de uma estação meteorológica usando um Arduino e alguns sensores, que fariam a medição do ambiente, da localização do usuário. Outro elemento necessário, era uma ferramenta que pudesse utilizar estes dados e convertê-los, de uma forma significativa para o utilizador, contextualizando o conteúdo do vídeo ao seu ambiente para enaltecer a sua experiência. Para tal, foi desenvolvido um protótipo digital, combinando o framework Processing e os dados do Arduino, que apresentaria ao espectador, um vídeo, uma história que seria contextualizada ao seu ambiente, às suas condições meteorológicas atuais. Estes vídeos são um conjunto de 6 histórias paralelas e não apenas simples vídeo com efeitos meteorológicos. São produção de vídeo ao vivo, com o objetivo de melhorar a conexão com os utilizadores, melhor a confecção entre as ações da cena e seu próprio ambiente, o do espetador. Para avaliar a relação entre os efeitos meteorológicos do vídeo, o seu conteúdo e o contexto do usuário, testes foram feitos para medir esse impacto na experiência do utilizador, resultando em percepções que ajudaram a influenciar e aprimorar a experiência do usuário. Estas descobertas sugerem que, contextualizar a configuração do vídeo com o ambiente atual do espectador, não será o ideal, poderá até mesmo ser prejudicial, enquanto os restantes cenários podem ser benéficos para a experiência do utilizador