MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Model

Face-to-face communication is a common scenario including roles of speakers and listeners. Most existing research methods focus on producing speaker videos, while the generation of listener heads remains largely overlooked. Responsive listening head generation is an important task that aims to model face-to-face communication scenarios by generating a listener head video given a speaker video and a listener head image. An ideal generated responsive listening video should respond to the speaker with attitude or viewpoint expressing while maintaining diversity in interaction patterns and accuracy in listener identity information. To achieve this goal, we propose the \textbf{M}ulti-\textbf{F}aceted \textbf{R}esponsive Listening Head Generation Network (MFR-Net). Specifically, MFR-Net employs the probabilistic denoising diffusion model to predict diverse head pose and expression features. In order to perform multi-faceted response to the speaker video, while maintaining accurate listener identity preservation, we design the Feature Aggregation Module to boost listener identity features and fuse them with other speaker-related features. Finally, a renderer finetuned with identity consistency loss produces the final listening head videos. Our extensive experiments demonstrate that MFR-Net not only achieves multi-faceted responses in diversity and speaker identity information but also in attitude and viewpoint expression.Comment: Accepted by ACM MM 202

Volumetric kombat:a case study on developing a VR game with Volumetric Video

This paper presents a case study on the development of a Virtual Reality (VR) game using Volumetric Video (VV) for character animation. We delve into the potential of VV, a technology that fuses video and depth sensor data, which has progressively matured since its initial introduction in 1995. Despite its potential to deliver unmatched realism and dynamic 4D sequences, VV applications are predominantly used in non-interactive scenarios. We explore the barriers to entry such as high costs associated with large-scale VV capture systems and the lack of tools optimized for VV in modern game engines. By actively using VV to develop a VR game, we examine and overcome these constraints developing a set of tools that address these challenges. Drawing lessons from past games, we propose an open-source data processing workflow for future VV games. This case study provides insights into the opportunities and challenges of VV in game development and contributes towards making VV more accessible for creators and researchers

Game | World | Architectonics

In its current digital, pictorial and viral ubiquity, architecture no longer has to be bodily present, but has a mediating role. As a medial hinge it folds different disciplines of media and art onto the realm of the everyday. Here, the idea of architectonics can be understood as the architectural implications of computer games in a broader sense to address the matter of architecture in game worlds as well as the architecture of computer games themselves. This anthology bundles transdisciplinary approaches around the topics of space, architecture, perception of and worldbuilding in computer games and their media-specific properties. The aim is to show how and under which aspects digital game worlds are constituted. The contributions depart from the beaten tracks of media and game studies, focusing on spatial, architectural and world-shaped phenomena within current digital media culture.In ihrer aktuellen digitalen, bildlichen wie auch viralen Ubiquität muss Architektur nicht mehr körperlich präsent sein und doch füllt sie eine vermittelnde Rolle aus. Als mediales Scharnier verschränkt sie unterschiedliche Disziplinen der Medien und Künste mit der Alltagswirklichkeit. Das Konzept der Architektonik umschreibt hierbei in weitem Sinne die architektonischen Implikationen der Computerspiele, um Architektur in Spielwelten als auch die Architektur der Computerspiele selbst greifbar zu machen. Dieser Sammelband bündelt transdisziplinäre Zugriffe rund um die Themen Raum, Architektur, Wahrnehmung von und Weltenbau in Computerspielen und deren medienspezifischen Eigenschaften. Ziel ist es aufzuzeigen, wie und unter welchen Aspekten sich digitale Spielwelten konstituieren. Die Beiträge verlassen dabei ausgetretene Pfade von Medienwissenschaft und Game Studies und fokussieren auf die räumlichen, architektonischen und weltförmigen Phänomene aktueller digitaler Medienkultur

Virtual YouTubers’ Self-Representation Between Extended and Divided Self

Virtual and Artificial YouTubers (VTubers) show us how the body becomes technologically embedded. They reveal arising complexities within the interface of digital and analog assemblies, bodies, and virtual environments. Thus, VTubers raise questions that are crucial to the core debate about personhood and the human subject in anthropology as well as critical posthumanism. By reading Feminist Anthropology and Critical Posthumanism dos-à-dos, the thesis engages with the three VTubers AI Angelica, CodeMiko, and Miquela Sousa. To answer the questions (1) how personhood unfolds in the VTubers’ self-representation(s), (2) how personhood is negotiated with the recipients, and (3) which aspects of the human subject (e.g., gender, race) are reproduced a methodological framework of Netnography and Critical Technocultural Discourse Analysis is applied. The thesis reveals that VTubers’ show a form of personhood in which the reflective self appears and speaks apart from the ‘I.’ This division reflects practices of self-designation in order to navigate between the extended self and the divided self; the content creator and the avatar; between the platform and the VTuber. This way, the self manifests itself simultaneously in the form of overlaps and displacements. Within this form of relationality, the notion of the glitch is reviewed to consider the VTuber’s personhood in respect of the discussion between critical posthumanist and humanist perspectives

Current trends on ICT technologies for enterprise information s²ystems

The proposed paper discusses the current trends on ICT technologies for Enterprise Information Systems. The paper starts by defining four big challenges of the next generation of information systems: (1) Data Value Chain Management; (2) Context Awareness; (3) Interaction and Visualization; and (4) Human Learning. The major contributions towards the next generation of information systems are elaborated based on the work and experience of the authors and their teams. This includes: (1) Ontology based solutions for semantic interoperability; (2) Context aware infrastructures; (3) Product Avatar based interactions; and (4) Human learning. Finally the current state of research is discussed highlighting the impact of these solutions on the economic and social landscape

FROM CELLULOID REALITIES TO BINARY DREAMSCAPES: CINEMA AND PERCEPTUAL EXPERIENCE IN THE AGE OF DIGITAL IMMERSION

Technologies in digital cinema are quickly changing the way contemporary filmmakers create films and how audiences currently perceive them. As we move onward into the digital turn, it becomes ever more apparent that the medium of film has been emancipated from its dependence on the photograph. Directors are no longer required to capture the objectively real as it sits before the photographic lens, but can essentially construct it via groundbreaking advancements in computer-generated imagery, motion capture technology, and digital 3D camera systems and display technologies. Since the origins of film, spectators and filmmakers have assumed an existing relationship between reality and the photographic image. Yet digital film technologies now provide us with hyper-facsimiles of reality that are perceived as photographic, but are often created by way of computer processes. Digital cinema currently allows the viewer to inhabit and interact with cinematic realities in unprecedented ways, and it is this contemporary paradigmatic shift from the analog to the digital that has catalyzed fundamentally new ways of looking at the filmic image. In this paper, I will examine the perceptual complexities of contemporary digital film through the lens of these cinematic technologies by examining their impact on the viewer’s experience