Fusing Multimedia Data Into Dynamic Virtual Environments

In spite of the dramatic growth of virtual and augmented reality (VR and AR) technology, content creation for immersive and dynamic virtual environments remains a significant challenge. In this dissertation, we present our research in fusing multimedia data, including text, photos, panoramas, and multi-view videos, to create rich and compelling virtual environments. First, we present Social Street View, which renders geo-tagged social media in its natural geo-spatial context provided by 360° panoramas. Our system takes into account visual saliency and uses maximal Poisson-disc placement with spatiotemporal filters to render social multimedia in an immersive setting. We also present a novel GPU-driven pipeline for saliency computation in 360° panoramas using spherical harmonics (SH). Our spherical residual model can be applied to virtual cinematography in 360° videos. We further present Geollery, a mixed-reality platform to render an interactive mirrored world in real time with three-dimensional (3D) buildings, user-generated content, and geo-tagged social media. Our user study has identified several use cases for these systems, including immersive social storytelling, experiencing the culture, and crowd-sourced tourism. We next present Video Fields, a web-based interactive system to create, calibrate, and render dynamic videos overlaid on 3D scenes. Our system renders dynamic entities from multiple videos, using early and deferred texture sampling. Video Fields can be used for immersive surveillance in virtual environments. Furthermore, we present VRSurus and ARCrypt projects to explore the applications of gestures recognition, haptic feedback, and visual cryptography for virtual and augmented reality. Finally, we present our work on Montage4D, a real-time system for seamlessly fusing multi-view video textures with dynamic meshes. We use geodesics on meshes with view-dependent rendering to mitigate spatial occlusion seams while maintaining temporal consistency. Our experiments show significant enhancement in rendering quality, especially for salient regions such as faces. We believe that Social Street View, Geollery, Video Fields, and Montage4D will greatly facilitate several applications such as virtual tourism, immersive telepresence, and remote education

Proceedings of the 10th international conference on disability, virtual reality and associated technologies (ICDVRAT 2014)

The proceedings of the conferenc

Enchanted objects: agency in the magic act and contemporary art practice

The following is taken directly from the abstract: 'In my research project I examine some of the ways in which the objects,strategies, and concepts of conjuring—or what Simon During has called ‘secular’magic—might be seen to converge with those of contemporary art practice. The theoretical concepts that I employ derive principally from Alfred Gell’s (anthropological) theory of art and agency. In Gell’s theory, an index/artwork is a mediatory (or secondary) agent, but an agent nonetheless, through which the (primary) agency of a social other can be communicated. Gell’s concept of enchantment, but also his interpretation of the status of the artwork as provisional and problematic, rather than aesthetically or semiotically determined, is deployed as a means of creating a productively meaningful relationship between art and magic, both of which can be said to occlude the ‘abduction’ of agency in distinctive ways. Finally, Gell’s concept of agency provides a robust yet fluid set of paradigms for exploring the mobile, tripartite relationship between artist,artwork, and spectator.

Decoupling and context in new media art

This dissertation presents a novel characterization of new media art, centered on media appropriation: the dialectal insertion of technological knowledge into the art practice. The thesis identifies some defining characteristics of new media art’s language, and indicates the defining role that explicitation plays. While media appropriation is not necessarily linked to the digital realm, it provides a natural substratum for it and so this thesis analyzes some aspects of the relationship between art and technology, where it introduces the user–programmer continuum and the perceptual cloud, a new paradigm of human–computer interaction that emerges from the functional and geographical decoupling of the computational and perceptual layers of interactive systems. Next, it analyzes the sociopolitical inscription of new media art, integrating the economic and political contexts of its practice into the analysis and providing a new reflection on new media art production from the geopolitical periphery. This thesis is proposed as a hybrid research–practice. A selected subset of the artworks created are presented and analyzed within the dissertation’s conceptual framework.Esta disertación presenta una nueva caracterización del new media art, centrada en la apropiación de los medios, es decir, en la inserción dialéctica de conocimiento tecnológico dentro de la práctica artística. La tesis identifica algunas características definitorias del lenguaje del new media art, e identifica el rol fundamental que la explicitación juega. Aunque la apropiación de los medios no está necesariamente unida a lo digital, éste provee un substrato natural para ella. Por ello, esta tesis analiza algunos aspectos entre el arte y la tecnología digital, introduciendo el continuo usuario\2013programador y la nube perceptual, un nuevo paradigma de interacción humano\2013computadora que emerge del desacople funcional y geográfico de las capas computacionales y perceptuales de los sistemas interactivos. A continuación, se analiza la inscripción sociopolítica del new media art, integrando los contextos económico y político, proveyendo una nueva reflexión acerca de la producción artística desde la periferia geopolítica. Esta tesis se propone como un híbrido investigación\2013producción. Un subconjunto seleccionado de las obras creadas durante el programa son presentadas y analizadas desde el marco conceptual de la disertación

Technology utilization program report, 1974

The adaptation of various technological innovations from the NASA space program to industrial and domestic applications is summarized

Cultural Techniques: Assembling Spaces, Texts & Collectives

Addressing cultural techniques from different disciplinary perspectives, this volume elaborates upon a concept originally developed in media studies. In a series of case studies, it reconstructs the basic operations of spatialization underlying more complex symbolic artefacts and articulations, which range from techniques of the body to landscapes, from paperwork to encyclopedias, from collections to collectives

Analyzing the Impact of Spatio-Temporal Sensor Resolution on Player Experience in Augmented Reality Games

Along with automating everyday tasks of human life, smartphones have become one of the most popular devices to play video games on due to their interactivity. Smartphones are embedded with various sensors which enhance their ability to adopt new new interaction techniques for video games. These integrated sen- sors, such as motion sensors or location sensors, make the device able to adopt new interaction techniques that enhance usability. However, despite their mobility and embedded sensor capacity, smartphones are limited in processing power and display area compared to desktop computer consoles. When it comes to evaluat- ing Player Experience (PX), players might not have as compelling an experience because the rich graphics environments that a desktop computer can provide are absent on a smartphone. A plausible alternative in this regard can be substituting the virtual game world with a real world game board, perceived through the device camera by rendering the digital artifacts over the camera view. This technology is widely known as Augmented Reality (AR). Smartphone sensors (e.g. GPS, accelerometer, gyro-meter, compass) have enhanced the capability for deploying Augmented Reality technology. AR has been applied to a large number of smartphone games including shooters, casual games, or puzzles. Because AR play environments are viewed through the camera, rendering the digital artifacts consistently and accurately is crucial because the digital characters need to move with respect to sensed orientation, then the accelerometer and gyroscope need to provide su ciently accurate and precise readings to make the game playable. In particular, determining the pose of the camera in space is vital as the appropriate angle to view the rendered digital characters are determined by the pose of the camera. This defines how well the players will be able interact with the digital game characters. Depending in the Quality of Service (QoS) of these sensors, the Player Experience (PX) may vary as the rendering of digital characters are affected by noisy sensors causing a loss of registration. Confronting such problem while developing AR games is di cult in general as it requires creating wide variety of game types, narratives, input modalities as well as user-testing. Moreover, current AR games developers do not have any specific guidelines for developing AR games, and concrete guidelines outlining the tradeoffs between QoS and PX for different genres and interaction techniques are required. My dissertation provides a complete view (a taxonomy) of the spatio-temporal sensor resolution depen- dency of the existing AR games. Four user experiments have been conducted and one experiment is proposed to validate the taxonomy and demonstrate the differential impact of sensor noise on gameplay of different genres of AR games in different aspect of PX. This analysis is performed in the context of a novel instru- mentation technology, which allows the controlled manipulation of QoS on position and orientation sensors. The experimental outcome demonstrated how the QoS of input sensor noise impacts the PX differently while playing AR game of different genre and the key elements creating this differential impact are - the input modality, narrative and game mechanics. Later, concrete guidelines are derived to regulate the sensor QoS as complete set of instructions to develop different genres or AR games

Interactive Fiction in Cinematic Virtual Reality: Epistemology, Creation and Evaluation

This dissertation presents the Interactive Fiction in Cinematic Virtual Reality (IFcVR), an interactive digital narrative (IDN) that brings together the cinematic virtual reality (cVR) and the creation of virtual environments through 360\ub0 video within an interactive fiction (IF) structure. This work is structured in three components: an epistemological approach to this kind of narrative and media hybrid; the creation process of IFcVR, from development to postproduction; and user evaluation of IFcVR. In order to set the foundations for the creation of interactive VR fiction films, I dissect the IFcVR by investigating the aesthetics, narratological and interactive notions that converge and diverge in it, proposing a medium-conscious narratology for this kind of artefact. This analysis led to the production of an IFcVR functional prototype: \u201cZENA\u201d, the first interactive VR film shot in Genoa. ZENA\u2019s creation process is reported proposing some guidelines for interactive and immersive film-makers. In order to evaluate the effectiveness of the IFcVR as an entertaining narrative form and a vehicle for diverse types of messages, this study also proposes a methodology to measure User Experience (UX) on IFcVR. The full evaluation protocol gathers both qualitative and quantitative data through ad hoc instruments. The proposed protocol is illustrated through its pilot application on ZENA. Findings show interactors' positive acceptance of IFcVR as an entertaining experience

Cultural Techniques

Addressing cultural techniques from different disciplinary perspectives, this volume elaborates upon a concept originally developed in media studies. In a series of case studies, it reconstructs the basic operations of spatialization underlying more complex symbolic artefacts and articulations, which range from techniques of the body to landscapes, from paperwork to encyclopedias, from collections to collectives