Navigation and interaction in a real-scale digital mock-up using natural language and user gesture

This paper tries to demonstrate a very new real-scale 3D system and sum up some firsthand and cutting edge results concerning multi-modal navigation and interaction interfaces. This work is part of the CALLISTO-SARI collaborative project. It aims at constructing an immersive room, developing a set of software tools and some navigation/interaction interfaces. Two sets of interfaces will be introduced here: 1) interaction devices, 2) natural language (speech processing) and user gesture. The survey on this system using subjective observation (Simulator Sickness Questionnaire, SSQ) and objective measurements (Center of Gravity, COG) shows that using natural languages and gesture-based interfaces induced less cyber-sickness comparing to device-based interfaces. Therefore, gesture-based is more efficient than device-based interfaces.FUI CALLISTO-SAR

Collaborative and Multi-Modal Mixed Reality for Enhancing Cultural Learning in Virtual Heritage

No abstrac

Developing a Framework for Heterotopias as Discursive Playgrounds: A Comparative Analysis of Non-Immersive and Immersive Technologies

The discursive space represents the reordering of knowledge gained through accumulation. In the digital age, multimedia has become the language of information, and the space for archival practices is provided by non-immersive technologies, resulting in the disappearance of several layers from discursive activities. Heterotopias are unique, multilayered epistemic contexts that connect other systems through the exchange of information. This paper describes a process to create a framework for Virtual Reality, Mixed Reality, and personal computer environments based on heterotopias to provide absent layers. This study provides virtual museum space as an informational terrain that contains a "world within worlds" and presents place production as a layer of heterotopia and the subject of discourse. Automation for the individual multimedia content is provided via various sorting and grouping algorithms, and procedural content generation algorithms such as Binary Space Partitioning, Cellular Automata, Growth Algorithm, and Procedural Room Generation. Versions of the framework were comparatively evaluated through a user study involving 30 participants, considering factors such as usability, technology acceptance, and presence. The results of the study show that the framework can serve diverse contexts to construct multilayered digital habitats and is flexible for integration into professional and daily life practices

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe

Beyond the Electronic Connection: The Technologically Manufactured Cyber-Human and Its Physical Human Counterpart in Performance: A Theory Related to Convergence Identities

This thesis is an investigation of the complex processes and relationships between the physical human performer and the technologically manufactured cyber-human counterpart. I acted as both researcher and the physical human performer, deeply engaged in the moment-to-moment creation of events unfolding within a shared virtual reality environment. As the primary instigator and activator of the cyber-human partner, I maintained a balance between the live and technological performance elements, prioritizing the production of content and meaning. By way of using practice as research, this thesis argues that in considering interactions between cyber-human and human performers, it is crucial to move beyond discussions of technology when considering interactions between cyber-humans and human performers to an analysis of emotional content, the powers of poetic imagery, the trust that is developed through sensory perception and the evocation of complex relationships. A theoretical model is constructed to describe the relationship between a cyber-human and a human performer in the five works created specifically for this thesis, which is not substantially different from that between human performers. Technological exploration allows for the observation and analysis of various relationships, furthering an expanded understanding of ‘movement as content’ beyond the electronic connection. Each of the works created for this research used new and innovative technologies, including virtual reality, multiple interactive systems, six generations of wearable computers, motion capture technology, high-end digital lighting projectors, various projection screens, smart electronically charged fabrics, multiple sensory sensitive devices and intelligent sensory charged alternative performance spaces. They were most often collaboratively created in order to augment all aspects of the performance and create the sense of community found in digital live dance performances/events. These works are identified as one continuous line of energy and discovery, each representing a slight variation on the premise that a working, caring, visceral and poetic content occurs beyond the technological tools. Consequently, a shift in the physical human’s psyche overwhelms the act of performance. Scholarship and reflection on the works have been integral to my creative process throughout. The goals of this thesis, the works created and the resulting methodologies are to investigate performance to heighten the multiple ways we experience and interact with the world. This maximizes connection and results in a highly interactive, improvisational, dynamic, non-linear, immediate, accessible, agential, reciprocal, emotional, visceral and transformative experience without boundaries between the virtual and physical for physical humans, cyborgs and cyber-humans alike.College of Fine Arts at the University of Texas at Austin, Department of Theatre & Dance at the University of Texas at Austi

Emerging ExG-based NUI Inputs in Extended Realities : A Bottom-up Survey

Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduceto the areaof XR.Peer reviewe