Navigation and interaction in a real-scale digital mock-up using natural language and user gesture

This paper tries to demonstrate a very new real-scale 3D system and sum up some firsthand and cutting edge results concerning multi-modal navigation and interaction interfaces. This work is part of the CALLISTO-SARI collaborative project. It aims at constructing an immersive room, developing a set of software tools and some navigation/interaction interfaces. Two sets of interfaces will be introduced here: 1) interaction devices, 2) natural language (speech processing) and user gesture. The survey on this system using subjective observation (Simulator Sickness Questionnaire, SSQ) and objective measurements (Center of Gravity, COG) shows that using natural languages and gesture-based interfaces induced less cyber-sickness comparing to device-based interfaces. Therefore, gesture-based is more efficient than device-based interfaces.FUI CALLISTO-SAR

An expandable walking in place platform

The control of locomotion in 3D virtual environments should be an ordinary task, from the user point-of-view. Several navigation metaphors have been explored to control locomotion naturally, such as: real walking, the use of simulators, and walking in place. These have proven that the more natural the approach used to control locomotion, the more immerse the user will feel inside the virtual environment. Overcoming the high cost and complexity for the use of most approaches in the field, we introduce a walking in place platform that is able to identify orientation, speed for displacement, as well as lateral steps, of a person mimicking walking pattern. The detection of this information is made without use of additional sensors attached to user body. Our device is simple to mount, inexpensive and allows almost natural use, with lazy steps, thus releasing the hands for other uses. Also, we explore and test a passive, tactile surface for safe use of our platform. The platform was conceived to be utilized as an interface to control navigation in virtual environments, and augmented reality. Extending our device and techniques, we have elaborated a redirection walking metaphor, to be used together with a cave automatic virtual environment. Another metaphor allowed the use of our technique for navigating in point clouds for tagging of data. We tested the use of our technique associated with two different navigation modes: human walking and vehicle driving. In the human walking approach, the virtual orientation inhibits the displacement when sharp turns are made by the user. In vehicle mode, the virtual orientation and displacement occur together, more similar to a vehicle driving approach. We applied tests to detect preferences of navigation mode and ability to use our device to 52 subjects. We identified a preference for the vehicle driving mode of navigation. The use of statistics revealed that users learned easily the use of our technique for navigation. Users were faster walking in vehicle mode; but human mode allowed precise walking in the virtual test environment. The tactile platform proved to allow safe use of our device, being an effective and simple solution for the field. More than 200 people tested our device: UFRGS Portas Abertas in 2013 and 2014, which was a event to present to local community academic works; during 3DUI 2014, where our work was utilized together with a tool for point cloud manipulation. The main contributions of our work are a new approach for detection of walking in place, which allows simple use, with naturalness of movements, expandable for utilization in large areas (such as public spaces), and that efficiently supply orientation and speed to use in virtual environments or augmented reality, with inexpensive hardware.O controle da locomoção em ambientes virtuais 3D deveria ser uma tarefa simples, do ponto de vista do usuário. Durante os anos, metáforas para navegação têm sido exploradas para permitir o controle da locomoção naturalmente, tais como: caminhada real; uso de simuladores e imitação de caminhada. Estas técnicas provaram que, quanto mais natural à abordagem utilizada para controlar a locomoção, mais imerso o usuário vai se sentir dentro do ambiente virtual. Superando o alto custo e complexidade de uso da maioria das abordagens na área, introduzimos uma plataforma para caminhada no lugar, (usualmente reportado como wal king in place), que é capaz de identificar orientação, velocidade de deslocamento, bem como passos laterais, de uma pessoa imitando a caminhada. A detecção desta informação é feita sem o uso de sensores presos no corpo dos usuários, apenas utilizando a plataforma. Nosso dispositivo é simples de montar, barato e permite seu uso por pessoas comuns de forma quase natural, com passos pequenos, assim deixando as mãos livres para outras tarefas. Nós também exploramos e testamos uma superfície táctil passiva para utilização segura de nossa plataforma. A plataforma foi concebida para ser utilizada como uma interface para navegação em ambientes virtuais. Estendendo o uso de nossa técnica e dis positivo, nós elaboramos uma metáfora para caminhada redirecionada, para ser utilizada em conjunto com cavernas de projeção, (usualmente reportado como Cave automatic vir tual environment (CAVE)). Criamos também uma segunda metáfora para navegação, a qual permitiu o uso de nossa técnica para navegação em nuvem de pontos, auxiliando no processo de etiquetagem destes, como parte da competição para o 3D User Interface que ocorreu em Minessota, nos Estados Unidos, em 2014. Nós testamos o uso da técnica e dispositivos associada com duas nuances de navegação: caminhada humana e controle de veiculo. Na abordagem caminhada humana, a taxa de mudança da orientação gerada pelo usuário ao utilizar nosso dispositivo, inibia o deslocamento quando curvas agudas eram efetuadas. No modo veículo, a orientação e o deslocamento ocorriam conjuntamente quando o usuário utilizava nosso dispositivo e técnicas, similarmente ao processo de controle de direção de um veículo. Nós aplicamos testes para determinar o modo de navegação de preferencia para uti lização de nosso dispositivo, em 52 sujeitos. Identificamos uma preferencia pelo modo de uso que se assimila a condução de um veículo. Testes estatísticos revelaram que os usuários aprenderam facilmente a usar nossa técnica para navegar em ambientes virtuais. Os usuários foram mais rápidos utilizando o modo veículo, mas o modo humano garantiu maior precisão no deslocamento no ambiente virtual. A plataforma táctil provou permi tir o uso seguro de nosso dispositivo, sendo uma solução efetiva e simples para a área. Mais de 200 pessoas testaram nosso dispositivo e técnicas: no evento Portas Abertas da UFRGS em 2013 e 2014, um evento onde são apresentados para a comunidade local os trabalhos executados na universidade; e no 3D User Interface, onde nossa técnica e dis positivos foram utilizados em conjunto com uma ferramenta de seleção de pontos numa competição. As principais contribuições do nosso trabalho são: uma nova abordagem para de tecção de imitação de caminhada, a qual permite um uso simples, com naturalidade de movimentos, expansível para utilização em áreas grandes, como espaços públicos e que efetivamente captura informações de uso e fornece orientação e velocidade para uso em ambientes virtuais ou de realidade aumentada, com uso de hardware barato

Immersion and interaction: Creating virtual 3d worlds for stage performances

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis formulates an approach towards the creation of a gesture activated and body movement controlled real time virtual 3d world in a dance performance context. It investigates immersion and navigation techniques derived from modern video games and methodologies and proposes how they can be used to further involve a performer into a virtual space as well as simultaneously offer a stimulating visual spectacle for an audience. The argument presented develops through practice-based methodology and artistic production strategies in interdisciplinary and collaborative contexts. Two choreographic performance/installations are used as cases studies to demonstrate in practice the proposed methodologies. First, the interactive dance work Suna No Onna, created in collaboration with Birringer/Danjoux and the Dap Lab, investigates the use of interactive pre-rendered animations in a real time setting and in real time by incorporating wearable sensors in the performance. Secondly, the potentials offered by the sensor technology and real time rendering engines led to the “creation scene", a key scene in the choreographic installation UKIYO (Moveable Worlds). This thesis investigates the design, creation and interaction qualities of virtual 3d spaces by exploring the potentialities offered by a shared space, between an intelligent space and a dancer in a hybrid world. The methodology applied uses as a theoretical base the phenomenological approach of Merleau-Ponty and Mark Hansen‟s mixed reality paradigm proposing the concept of the “space schema", a system which replicates and embeds proprioception, perception and motility into the space fabric offering a world which “lives”, functions and interacts with the performer. The outcome of the research is the generation of an interactive, non-linear, randomized 3d virtual space that collaborates with a technologically embedded performer in creating a 3d world which evolves and transforms, driven by the performer‟s intention and agency. This research contributes to the field of interactive performance art by making transparent the methodology, the instruments and the code used, in a non-technical terminology, making it accessible for both team members with less technological expertise as well as artists aspiring to engage interactive 3d media promoting further experimentation and conceptual discussions

Immersion and interaction : creating virtual 3D worlds for stage performances

This thesis formulates an approach towards the creation of a gesture activated and body movement controlled real time virtual 3d world in a dance performance context. It investigates immersion and navigation techniques derived from modern video games and methodologies and proposes how they can be used to further involve a performer into a virtual space as well as simultaneously offer a stimulating visual spectacle for an audience. The argument presented develops through practice-based methodology and artistic production strategies in interdisciplinary and collaborative contexts. Two choreographic performance/installations are used as cases studies to demonstrate in practice the proposed methodologies. First, the interactive dance work Suna No Onna, created in collaboration with Birringer/Danjoux and the Dap Lab, investigates the use of interactive pre-rendered animations in a real time setting and in real time by incorporating wearable sensors in the performance. Secondly, the potentials offered by the sensor technology and real time rendering engines led to the “creation scene", a key scene in the choreographic installation UKIYO (Moveable Worlds). This thesis investigates the design, creation and interaction qualities of virtual 3d spaces by exploring the potentialities offered by a shared space, between an intelligent space and a dancer in a hybrid world. The methodology applied uses as a theoretical base the phenomenological approach of Merleau-Ponty and Mark Hansen‟s mixed reality paradigm proposing the concept of the “space schema", a system which replicates and embeds proprioception, perception and motility into the space fabric offering a world which “lives”, functions and interacts with the performer. The outcome of the research is the generation of an interactive, non-linear, randomized 3d virtual space that collaborates with a technologically embedded performer in creating a 3d world which evolves and transforms, driven by the performer‟s intention and agency. This research contributes to the field of interactive performance art by making transparent the methodology, the instruments and the code used, in a non-technical terminology, making it accessible for both team members with less technological expertise as well as artists aspiring to engage interactive 3d media promoting further experimentation and conceptual discussions.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Control design and experimental evaluation of the 2D CyberWalk platform

The Cyber Walk is a large size 2D omni-directional platform that allows unconstrained locomotion possibilities to I walking user for VR exploration. In this paper we present the motion control design for the platform, which has been developed within the homonymous European research project. The objective is to compensate the intentional motion of the user, so as to keep her/him always close to the platform center while limiting the perceptual effects due to actuation commands. the controller acts at the acceleration level, using suitable observers to estimate the unmeasurable intentional walker's velocity and acceleration. A moving reference position is used to limit the accelerations felt by the user in critical transients, e.g., when the walker suddenly stops motion. Experimental results are reported that show the benefit of designing separate control gains in the two orthogonal directions (lateral and sagittal) of frame attached to the walker

Interacción Natural Basada en un Conjunto Mínimo de Sensores Inerciales para Realidad Virtual sin Cables

La Realidad Virtual tiene un enorme potencial aún por explotar. Esta tesis doctoral pretende ir un paso más allá en el desarrollo de sistemas de Realidad Virtual inmersivos. En concreto, su objetivo fundamental es diseñar, desarrollar y evaluar una plataforma experimental sin cables para investigación en Realidad Virtual inmersiva con navegación natural e interacción manual basada en un conjunto mínimo de sensores inerciales. Para ello se emplea metodología científica desde la perspectiva de la interacción persona computador (Human Computer Interaction, HCI). A partir del objetivo fundamental, se elaboran las recomendaciones de diseño y especificaciones del sistema a desarrollar. Tras revisar en detalle el estado del arte y establecer el planteamiento metodológico, comienza el desarrollo de herramientas en las que se basará la creación de prototipos. Durante la tesis doctoral se desarrollan 3 herramientas de investigación y 5 prototipos que se evalúan a través de diversas pruebas con usuarios y 2 experimentos. En total, participan generosamente más de 85 personas. El desarrollo de prototipos da lugar a técnicas específicas que resultan de interés por sí mismas para la comunidad científica. Por otra parte, los experimentos también aportan resultados susceptibles de ser divulgados. Uno de los experimentos realizados permite evaluar las técnicas desarrolladas para implementar un sistema de Realidad Virtual con navegación natural. El otro experimento, estudia el comportamiento del sistema de tracking para interacción manual desarrollado durante el proyecto de investigación. Además, utiliza una televisión 3D y el casco de Realidad Virtual Oculus Rift para realizar un estudio comparativo de diversos aspectos como el rendimiento, usabilidad, nivel de presencia, dificultad y preferencia. El proyecto de investigación asociado a esta tesis doctoral da lugar a varias aportaciones de distinta naturaleza como publicaciones científicas, herramientas de investigación, algoritmos y trazas de datos, además de la propia plataforma experimental que permitirá abordar nuevos estudios de Realidad Virtual inmersiva con navegación natural e interacción manual