The benefits of using a walking interface to navigate virtual environments

Navigation is the most common interactive task performed in three-dimensional virtual environments (VEs), but it is also a task that users often find difficult. We investigated how body-based information about the translational and rotational components of movement helped participants to perform a navigational search task (finding targets hidden inside boxes in a room-sized space). When participants physically walked around the VE while viewing it on a head-mounted display (HMD), they then performed 90% of trials perfectly, comparable to participants who had performed an equivalent task in the real world during a previous study. By contrast, participants performed less than 50% of trials perfectly if they used a tethered HMD (move by physically turning but pressing a button to translate) or a desktop display (no body-based information). This is the most complex navigational task in which a real-world level of performance has been achieved in a VE. Behavioral data indicates that both translational and rotational body-based information are required to accurately update one's position during navigation, and participants who walked tended to avoid obstacles, even though collision detection was not implemented and feedback not provided. A walking interface would bring immediate benefits to a number of VE applications

Doctor of Philosophy

dissertationThough many who walk along roadside cambers and hill edges may have an interest in making their travels sure and effective, those most concerned may be soldiers in the infantry. They need to be injury free and have as much energy as possible when they march into battle. Walking on uneven ground without being injured by falling down (particularly with a heavy backpack) is generally accomplished by maintaining stability. This present study was conducted to determine an individual’s most stable position (using a stability formula which compares dynamic center of mass with center of pressure) when wearing a backpack under differing load positions - low back, middle back or high back â€" and differing walking angles: level, as well as along a cross-sloped surface. Furthermore, this study investigated the stability of persons walking along a cross-slope without a load. Finally, this study attempted to determine which combination of backpack load location and slope tilt best conserved metabolic energy. To carry out this backpack stability research, a group of 15 participants were asked to walk along an indoor track under the varying conditions mentioned (i.e., low to high backpack load positions and level to 10 degree tilted cross-slopes). The trials of their walks were performed randomly. The participants were recorded in a motion capture system and force plates documented their stepping times and locations. Again, the same 15 participants walked along the track under the same conditions, but without the loads to determine the effect of different cross-slope angles on their stability. Lastly, the same participants walked the track under the various conditions wearing portable oxygen sensors to analyze their energy expenditure. The results of these limited tests indicate no significant stability differences between 0, 5 or 10 degree angles in cross-slope walking loaded or unloaded. Nor was any significant stability differences noted between the various load locations of the backpacks. Nor was there a significant energy difference between the conditions

Studio e progettazione preliminare di un sistema di retroazione di forza innovativo per interfacce di locomozione in ambienti virtuali.

Titolo della tesi: Studio e progettazione preliminare di un sistema di retroazione di forza innovativo per interfacce di locomozione in ambienti virtuali. Riassunto: oggetto del presente lavoro è l’analisi e progettazione di un sistema robotico a tre gradi di libertà attuati, capace di riprodurre una forza di direzione generica in prossimità del baricentro di un utente in libero movimento su di una interfaccia di locomozione per ambienti virtuali. Scopo del sistema è una più fedele simulazione delle effettive condizioni di moto (approssimazione di effetti inerziali, variazione risultante delle forze percepite a differenti pendenze del terreno, presenza di ostacoli, ecc.). L'individuazione della configurazione cinematica adottata è stata effettuata tramite confronto di soluzioni, sia seriali che parallele, capaci di soddisfare i requisiti imposti ed ottimizzate geometricamente e staticamente sullo spazio di lavoro considerato, tramite programmazione di opportune funzioni di costo. La soluzione individuata, a cinematica parallela a tre DOF con aggiunta di un polso sferico passivo, è stata ulteriormente ottimizzata per minimizzare sia le coppie di attuazione richieste che le reazioni vincolari generate sulla struttura. La successiva fase di modellazione tramite CAD tridimensionale parametrico associativo ha consentito di effettuare le necessarie verifiche sia geometriche che costruttive, alcune effettuate tramite modelli agli elementi finiti. Il sistema cosi' ottenuto è unico nel suo genere sia per ambito applicativo che per potenzialità concesse nell'applicazione delle forze di simulazione. Thesis title: Analysis and preliminary design of an innovative force feedback system to locomotion interfaces for virtual environments. Abstract: scope of the present work is the analysis and preliminary design of a 3DOF robotic system, capable of producing generic forces approximately at the center of gravity of a user that is freely moving on a locomotion interface. The system main goal is a more compelling simulation of actual real motion conditions and effects (inertial effects, variation of resultant gravity force direction relative to ground slope, obstacles, etc. ). The adopted kinematic configuration has been defined by comparison of both serial and parallel solutions able to achieve the imposed requirements. Before comparison all solutions have been optimised geometrically and statically on the desired workspace by implementation of proper cost functions. The adopted solution, that is characterised by a 3DOF translating parallel mechanism with the addition of a passive serial wrist, has been further optimised to minimise actuators requirements and constraints reactions. Then a detailed system model has been realized by an associative 3D parametric CAD software, allowing for the necessary geometrical and constructive verifies, some of which have been realized by finite element models. The final system is innovative both for field of application and the improved potential in producing simulation forces

Spatial representation of described environments: the characteristics of verbal descriptions and the role of physical movement

Recent findings support the assumption that the verbal description of an environment allows the creation of a mental representation of the environment that is functionally equivalent to that deriving from direct perception. Moreover, the verbal descriptions of an environment are commonly used in daily life to communicate with both sighted people in remote environment or visually impaired people. However, many questions on this topic still need to be answered, especially regarding the characteristics of the verbal descriptions and the potential fostering effect of physical movements. Thus, the aim of the present work was to shed light on the role of the physical movements in supporting the verbal descriptions, after a brief examination of some features important for the verbal descriptions. The first part of the manuscript deals with the specific characteristics of the verbal description which could affect the corresponding mental representation. Thus, the influence of the serial position effect on different types of verbal stimuli has been investigated in order to clarify whether people are able to remember all the information provided with a verbal description of an environment or systematically lose information positioned in the central part of the description. Then, the effect of the direction of the verbal description (Clockwise or Counterclockwise) on the recall performance/modality has been considered; I found that people prefers to recall spatial relations that are congruent with the description encoded, extending a well-established effect of memory congruency also to the domain of spatial representation. The second part of the thesis, instead, deals with the influence of physical movement on spatial updating within described environments. Even though several researchers focused on the role of movement in immediate and remote environments, only few of them investigated its role on verbally described environments; moreover, they mainly targeted their studies on imagined movements, rotation and translation, neglecting more ecological movements, such as physical walking. Thus, two separate experiments shed light on the contribution of physical walking compared to both physical and imagined rotation, on spatial updating \u2013 that is, the ability to keep track of the self-to-object relations during observer\u2019s movement. By manipulating the movements required during the encoding of the description or after the description, results suggested different effects of the multisensory pattern of information provided by physical walking on spatial updating, depending on the situation in which the movements were executed. Finally, a third experiment, which used an ecological experimental procedure, examined the effectiveness of the physical exploration of an environment in fostering the development of an adequate spatial representation. In conclusion, the results provide further evidence for the effectiveness of adopting verbal material to describe an environment, and highlight the important role of the physical movements in enhancing people\u2019s ability to successfully interact with the described environments

3D Multimodal Interaction with Physically-based Virtual Environments

The virtual has become a huge field of exploration for researchers: it could assist the surgeon, help the prototyping of industrial objects, simulate natural phenomena, be a fantastic time machine or entertain users through games or movies. Far beyond the only visual rendering of the virtual environment, the Virtual Reality aims at -literally- immersing the user in the virtual world. VR technologies simulate digital environments with which users can interact and, as a result, perceive through different modalities the effects of their actions in real time. The challenges are huge: the user's motions need to be perceived and to have an immediate impact on the virtual world by modifying the objects in real-time. In addition, the targeted immersion of the user is not only visual: auditory or haptic feedback needs to be taken into account, merging all the sensory modalities of the user into a multimodal answer. The global objective of my research activities is to improve 3D interaction with complex virtual environments by proposing novel approaches for physically-based and multimodal interaction. I have laid the foundations of my work on designing the interactions with complex virtual worlds, referring to a higher demand in the characteristics of the virtual environments. My research could be described within three main research axes inherent to the 3D interaction loop: (1) the physically-based modeling of the virtual world to take into account the complexity of the virtual object behavior, their topology modifications as well as their interactions, (2) the multimodal feedback for combining the sensory modalities into a global answer from the virtual world to the user and (3) the design of body-based 3D interaction techniques and devices for establishing the interfaces between the user and the virtual world. All these contributions could be gathered in a general framework within the 3D interaction loop. By improving all the components of this framework, I aim at proposing approaches that could be used in future virtual reality applications but also more generally in other areas such as medical simulation, gesture training, robotics, virtual prototyping for the industry or web contents.Le virtuel est devenu un vaste champ d'exploration pour la recherche et offre de nos jours de nombreuses possibilités : assister le chirurgien, réaliser des prototypes de pièces industrielles, simuler des phénomènes naturels, remonter dans le temps ou proposer des applications ludiques aux utilisateurs au travers de jeux ou de films. Bien plus que le rendu purement visuel d'environnement virtuel, la réalité virtuelle aspire à -littéralement- immerger l'utilisateur dans le monde virtuel. L'utilisateur peut ainsi interagir avec le contenu numérique et percevoir les effets de ses actions au travers de différents retours sensoriels. Permettre une véritable immersion de l'utilisateur dans des environnements virtuels de plus en plus complexes confronte la recherche en réalité virtuelle à des défis importants: les gestes de l'utilisateur doivent être capturés puis directement transmis au monde virtuel afin de le modifier en temps-réel. Les retours sensoriels ne sont pas uniquement visuels mais doivent être combinés avec les retours auditifs ou haptiques dans une réponse globale multimodale. L'objectif principal de mes activités de recherche consiste à améliorer l'interaction 3D avec des environnements virtuels complexes en proposant de nouvelles approches utilisant la simulation physique et exploitant au mieux les différentes modalités sensorielles. Dans mes travaux, je m'intéresse tout particulièrement à concevoir des interactions avec des mondes virtuels complexes. Mon approche peut être décrite au travers de trois axes principaux de recherche: (1) la modélisation dans les mondes virtuels d'environnements physiques plausibles où les objets réagissent de manière naturelle, même lorsque leur topologie est modifiée ou lorsqu'ils sont en interaction avec d'autres objets, (2) la mise en place de retours sensoriels multimodaux vers l'utilisateur intégrant des composantes visuelles, haptiques et/ou sonores, (3) la prise en compte de l'interaction physique de l'utilisateur avec le monde virtuel dans toute sa richesse : mouvements de la tête, des deux mains, des doigts, des jambes, voire de tout le corps, en concevant de nouveaux dispositifs ou de nouvelles techniques d'interactions 3D. Les différentes contributions que j'ai proposées dans chacun de ces trois axes peuvent être regroupées au sein d'un cadre plus général englobant toute la boucle d'interaction 3D avec les environnements virtuels. Elles ouvrent des perspectives pour de futures applications en réalité virtuelle mais également plus généralement dans d'autres domaines tels que la simulation médicale, l'apprentissage de gestes, la robotique, le prototypage virtuel pour l'industrie ou bien les contenus web

3D Multimodal Interaction with Physically-based Virtual Environments

The virtual has become a huge field of exploration for researchers: it could assist the surgeon, help the prototyping of industrial objects, simulate natural phenomena, be a fantastic time machine or entertain users through games or movies. Far beyond the only visual rendering of the virtual environment, the Virtual Reality aims at -literally- immersing the user in the virtual world. VR technologies simulate digital environments with which users can interact and, as a result, perceive through different modalities the effects of their actions in real time. The challenges are huge: the user's motions need to be perceived and to have an immediate impact on the virtual world by modifying the objects in real-time. In addition, the targeted immersion of the user is not only visual: auditory or haptic feedback needs to be taken into account, merging all the sensory modalities of the user into a multimodal answer. The global objective of my research activities is to improve 3D interaction with complex virtual environments by proposing novel approaches for physically-based and multimodal interaction. I have laid the foundations of my work on designing the interactions with complex virtual worlds, referring to a higher demand in the characteristics of the virtual environments. My research could be described within three main research axes inherent to the 3D interaction loop: (1) the physically-based modeling of the virtual world to take into account the complexity of the virtual object behavior, their topology modifications as well as their interactions, (2) the multimodal feedback for combining the sensory modalities into a global answer from the virtual world to the user and (3) the design of body-based 3D interaction techniques and devices for establishing the interfaces between the user and the virtual world. All these contributions could be gathered in a general framework within the 3D interaction loop. By improving all the components of this framework, I aim at proposing approaches that could be used in future virtual reality applications but also more generally in other areas such as medical simulation, gesture training, robotics, virtual prototyping for the industry or web contents.Le virtuel est devenu un vaste champ d'exploration pour la recherche et offre de nos jours de nombreuses possibilités : assister le chirurgien, réaliser des prototypes de pièces industrielles, simuler des phénomènes naturels, remonter dans le temps ou proposer des applications ludiques aux utilisateurs au travers de jeux ou de films. Bien plus que le rendu purement visuel d'environnement virtuel, la réalité virtuelle aspire à -littéralement- immerger l'utilisateur dans le monde virtuel. L'utilisateur peut ainsi interagir avec le contenu numérique et percevoir les effets de ses actions au travers de différents retours sensoriels. Permettre une véritable immersion de l'utilisateur dans des environnements virtuels de plus en plus complexes confronte la recherche en réalité virtuelle à des défis importants: les gestes de l'utilisateur doivent être capturés puis directement transmis au monde virtuel afin de le modifier en temps-réel. Les retours sensoriels ne sont pas uniquement visuels mais doivent être combinés avec les retours auditifs ou haptiques dans une réponse globale multimodale. L'objectif principal de mes activités de recherche consiste à améliorer l'interaction 3D avec des environnements virtuels complexes en proposant de nouvelles approches utilisant la simulation physique et exploitant au mieux les différentes modalités sensorielles. Dans mes travaux, je m'intéresse tout particulièrement à concevoir des interactions avec des mondes virtuels complexes. Mon approche peut être décrite au travers de trois axes principaux de recherche: (1) la modélisation dans les mondes virtuels d'environnements physiques plausibles où les objets réagissent de manière naturelle, même lorsque leur topologie est modifiée ou lorsqu'ils sont en interaction avec d'autres objets, (2) la mise en place de retours sensoriels multimodaux vers l'utilisateur intégrant des composantes visuelles, haptiques et/ou sonores, (3) la prise en compte de l'interaction physique de l'utilisateur avec le monde virtuel dans toute sa richesse : mouvements de la tête, des deux mains, des doigts, des jambes, voire de tout le corps, en concevant de nouveaux dispositifs ou de nouvelles techniques d'interactions 3D. Les différentes contributions que j'ai proposées dans chacun de ces trois axes peuvent être regroupées au sein d'un cadre plus général englobant toute la boucle d'interaction 3D avec les environnements virtuels. Elles ouvrent des perspectives pour de futures applications en réalité virtuelle mais également plus généralement dans d'autres domaines tels que la simulation médicale, l'apprentissage de gestes, la robotique, le prototypage virtuel pour l'industrie ou bien les contenus web

Simulating side slopes on locomotion interfaces using torso forces

This paper describes the biomechanical experimental validation of simulating side slope during walking on a treadmill style locomotion interface. The side slope effect is achieved by means of a lateral force applied to the waist of the walking subject. Results are provided and discussed for both simulated and real side slopes, showing a substantial biomechanical equivalence in the walking pattern for the real side slope and lateral torso force.