Evaluating 3D pointing techniques

"This dissertation investigates various issues related to the empirical evaluation of 3D pointing interfaces. In this context, the term ""3D pointing"" is appropriated from analogous 2D pointing literature to refer to 3D point selection tasks, i.e., specifying a target in three-dimensional space. Such pointing interfaces are required for interaction with virtual 3D environments, e.g., in computer games and virtual reality. Researchers have developed and empirically evaluated many such techniques. Yet, several technical issues and human factors complicate evaluation. Moreover, results tend not to be directly comparable between experiments, as these experiments usually use different methodologies and measures. Based on well-established methods for comparing 2D pointing interfaces this dissertation investigates different aspects of 3D pointing. The main objective of this work is to establish methods for the direct and fair comparisons between 2D and 3D pointing interfaces. This dissertation proposes and then validates an experimental paradigm for evaluating 3D interaction techniques that rely on pointing. It also investigates some technical considerations such as latency and device noise. Results show that the mouse outperforms (between 10% and 60%) other 3D input techniques in all tested conditions. Moreover, a monoscopic cursor tends to perform better than a stereo cursor when using stereo display, by as much as 30% for deep targets. Results suggest that common 3D pointing techniques are best modelled by first projecting target parameters (i.e., distance and size) to the screen plane.

Softwarearchitektur für die interaktive Simulation mobiler Arbeitsmaschinen in virtuellen Umgebungen

Die numerische Simulation ist ein unverzichtbares Werkzeug bei der Produktentwicklung geworden. Bereits in den frühen Phasen von Studien und Konzepten können unterschiedliche Lösungsansätze für eine Aufgabenstellung bewertet werden. In diesem Zusammenhang wird von virtuellen Prototypen gesprochen. Bei der Simulation mobiler Arbeitsmaschinen erfordert das die Einbindung des Bedieners. Die technologische Leistungsfähigkeit wird wesentlich durch die Interaktion zwischen Bediener und Maschine geprägt. Die Beschreibung des Bedieners durch mathematische Modelle ist bis zum gegenwärtigen Zeitpunkt nicht mit belastbaren Resultaten erfolgt. Die Entwicklungen im Bereich der Simulationstechnologie und der Computergrafik ermöglichen die Durchführung interaktiver Simulationen in komplexen virtuellen Welten. Damit lässt sich der Bediener direkt in die Simulation einbinden und es können zusätzliche Potenziale, wie z. B. bei der Untersuchung der Mensch-Maschine-Interaktion erschlossen werden. Durch die interaktive Simulation in Virtual-Reality-Systemen werden neue Anforderungen an die Simulationssoftware gestellt. Zur Interaktion mit dem Bediener müssen die Eingaben aus den Bedienelementen in Echtzeit verarbeitet und audiovisuelle Ausgaben generiert werden. Dabei sind neben den mathematisch-physikalischen Aspekten der Simulation auch Problemfelder wie Synchronisation, Kommunikation, Bussysteme und Computergrafik zu behandeln. Die Anpassung des Simulationssystems an unterschiedliche Aufgabenstellungen erfordert ein flexibel konfigurierbares Softwaresystem. Als Lösung dieser Aufgabenstellung wird eine Softwarearchitektur vorgestellt, welche die unterschiedlichen Problemfelder durch klar voneinander abgegrenzte Komponenten mit entsprechenden Schnittstellen behandelt. Das entstandene Softwaresystem ist flexibel und erweiterbar. Die Simulationsaufgabe wird durch die Konfiguration des Komponentensystems spezifiziert. Die entstehenden Konfigurationsdateien bilden die Anwendungslogik ab und stellen daher einen der wesentlichen Kostenfaktoren bei der Realisierung interaktiver Simulationen dar. Zur Erhöhung der Wiederverwendbarkeit bestehender Konfigurationsfragmente wird ein kompositionsbasierter Ansatz auf der Basis von Skriptsprachen gewählt.Numerical simulation has evolved into an indispensable tool in modern product development. Even in the early design phases of studies and concepts several different approaches for one individual task can be evaluated. In this context the term virtual prototypes can be used. For effective simulation of mobile construction site machinery incorporation of the operator\'s influence is required. The technological performance of the machinery is essentially influenced by the interaction between the operator and the machine. Currently, there are no known mathematical models describing the operator\'s behaviour, which give substantiated results. The latest developments in computing technology and computer graphics facilitate interactive simulations in complex virtual worlds. This allows not only the operator to be linked to the simulation but also the investigation of additional research areas such as human-machine-interaction. The application of interactive simulation in virtual reality systems places new demands on the simulation software. Due to the interaction not only input signals from the instruments have to be processed but also audio and visual output has to be generated in real time. In addition to the mathematical and physical aspects of simulation, problems in the areas of synchronisation, communication, bus systems and computer graphics also have to be solved. The adaption of the simulation system to new tasks requires a flexible and highly configurable software system. As a response to these demands, a software architecture is presented which partitions the various problems into finite components with corresponding interfaces. The partitioning results in a flexible and extendable software system. The simulation task is specified by the configuration of the component system. The resulting configuration files reflect the application logic and therefore represent one of the main cost factors in the realisation of the interactive simulations. A composition-based approach is chosen as it raises the level of reuse of existing configuration fragments. This approach is based on scripting languages

The role of multisensory feedback in the objective and subjective evaluations of fidelity in virtual reality environments.

The use of virtual reality in academic and industrial research has been rapidly expanding in recent years therefore evaluations of the quality and effectiveness of virtual environments are required. The assessment process is usually done through user evaluation that is being measured whilst the user engages with the system. The limitations of this method in terms of its variability and user bias of pre and post-experience have been recognised in the research literature. Therefore, there is a need to design more objective measures of system effectiveness that could complement subjective measures and provide a conceptual framework for the fidelity assessment in VR. There are many technological and perceptual factors that can influence the overall experience in virtual environments. The focus of this thesis was to investigate how multisensory feedback, provided during VR exposure, can modulate a user’s qualitative and quantitative experience in the virtual environment. In a series of experimental studies, the role of visual, audio, haptic and motion cues on objective and subjective evaluations of fidelity in VR was investigated. In all studies, objective measures of performance were collected and compared to the subjective measures of user perception. The results showed that the explicit evaluation of environmental and perceptual factors available within VR environments modulated user experience. In particular, the results shown that a user’s postural responses can be used as a basis for the objective measure of fidelity. Additionally, the role of augmented sensory cues was investigated during a manual assembly task. By recording and analysing the objective and subjective measures it was shown that augmented multisensory feedback modulated the user’s acceptability of the virtual environment in a positive manner and increased overall task performance. Furthermore, the presence of augmented cues mitigated the negative effects of inaccurate motion tracking and simulation sickness. In the follow up study, the beneficial effects of virtual training with augmented sensory cues were observed in the transfer of learning when the same task was performed in a real environment. Similarly, when the effects of 6 degrees of freedom motion cuing on user experience were investigated in a high fidelity flight simulator, the consistent findings between objective and subjective data were recorded. By measuring the pilot’s accuracy to follow the desired path during a slalom manoeuvre while perceived task demand was increased, it was shown that motion cuing is related to effective task performance and modulates the levels of workload, sickness and presence. The overall findings revealed that multisensory feedback plays an important role in the overall perception and fidelity evaluations of VR systems and as such user experience needs to be included when investigating the effectiveness of sensory feedback signals. Throughout this thesis it was consistently shown that subjective measures of user perception in VR are directly comparable to the objective measures of performance and therefore both should be used in order to obtain a robust results when investigating the effectiveness of VR systems. This conceptual framework can provide an effective method to study human perception, which can in turn provide a deeper understanding of the environmental and cognitive factors that can influence the overall user experience, in terms of fidelity requirements, in virtual reality environments

Exploring the Influence of Haptic Force Feedback on 3D Selection

This thesis studies the effects of haptic force feedback on 3D interaction performance. To date, Human-Computer Interaction (HCI) in three dimensions is not well understood. Within platforms, such as Immersive Virtual Environments (IVEs), implementing `good' methods of interaction is difficult. As reflected by the lack of 3D IVE applications in common use, typical performance constraints include inaccurate tracking, lack of additional sensory inputs, in addition to general design issues related to the implemented interaction technique and connected input devices. In total, this represents a broad set of multi-disciplinary challenges. By implementing techniques that address these problems, we intend to use IVE platforms to study human 3D interaction and the effects of different types of feedback. A promising area of work is the development of haptic force feedback devices. Also called haptic interfaces, these devices can exert a desired force onto the user simulating a physical interaction. When described as a sensory cue, it is thought that this information is important for the selection and manipulation of 3D objects. To date, there are a lot of studies investigating how best to integrate haptic devices within IVEs. Whilst there are still fundamental integration and device level problems to solve, previous work demonstrates that haptic force feedback can improve 3D interaction performance. By investigating this claim further, this thesis explores the role of haptic force feedback on 3D interaction performance in more detail. In particular, we found additional complexities whereby different types of haptic force feedback conditions can either help but also hinder user performance. By discussing these new results, we begin to examine the utility of haptic force feedback. By focusing our user studies on 3D selection, we explored the influence of haptic force feedback on the strategies taken to target virtual objects when using either `distal' and `natural' interaction technique designs. We first outlined novel methods for integrating and calibrating large scale haptic devices within a CAVE-like IVE. Secondly, we described our implementation of distal and natural selection techniques tailored to the available hardware, including the collision detection mechanisms used to render different haptic responses. Thirdly, we discussed the evaluation framework used to assess different interaction techniques and haptic force feedback responses within a common IVE setup. Finally, we provide a detailed assessment of user performance highlighting the effects of haptic force feedback on 3D selection, which is the main contribution of this work. We expect the presented findings will add to the existing literature that evaluates novel 3D interaction technique designs for IVEs. We also hope that this thesis will provide a basis to develop future interaction models that include the effects of haptic force feedback