Exploring the Influence of Haptic Force Feedback on 3D Selection

This thesis studies the effects of haptic force feedback on 3D interaction performance. To date, Human-Computer Interaction (HCI) in three dimensions is not well understood. Within platforms, such as Immersive Virtual Environments (IVEs), implementing `good' methods of interaction is difficult. As reflected by the lack of 3D IVE applications in common use, typical performance constraints include inaccurate tracking, lack of additional sensory inputs, in addition to general design issues related to the implemented interaction technique and connected input devices. In total, this represents a broad set of multi-disciplinary challenges. By implementing techniques that address these problems, we intend to use IVE platforms to study human 3D interaction and the effects of different types of feedback. A promising area of work is the development of haptic force feedback devices. Also called haptic interfaces, these devices can exert a desired force onto the user simulating a physical interaction. When described as a sensory cue, it is thought that this information is important for the selection and manipulation of 3D objects. To date, there are a lot of studies investigating how best to integrate haptic devices within IVEs. Whilst there are still fundamental integration and device level problems to solve, previous work demonstrates that haptic force feedback can improve 3D interaction performance. By investigating this claim further, this thesis explores the role of haptic force feedback on 3D interaction performance in more detail. In particular, we found additional complexities whereby different types of haptic force feedback conditions can either help but also hinder user performance. By discussing these new results, we begin to examine the utility of haptic force feedback. By focusing our user studies on 3D selection, we explored the influence of haptic force feedback on the strategies taken to target virtual objects when using either `distal' and `natural' interaction technique designs. We first outlined novel methods for integrating and calibrating large scale haptic devices within a CAVE-like IVE. Secondly, we described our implementation of distal and natural selection techniques tailored to the available hardware, including the collision detection mechanisms used to render different haptic responses. Thirdly, we discussed the evaluation framework used to assess different interaction techniques and haptic force feedback responses within a common IVE setup. Finally, we provide a detailed assessment of user performance highlighting the effects of haptic force feedback on 3D selection, which is the main contribution of this work. We expect the presented findings will add to the existing literature that evaluates novel 3D interaction technique designs for IVEs. We also hope that this thesis will provide a basis to develop future interaction models that include the effects of haptic force feedback

Exploring Effects of Information Filtering With a VR Interface for Multi-Robot Supervision

Supervising and controlling remote robot systems currently requires many specialised operators to have knowledge of the internal state of the system in addition to the environment. For applications such as remote maintenance of future nuclear fusion reactors, the number of robots (and hence supervisors) required to maintain or decommission a facility is too large to be financially feasible. To address this issue, this work explores the idea of intelligently filtering information so that a single user can supervise multiple robots safely. We gathered feedback from participants using five methods for teleoperating a semi-autonomous multi-robot system via Virtual Reality (VR). We present a novel 3D interaction method to filter the displayed information to allow the user to read information from the environment without being overwhelmed. The novelty of the interface design is the link between Semantic and Spatial filtering and the hierarchical information contained within the multi robot system. We conducted a user study including a cohort of expert robot teleoperators comparing these methods; highlighting the significant effects of 3D interface design on the performance and perceived workload of a user teleoperating many robot agents in complex environments. The results from this experiment and subjective user feedback will inform future investigations that build upon this initial work

Toward autonomous architecture: The convergence of digital design, robotics, and the built environment

The way we design, construct, and inhabit buildings is changing—moving toward greater integration of robotic and autonomous systems that challenge our preconceived notions of how buildings are made, what they are, or what they should be

UAV Path Planning System Based on 3D Informed RRT* for Dynamic Obstacle Avoidance

A path planning system based on the Informed RRT* path planner was developed to enable an unmanned aerial vehicle (UAV) to avoid moving obstacles in a cluttered 3D environment. For congested environments such as a construction site, path planning systems that help a UAV to safely manoeuvre around dynamic objects and potential co-workers operating within the same workspace is needed. Instead of using a general RRT* path planner approach which will generate a sinuous path, we proposed a flexible approach to increase the convergence of our path planner by re-defining the search space based on 2D Informed RRT* path planner. General RRT* has a relatively low convergence speed to optimize its original solution. By using motion tracking cameras, we obtained real-time feedback of the UAVs pose as well as map structuring and obstacle positions. With this setup, the performance of our proposed path planning approach was assessed using a set of diverse scenarios to compare against general RRT* in convergence rate, quality of solution and ability to handle multiple obstacle situation

Evaluating the Influence of Haptic Force-Feedback on 3D Selection Tasks using Natural Egocentric Gestures

Immersive Virtual Environments (IVEs) allow participants to interact with their 3D surroundings using natural hand gestures. Previous work shows that the addition of haptic feedback cues improves performance on certain 3D tasks. However, we believe this is not true for all situations. Depending on the difficulty of the task, we suggest that we should expect differences in the ballistic movement of our hands when presented with different types of haptic force-feedback conditions. We investigated how hard, soft and no haptic force-feedback responses, experienced when in contact with the surface of an object, affected user performance on a task involving selection of multiple targets. To do this, we implemented a natural egocentric selection interaction technique by integrating a two-handed large-scale force-feedback device in to a CAVE (TM)-like IVE system. With this, we performed a user study where we show that participants perform selection tasks best when interacting with targets that exert soft haptic force-feedback cues. For targets that have hard and no force-feedback properties, we highlight certain associated hand movement that participants make under these conditions, that we hypothesise reduce their performance

Efficient Environment Guided Approach for Exploration of Complex Environments

Remote inspection of a complex environment is a difficult, time consuming task for human operators to perform. The need to manually avoid obstacles whilst considering other performance factors i.e. time taken, joint effort and information gained represents significant challenges to continuous operation. This paper proposes an autonomous robotic solution for exploration of an unknown, complex environment using a high DoF robot arm with an eye in hand depth sensor. The main contribution of this work is a new strategy to find the next best view by evaluating frontier regions of the map to maximise coverage, in contrast to many current approaches which densely sample joint or workspace configurations of the robot. Multiple utility functions were evaluated that showed different behaviours. Our results indicated that the presented algorithm can explore an arbitrary environment efficiently while optimising various performance criteria based on the utility function chosen, application constraints and the desires of the user

Evaluating immersive teleoperation interfaces: coordinating robot radiation monitoring tasks in nuclear facilities

We present a virtual reality (VR) teleoperation interface for a ground-based robot, featuring dense 3D environment reconstruction and a low latency video stream, with which operators can immersively explore remote environments. At the UK Atomic Energy Authority's (UKAEA) Remote Applications in Challenging Environments (RACE) facility, we applied the interface in a user study where trained robotics operators completed simulated nuclear monitoring and decommissioning style tasks to compare VR and traditional teleoperation interface designs. We found that operators in the VR condition took longer to complete the experiment, had reduced collisions, and rated the generated 3D map with higher importance when compared to non-VR operators. Additional physiological data suggested that VR operators had a lower objective cognitive workload during the experiment but also experienced increased physical demand. Overall the presented results show that VR interfaces may benefit work patterns in teleoperation tasks within the nuclear industry, but further work is needed to investigate how such interfaces can be integrated into real world decommissioning workflows

Reinforced Safety-Related Condition Awareness in a Motion Planning System for Remote Manipulators in Nuclear Decommissioning

Motion planning of remote manipulators is a challenging task in nuclear decommissioning practice. Operators have to consider different safety-related factors to plan safe motions in such critical applications. Novel motion planning systems assist operators in planning collision-free motions. However, there is no effective way to reinforce the operators' awareness of multiple safety-related conditions, which are beyond the considerations of standard motion planning requirements in other application fields. This paper presents a human-machine interaction approach that monitors safety-related conditions and reinforces operators' awareness, when potential risks likely emerge. Three functions, which reinforce the awareness of manipulator-environment clearance, radiation exposure, and system limits, are implemented to realize the proposed interaction approach and reduce operators' cognitive load. A pilot study was carried out using a motion planning system enhanced with the proposed approach. The effectiveness has been qualitatively verified through the pilot study

In-vivo high resolution AFM topographic imaging of Caenorhabditis elegans reveals previously unreported surface structures of cuticle mutants

Atomic force microscopy (AFM) is a powerful method for topographic imaging of surfaces with nanometer resolution. AFM offers significant advantages over scanning electron microscopy (SEM) including the acquisition of quantitative 3D-images and biomechanical information. More importantly, for in-vivo biological imaging, AFM does not require sample dehydration/labeling. We show for the first time high-resolution topographical images of the cuticle of the model organism C. elegans under physiological conditions using AFM. C. elegans is used extensively for drug screening and to study pathogen adherence in innate immunity; both applications highly depend on the integrity of the nematode's cuticle. Mutations affecting both drug adsorption and pathogen clearance have been proposed to relate to changes in the cuticle structure, but never visually examined in high resolution. In this study we use AFM to visualize the topography of wild-type adult C. elegans as well as several cuticle collagen mutants and describe previously unseen anatomical differences