Kinesthetic Haptics Sensing and Discovery with Bilateral Teleoperation Systems

In the mechanical engineering field of robotics, bilateral teleoperation is a classic but still increasing research topic. In bilateral teleoperation, a human operator moves the master manipulator, and a slave manipulator is controlled to follow the motion of the master in a remote, potentially hostile environment. This dissertation focuses on kinesthetic perception analysis in teleoperation systems. Design of the controllers of the systems is studied as the influential factor of this issue. The controllers that can provide different force tracking capability are compared using the same experimental protocol. A 6 DOF teleoperation system is configured as the system testbed. An innovative master manipulator is developed and a 7 DOF redundant manipulator is used as the slave robot. A singularity avoidance inverse kinematics algorithm is developed to resolve the redundancy of the slave manipulator. An experimental protocol is addressed and three dynamics attributes related to kineshtetic feedback are investigated: weight, center of gravity and inertia. The results support our hypothesis: the controller that can bring a better force feedback can improve the performance in the experiments

Passive Control Architectures for Collaborative Virtual Haptic Interaction and Bilateral Teleoperation over Unreliable Packet-Switched Digital Network

This PhD dissertation consists of two major parts: collaborative haptic interaction (CHI) and bilateral teleoperation over the Internet. For the CHI, we propose a novel hybrid peer-to-peer (P2P) architecture including the shared virtual environment (SVE) simulation, coupling between the haptic device and VE, and P2P synchronization control among all VE copies. This framework guarantees the interaction stability for all users with general unreliable packet-switched communication network which is the most challenging problem for CHI control framework design. This is achieved by enforcing our novel \emph{passivity condition} which fully considers time-varying non-uniform communication delays, random packet loss/swapping/duplication for each communication channel. The topology optimization method based on graph algebraic connectivity is also developed to achieve optimal performance under the communication bandwidth limitation. For validation, we implement a four-user collaborative haptic system with simulated unreliable packet-switched network connections. Both the hybrid P2P architecture design and the performance improvement due to the topology optimization are verified. In the second part, two novel hybrid passive bilateral teleoperation control architectures are proposed to address the challenging stability and performance issues caused by the general Internet communication unreliability (e.g. varying time delay, packet loss, data duplication, etc.). The first method--Direct PD Coupling (DPDC)--is an extension of traditional PD control to the hybrid teleoperation system. With the assumption that the Internet communication unreliability is upper bounded, the passive gain setting condition is derived and guarantees the interaction stability for the teleoperation system which interacts with unknown/unmodeled passive human and environment. However, the performance of DPDC degrades drastically when communication unreliability is severe because its feasible gain region is limited by the device viscous damping. The second method--Virtual Proxy Based PD Coupling (VPDC)--is proposed to improve the performance while providing the same interaction stability. Experimental and quantitative comparisons between DPDC and VPDC are conducted, and both interaction stability and performance difference are validated

Human Performance and Cognitive Workload in Haptic, Audio and Visual Environments

The ability to efficiently perform a task in a human-in-the-loop system and in multi-sensory virtual environments is highly dependent on the type of sensory feedback the operator is receiving and the amount of workload the operator is exposed to. Despite the vast amount of research on Collaborative Virtual Environments (CVEs) and Human Machine Interactions (HMIs), little is known about what type of feedback increases the performance of a human operator and what type of sensory feedback minimizes the amount of workload the operator is exposed to. While individual differences influence human performance outcomes, the physiological processes a human being set the fundamental guidelines for assessing human performance. The purpose of this study is to evaluate the performance of participants for a combination of sensory two feedback modes (audio-visual, haptic-visual or audio-haptic) in a primary task to find the optimum feedback model for CVE and HMI applications. A concurrent secondary task is also designed to evaluate workload of each feedback mode (audio, haptic or visual) and the effect of different levels of workload on task completion time and task accuracy. For example, a car driver performs a primary task by steering the car in the correct direction. A secondary task, in the same context, would be monitoring the fuel level or checking the speed limit. In the primary task, participants are required to press a virtual button from a set of three (right button, left button or up button). The secondary task evaluates the amount of workload the participant is exposed to in three different feedback modes (haptic, audio or visual). Each participant is required to recognize a Morse code. In this study, participants perform three trials. In first trial, participants perform one task the primary task alone. In the second and third trials, participants perform the primary task and the secondary task concurrently. The primary task evaluates human performance and includes combined sensory modalities as a feedback mode (audio-visual, haptic-visual or audio-haptic). The time it takes the participant to press the virtual button (primary task response time), the number of correct button presses (primary task accuracy), the time it takes the participant to recognize the Morse code (secondary task response time) and the number of the correct codes (secondary task accuracy) are all collected. In addition, NASA Task Load Index (TLX) questionnaire is used after each trial to assess the subjective performance and subjective workload of participants. The data collected is tested for normality using Lilliefors test, filtered using Grubb’s test to eliminate outlying data and analyzed using one-way ANOVA and multiple two-sample t-tests. A Tukey HSD is also used to show the differences between experimental conditions. The results of this study indicate that the hypothesis that all combinations of feedback provide the same performance can be rejected for the primary task response time. For instance, the results show that the there is a difference in response time between the audio-haptic and the audio-visual feedback modes in the first, second and third trials. The results of this study also indicate that the hypothesis that all sensory feedback modes provide the same workload can be rejected for the secondary task accuracy. Results show that there is a difference between haptic and auditory conditions and shows that visual condition has a lower accuracy than the other feedback modes

Georganas, “Architecture and Evaluation of Tele-Haptic Environments

A collaborative, haptic, audio and visual environment (C-HAVE) consists of a network of nodes. Each node in the C-HAVE world contributes to the shared environment with some virtual objects. These can be static, e.g., a sculpture or the ground, or dynamic, e.g., an object that can be virtually manipulated. We aim at developing a heterogeneous scalable architecture for large collaborative haptics environments where a number of potential users participate with different kinds of haptic devices. The main objective of the presented research is the development of three prototypes to demonstrate quantitatively the effects of adding haptics to a task. The experimental results reveal the effects of the different implementations on the performance and time delay of a particular task through objective measurement results. 1