A Tele-Operated Display With a Predictive Display Algorithm

Tele-operated display systems with head mounted displays (HMD) are becoming popular as visual feedback systems for tele-operation systems. However, the users are suffered from time-varying bidirectional delays caused by the latency and limited bandwidth of wireless communication networks. Here, we develop a tele-operated display system and a predictive display algorithm allowing comfortable use of HMDs by operators of tele-operation systems. Inspired by the kinematic model of the human head-neck complex, we built a robot neck-camera system to capture the field of view in any desired orientation. To reduce the negative effects of the time-varying bidirectional communication delay and operation delay of the robot neck, we developed a predictive display algorithm based on a kinematic model of the human/robot neck-camera system, and a geometrical model of a camera. Experimental results showed that the system provide predicted images with high frame rate to the user

A Wave Variable Approach with Multiple Channel Architecture for Teleoperated System

© 2013 IEEE. Performance of teleoperation can be greatly influenced by time delay in the process of tele-manipulation with respect to accuracy and transparency. Wave variable is an effective algorithm to achieve a good stable capability. However, some traditional wave variable methods may decrease the performance of transparency and suffer the impacts of wave reflection. To deal with the problem of stability and transparency in teleoperation, in this paper, a novel wave variable method with four channel is presented to achieve stable tracking in position and force. In addition, the proposed method can achieve the distortion compensation and reduce the impacts of wave reflection. The simulation experimental results verified the tracking performance of the proposed method

Expert-in-the-Loop Multilateral Telerobotics for Haptics-Enabled Motor Function and Skills Development

Among medical robotics applications are Robotics-Assisted Mirror Rehabilitation Therapy (RAMRT) and Minimally-Invasive Surgical Training (RAMIST) that extensively rely on motor function development. Haptics-enabled expert-in-the-loop motor function development for such applications is made possible through multilateral telerobotic frameworks. While several studies have validated the benefits of haptic interaction with an expert in motor learning, contradictory results have also been reported. This emphasizes the need for further in-depth studies on the nature of human motor learning through haptic guidance and interaction. The objective of this study was to design and evaluate expert-in-the-loop multilateral telerobotic frameworks with stable and human-safe control loops that enable adaptive “hand-over-hand” haptic guidance for RAMRT and RAMIST. The first prerequisite for such frameworks is active involvement of the patient or trainee, which requires the closed-loop system to remain stable in the presence of an adaptable time-varying dominance factor. To this end, a wave-variable controller is proposed in this study for conventional trilateral teleoperation systems such that system stability is guaranteed in the presence of a time-varying dominance factor and communication delay. Similar to other wave-variable approaches, the controller is initially developed for the Velocity-force Domain (VD) based on the well-known passivity assumption on the human arm in VD. The controller can be applied straightforwardly to the Position-force Domain (PD), eliminating position-error accumulation and position drift, provided that passivity of the human arm in PD is addressed. However, the latter has been ignored in the literature. Therefore, in this study, passivity of the human arm in PD is investigated using mathematical analysis, experimentation as well as user studies involving 12 participants and 48 trials. The results, in conjunction with the proposed wave-variables, can be used to guarantee closed-loop PD stability of the supervised trilateral teleoperation system in its classical format. The classic dual-user teleoperation architecture does not, however, fully satisfy the requirements for properly imparting motor function (skills) in RAMRT (RAMIST). Consequently, the next part of this study focuses on designing novel supervised trilateral frameworks for providing motor learning in RAMRT and RAMIST, each customized according to the requirements of the application. The framework proposed for RAMRT includes the following features: a) therapist-in-the-loop mirror therapy; b) haptic feedback to the therapist from the patient side; c) assist-as-needed therapy realized through an adaptive Guidance Virtual Fixture (GVF); and d) real-time task-independent and patient-specific motor-function assessment. Closed-loop stability of the proposed framework is investigated using a combination of the Circle Criterion and the Small-Gain Theorem. The stability analysis addresses the instabilities caused by: a) communication delays between the therapist and the patient, facilitating haptics-enabled tele- or in-home rehabilitation; and b) the integration of the time-varying nonlinear GVF element into the delayed system. The platform is experimentally evaluated on a trilateral rehabilitation setup consisting of two Quanser rehabilitation robots and one Quanser HD2 robot. The framework proposed for RAMIST includes the following features: a) haptics-enabled expert-in-the-loop surgical training; b) adaptive expertise-oriented training, realized through a Fuzzy Interface System, which actively engages the trainees while providing them with appropriate skills-oriented levels of training; and c) task-independent skills assessment. Closed-loop stability of the architecture is analyzed using the Circle Criterion in the presence and absence of haptic feedback of tool-tissue interactions. In addition to the time-varying elements of the system, the stability analysis approach also addresses communication delays, facilitating tele-surgical training. The platform is implemented on a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA), integrated with the da Vinci Research Kit (dVRK) motor controllers, and the dV-Trainer master console (Mimic Technology Inc., Seattle, WA). In order to save on the expert\u27s (therapist\u27s) time, dual-console architectures can also be expanded to accommodate simultaneous training (rehabilitation) for multiple trainees (patients). As the first step in doing this, the last part of this thesis focuses on the development of a multi-master/single-slave telerobotic framework, along with controller design and closed-loop stability analysis in the presence of communication delays. Various parts of this study are supported with a number of experimental implementations and evaluations. The outcomes of this research include multilateral telerobotic testbeds for further studies on the nature of human motor learning and retention through haptic guidance and interaction. They also enable investigation of the impact of communication time delays on supervised haptics-enabled motor function improvement through tele-rehabilitation and mentoring

Wave Prediction and Delay Modeling for Teleoperation via Internet

This paper propose a novel approach for modeling the end-to-end time delay dynamics of the internet using system identification, and use it for controlling real-time internet-based telerobotic operations. When a single model is used, it needs to adapt to the operating conditions before an appropriate control mechanism can be applied. Slow adaptation may result in large transient errors. As an alternative, we propose to use an adaptive multiple model framework, and determine the best model for the current operating conditions to activate the corresponding controller. We employ multivariable wave prediction method to achieve this objective

Experimental Evaluation of Novel Master-Slave Configurations for Position Control under Random Network Delay and Variable Load for Teleoperation

This paper proposes two novel master-slave configurations that provide improvements in both control and communication aspects of teleoperation systems to achieve an overall improved performance in position control. The proposed novel master-slave configurations integrate modular control and communication approaches, consisting of a delay regulator to address problems related to variable network delay common to such systems, and a model tracking control that runs on the slave side for the compensation of uncertainties and model mismatch on the slave side. One of the configurations uses a sliding mode observer and the other one uses a modified Smith predictor scheme on the master side to ensure position transparency between the master and slave, while reference tracking of the slave is ensured by a proportional-differentiator type controller in both configurations. Experiments conducted for the networked position control of a single-link arm under system uncertainties and randomly varying network delays demonstrate significant performance improvements with both configurations over the past literature