The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

Haptic Effects of Surgical Teleoperator Flexibility

Minimally invasive surgery systems typically involve thin and cable-driven surgical instruments. This introduces link and joint flexibility in the slave robot of a master-slave teleoperation system, reducing the effective stiffness of the slave and the transparency of teleoperation. In this paper, we analyze transparency under slave link and joint flexibility (tool flexibility). We also evaluate the added benefits of using extra sensors at the tip of the flexible robot. It is shown that tip velocity (or position) feedback improves free-space position tracking performance in the presence of robot flexibility. Also, when the interaction forces with an environment are measured by a force sensor and fed back to the user’s hand, tip velocity feedback improves hard-contact force tracking performance. During a hard contact task, tip velocity feedback can also eliminate the transmission of robot flexibility to the user’s hand. Parts of this research have previously been published as M. Tavakoli, R.D. Howe, The Effect of Joint Elasticity on Bilateral Teleoperation, In Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots & Systems, pp. 1618-1623, San Diego, CA, 2007, and as M. Tavakoli, R.D. Howe, Haptic Implications of Tool Flexibility in Surgical Teleoperation, In Proceedings of the 16th Symposium on Haptic Interfaces for Virtual Environments & Teleoperator Systems, pp. 377-378, Reno, NV, March 2008.Engineering and Applied Science

Expert-in-the-Loop Multilateral Telerobotics for Haptics-Enabled Motor Function and Skills Development

Among medical robotics applications are Robotics-Assisted Mirror Rehabilitation Therapy (RAMRT) and Minimally-Invasive Surgical Training (RAMIST) that extensively rely on motor function development. Haptics-enabled expert-in-the-loop motor function development for such applications is made possible through multilateral telerobotic frameworks. While several studies have validated the benefits of haptic interaction with an expert in motor learning, contradictory results have also been reported. This emphasizes the need for further in-depth studies on the nature of human motor learning through haptic guidance and interaction. The objective of this study was to design and evaluate expert-in-the-loop multilateral telerobotic frameworks with stable and human-safe control loops that enable adaptive “hand-over-hand” haptic guidance for RAMRT and RAMIST. The first prerequisite for such frameworks is active involvement of the patient or trainee, which requires the closed-loop system to remain stable in the presence of an adaptable time-varying dominance factor. To this end, a wave-variable controller is proposed in this study for conventional trilateral teleoperation systems such that system stability is guaranteed in the presence of a time-varying dominance factor and communication delay. Similar to other wave-variable approaches, the controller is initially developed for the Velocity-force Domain (VD) based on the well-known passivity assumption on the human arm in VD. The controller can be applied straightforwardly to the Position-force Domain (PD), eliminating position-error accumulation and position drift, provided that passivity of the human arm in PD is addressed. However, the latter has been ignored in the literature. Therefore, in this study, passivity of the human arm in PD is investigated using mathematical analysis, experimentation as well as user studies involving 12 participants and 48 trials. The results, in conjunction with the proposed wave-variables, can be used to guarantee closed-loop PD stability of the supervised trilateral teleoperation system in its classical format. The classic dual-user teleoperation architecture does not, however, fully satisfy the requirements for properly imparting motor function (skills) in RAMRT (RAMIST). Consequently, the next part of this study focuses on designing novel supervised trilateral frameworks for providing motor learning in RAMRT and RAMIST, each customized according to the requirements of the application. The framework proposed for RAMRT includes the following features: a) therapist-in-the-loop mirror therapy; b) haptic feedback to the therapist from the patient side; c) assist-as-needed therapy realized through an adaptive Guidance Virtual Fixture (GVF); and d) real-time task-independent and patient-specific motor-function assessment. Closed-loop stability of the proposed framework is investigated using a combination of the Circle Criterion and the Small-Gain Theorem. The stability analysis addresses the instabilities caused by: a) communication delays between the therapist and the patient, facilitating haptics-enabled tele- or in-home rehabilitation; and b) the integration of the time-varying nonlinear GVF element into the delayed system. The platform is experimentally evaluated on a trilateral rehabilitation setup consisting of two Quanser rehabilitation robots and one Quanser HD2 robot. The framework proposed for RAMIST includes the following features: a) haptics-enabled expert-in-the-loop surgical training; b) adaptive expertise-oriented training, realized through a Fuzzy Interface System, which actively engages the trainees while providing them with appropriate skills-oriented levels of training; and c) task-independent skills assessment. Closed-loop stability of the architecture is analyzed using the Circle Criterion in the presence and absence of haptic feedback of tool-tissue interactions. In addition to the time-varying elements of the system, the stability analysis approach also addresses communication delays, facilitating tele-surgical training. The platform is implemented on a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA), integrated with the da Vinci Research Kit (dVRK) motor controllers, and the dV-Trainer master console (Mimic Technology Inc., Seattle, WA). In order to save on the expert\u27s (therapist\u27s) time, dual-console architectures can also be expanded to accommodate simultaneous training (rehabilitation) for multiple trainees (patients). As the first step in doing this, the last part of this thesis focuses on the development of a multi-master/single-slave telerobotic framework, along with controller design and closed-loop stability analysis in the presence of communication delays. Various parts of this study are supported with a number of experimental implementations and evaluations. The outcomes of this research include multilateral telerobotic testbeds for further studies on the nature of human motor learning and retention through haptic guidance and interaction. They also enable investigation of the impact of communication time delays on supervised haptics-enabled motor function improvement through tele-rehabilitation and mentoring

Enhanced teleoperation performance using hybrid control and virtual fixture

To develop secure, natural and effective teleoperation, the perception of the slave plays a key role for the interaction of a human operator with the environment. By sensing slave information, the human operator can choose the correct operation in a process during the human–robot interaction.This paper develops an integrated scheme based on a hybrid control and virtual fixture approach for the telerobot. The human operator can sense the slave interaction condition and adjust the master device via the surface electromyographic signal. This hybrid control method integrates the proportional-derivative control and the variable stiffness control, and involves the muscle activation at the same time. It is proposed to quantitatively analyse the human operator’s control demand to enhance the control performance of the teleoperation system. In addition, due to unskilful operation and muscle physiological tremor of the human operator, a virtual fixture method is developed to ensure accuracy of operation and to reduce the operation pressure on the human operator. Experimental results demonstrated the effectiveness of the proposed method for the teleoperated robot

Experimental Evaluation of the Projection-based Force Reflection Algorithms for Haptic Interaction with Virtual Environment

Haptic interaction with virtual environments is currently a major and growing area of research with a number of emerging applications, particularly in the field of robotics. Digital implementation of the virtual environments, however, introduces errors which may result in instability of the haptic displays. This thesis deals with experimental investigation of the Projection-Based Force Reflection Algorithms (PFRAs) for haptic interaction with virtual environments, focusing on their performance in terms of stability and transparency. Experiments were performed to compare the PFRA in terms of performance for both non-delayed and delayed haptic interactions with more conventional haptic rendering methods, such as the Virtual Coupling (VC) and Wave Variables (WV). The results demonstrated that the PFRA is more stable, guarantees higher levels of transparency, and is less sensitive to decrease in update rates

Haptics in Robot-Assisted Surgery: Challenges and Benefits

Robotic surgery is transforming the current surgical practice, not only by improving the conventional surgical methods but also by introducing innovative robot-enhanced approaches that broaden the capabilities of clinicians. Being mainly of man-machine collaborative type, surgical robots are seen as media that transfer pre- and intra-operative information to the operator and reproduce his/her motion, with appropriate filtering, scaling, or limitation, to physically interact with the patient. The field, however, is far from maturity and, more critically, is still a subject of controversy in medical communities. Limited or absent haptic feedback is reputed to be among reasons that impede further spread of surgical robots. In this paper objectives and challenges of deploying haptic technologies in surgical robotics is discussed and a systematic review is performed on works that have studied the effects of providing haptic information to the users in major branches of robotic surgery. It has been tried to encompass both classical works and the state of the art approaches, aiming at delivering a comprehensive and balanced survey both for researchers starting their work in this field and for the experts

A Wave Variable Approach with Multiple Channel Architecture for Teleoperated System

© 2013 IEEE. Performance of teleoperation can be greatly influenced by time delay in the process of tele-manipulation with respect to accuracy and transparency. Wave variable is an effective algorithm to achieve a good stable capability. However, some traditional wave variable methods may decrease the performance of transparency and suffer the impacts of wave reflection. To deal with the problem of stability and transparency in teleoperation, in this paper, a novel wave variable method with four channel is presented to achieve stable tracking in position and force. In addition, the proposed method can achieve the distortion compensation and reduce the impacts of wave reflection. The simulation experimental results verified the tracking performance of the proposed method

Steering control for haptic feedback and active safety functions

Steering feedback is an important element that defines driver–vehicle interaction. It strongly affects driving performance and is primarily dependent on the steering actuator\u27s control strategy. Typically, the control method is open loop, that is without any reference tracking; and its drawbacks are hardware dependent steering feedback response and attenuated driver–environment transparency. This thesis investigates a closed-loop control method for electric power assisted steering and steer-by-wire systems. The advantages of this method, compared to open loop, are better hardware impedance compensation, system independent response, explicit transparency control and direct interface to active safety functions.The closed-loop architecture, outlined in this thesis, includes a reference model, a feedback controller and a disturbance observer. The feedback controller forms the inner loop and it ensures: reference tracking, hardware impedance compensation and robustness against the coupling uncertainties. Two different causalities are studied: torque and position control. The two are objectively compared from the perspective of (uncoupled and coupled) stability, tracking performance, robustness, and transparency.The reference model forms the outer loop and defines a torque or position reference variable, depending on the causality. Different haptic feedback functions are implemented to control the following parameters: inertia, damping, Coulomb friction and transparency. Transparency control in this application is particularly novel, which is sequentially achieved. For non-transparent steering feedback, an environment model is developed such that the reference variable is a function of virtual dynamics. Consequently, the driver–steering interaction is independent from the actual environment. Whereas, for the driver–environment transparency, the environment interaction is estimated using an observer; and then the estimated signal is fed back to the reference model. Furthermore, an optimization-based transparency algorithm is proposed. This renders the closed-loop system transparent in case of environmental uncertainty, even if the initial condition is non-transparent.The steering related active safety functions can be directly realized using the closed-loop steering feedback controller. This implies, but is not limited to, an angle overlay from the vehicle motion control functions and a torque overlay from the haptic support functions.Throughout the thesis, both experimental and the theoretical findings are corroborated. This includes a real-time implementation of the torque and position control strategies. In general, it can be concluded that position control lacks performance and robustness due to high and/or varying system inertia. Though the problem is somewhat mitigated by a robust H-infinity controller, the high frequency haptic performance remains compromised. Whereas, the required objectives are simultaneously achieved using a torque controller

Posture-Dependent Projection-Based Force Reflection Algorithms for Bilateral Teleoperators

It was previously established that the projection-based force reflection (PBFR) algorithms improve the overall stability of a force reflecting teleoperation system. The idea behind the PBFR algorithms is to identify the component of the reflected force which is compensated by interaction with the operator\u27s hand, and subsequently attenuate the residual component of the reflected force. If there is no a priori information regarding the behaviour of the human operator, the PBFR gain is selected equal to sufficiently small constant in order to guarantee stability for a wide range of human operator responses. Small PBRF gains, however, may deteriorate the transparency of a teleoperator system. In this thesis, a new method for selecting the PBFR gain is introduced which depends on human postures. Using an online human posture estimation, the introduced posture-dependent PBFR algorithm has been applied to a teleoperation system with force feedback. It is experimentally demonstrated that the developed method for selection of the PBFR gain based on human postures improves the transparency of the teleoperator system while the stability is preserved. Finally, preliminary results that deal with an extension of the developed methods towards a more realistic model of the human arm with 4 degrees of freedom and three dimensional movements are presented