Enhanced teleoperation performance using hybrid control and virtual fixture

To develop secure, natural and effective teleoperation, the perception of the slave plays a key role for the interaction of a human operator with the environment. By sensing slave information, the human operator can choose the correct operation in a process during the human–robot interaction.This paper develops an integrated scheme based on a hybrid control and virtual fixture approach for the telerobot. The human operator can sense the slave interaction condition and adjust the master device via the surface electromyographic signal. This hybrid control method integrates the proportional-derivative control and the variable stiffness control, and involves the muscle activation at the same time. It is proposed to quantitatively analyse the human operator’s control demand to enhance the control performance of the teleoperation system. In addition, due to unskilful operation and muscle physiological tremor of the human operator, a virtual fixture method is developed to ensure accuracy of operation and to reduce the operation pressure on the human operator. Experimental results demonstrated the effectiveness of the proposed method for the teleoperated robot

Teleoperation control based on combination of wave variable and neural networks

In this paper, a novel control scheme is developed for a teleoperation system, combining the radial basis function (RBF) neural networks (NNs) and wave variable technique to simultaneously compensate for the effects caused by communication delays and dynamics uncertainties. The teleoperation system is set up with a TouchX joystick as the master device and a simulated Baxter robot arm as the slave robot. The haptic feedback is provided to the human operator to sense the interaction force between the slave robot and the environment when manipulating the stylus of the joystick. To utilize the workspace of the telerobot as much as possible, a matching process is carried out between the master and the slave based on their kinematics models. The closed loop inverse kinematics method and RBF NN approximation technique are seamlessly integrated in the control design. To overcome the potential instability problem in the presence of delayed communication channels, wave variables and their corrections are effectively embedded into the control system, and Lyapunov-based analysis is performed to theoretically establish the closed-loop stability. Comparative experiments have been conducted for a trajectory tracking task, under the different conditions of various communication delays. Experimental results show that in terms of tracking performance and force reflection, the proposed control approach shows superior performance over the conventional methods

An adaptive fuzzy control for human-in-the-loop operations with varying communication time delays

Time delay, especially varying time delay, is always an important factor affecting the stability to the human-in-the-loop system. Previous research usually focuses on the performance of the internal signal transmission part, but rarely considers the whole system with human and environmental factors comprehensively. For the problem, we investigate this issue by using an improved proportional-derivative-like plus damping (PD-like + d) control method, the derivative term of which is calculated based on the estimation of time delays. An embedded adaptive fuzzy logic systems (FLS)-based observer is developed to estimate and compensate for the errors caused by time delay estimations and uncertain force/ torque measuring errors. The advantages of the proposed control scheme are discussed by building an environmental input energy function and the effectiveness is also verified by the comparative simulations. The results show that under the same simulation conditions, the follower can track the leader's movements well, and the energy introduced into the environment is the same as that of the leader, which means the extra energy is dissipated to enable the object to be manipulated as desired by the leader side

Security framework for industrial collaborative robotic cyber-physical systems

The paper introduces a security framework for the application of human-robot collaboration in a futuristic industrial cyber-physical system (CPS) context of industry 4.0. The basic elements and functional requirements of a secure collaborative robotic cyber-physical system are explained and then the cyber-attack modes are discussed in the context of collaborative CPS whereas a defense mechanism strategy is proposed for such a complex system. The cyber-attacks are categorized according to the extent on controllability and the possible effects on the performance and efficiency of such CPS. The paper also describes the severity and categorization of such cyber-attacks and the causal effect on the human worker safety during human-robot collaboration. Attacks in three dimensions of availability, authentication and confidentiality are proposed as the basis of a consolidated mitigation plan. We propose a security framework based on a two-pronged strategy where the impact of this methodology is demonstrated on a teleoperation benchmark (NeCS-Car). The mitigation strategy includes enhanced data security at important interconnected adaptor nodes and development of an intelligent module that employs a concept similar to system health monitoring and reconfiguration

Expert-in-the-Loop Multilateral Telerobotics for Haptics-Enabled Motor Function and Skills Development

Among medical robotics applications are Robotics-Assisted Mirror Rehabilitation Therapy (RAMRT) and Minimally-Invasive Surgical Training (RAMIST) that extensively rely on motor function development. Haptics-enabled expert-in-the-loop motor function development for such applications is made possible through multilateral telerobotic frameworks. While several studies have validated the benefits of haptic interaction with an expert in motor learning, contradictory results have also been reported. This emphasizes the need for further in-depth studies on the nature of human motor learning through haptic guidance and interaction. The objective of this study was to design and evaluate expert-in-the-loop multilateral telerobotic frameworks with stable and human-safe control loops that enable adaptive “hand-over-hand” haptic guidance for RAMRT and RAMIST. The first prerequisite for such frameworks is active involvement of the patient or trainee, which requires the closed-loop system to remain stable in the presence of an adaptable time-varying dominance factor. To this end, a wave-variable controller is proposed in this study for conventional trilateral teleoperation systems such that system stability is guaranteed in the presence of a time-varying dominance factor and communication delay. Similar to other wave-variable approaches, the controller is initially developed for the Velocity-force Domain (VD) based on the well-known passivity assumption on the human arm in VD. The controller can be applied straightforwardly to the Position-force Domain (PD), eliminating position-error accumulation and position drift, provided that passivity of the human arm in PD is addressed. However, the latter has been ignored in the literature. Therefore, in this study, passivity of the human arm in PD is investigated using mathematical analysis, experimentation as well as user studies involving 12 participants and 48 trials. The results, in conjunction with the proposed wave-variables, can be used to guarantee closed-loop PD stability of the supervised trilateral teleoperation system in its classical format. The classic dual-user teleoperation architecture does not, however, fully satisfy the requirements for properly imparting motor function (skills) in RAMRT (RAMIST). Consequently, the next part of this study focuses on designing novel supervised trilateral frameworks for providing motor learning in RAMRT and RAMIST, each customized according to the requirements of the application. The framework proposed for RAMRT includes the following features: a) therapist-in-the-loop mirror therapy; b) haptic feedback to the therapist from the patient side; c) assist-as-needed therapy realized through an adaptive Guidance Virtual Fixture (GVF); and d) real-time task-independent and patient-specific motor-function assessment. Closed-loop stability of the proposed framework is investigated using a combination of the Circle Criterion and the Small-Gain Theorem. The stability analysis addresses the instabilities caused by: a) communication delays between the therapist and the patient, facilitating haptics-enabled tele- or in-home rehabilitation; and b) the integration of the time-varying nonlinear GVF element into the delayed system. The platform is experimentally evaluated on a trilateral rehabilitation setup consisting of two Quanser rehabilitation robots and one Quanser HD2 robot. The framework proposed for RAMIST includes the following features: a) haptics-enabled expert-in-the-loop surgical training; b) adaptive expertise-oriented training, realized through a Fuzzy Interface System, which actively engages the trainees while providing them with appropriate skills-oriented levels of training; and c) task-independent skills assessment. Closed-loop stability of the architecture is analyzed using the Circle Criterion in the presence and absence of haptic feedback of tool-tissue interactions. In addition to the time-varying elements of the system, the stability analysis approach also addresses communication delays, facilitating tele-surgical training. The platform is implemented on a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA), integrated with the da Vinci Research Kit (dVRK) motor controllers, and the dV-Trainer master console (Mimic Technology Inc., Seattle, WA). In order to save on the expert\u27s (therapist\u27s) time, dual-console architectures can also be expanded to accommodate simultaneous training (rehabilitation) for multiple trainees (patients). As the first step in doing this, the last part of this thesis focuses on the development of a multi-master/single-slave telerobotic framework, along with controller design and closed-loop stability analysis in the presence of communication delays. Various parts of this study are supported with a number of experimental implementations and evaluations. The outcomes of this research include multilateral telerobotic testbeds for further studies on the nature of human motor learning and retention through haptic guidance and interaction. They also enable investigation of the impact of communication time delays on supervised haptics-enabled motor function improvement through tele-rehabilitation and mentoring