Novel observers for compensation of communication delay in bilateral control systems

The problem of communication delay in bilateral or teleoperation systems is even more emphasized with the use of the internet for communication, which may give rise to loss of transparency and even instability. To address the problem, numerous methods have been proposed. This study is among the few recent studies taking a disturbance observer approach to the problem of time delay, and introduces a novel sliding-mode observer to overcome specifically the effects of communication delay in the feedback loop. The observer operates in combination with a PD+ controller which controls the system dynamics, while also compensating load torque uncertainties on the slave side. To this aim, an EKF based load estimation algorithm is performed on the slave side. The performance of this approach is tested with computer simulations for the teleoperation of a 1-DOF robotic arm. The simulations reveal an acceptable amount of accuracy and transparency between the estimated slave and actual slave position under both constant and random measurement delay and variable and step-type load variations on the slave side, motivating the use of the approach for internet-based bilateral control systems

Experimental Evaluation of Novel Master-Slave Configurations for Position Control under Random Network Delay and Variable Load for Teleoperation

This paper proposes two novel master-slave configurations that provide improvements in both control and communication aspects of teleoperation systems to achieve an overall improved performance in position control. The proposed novel master-slave configurations integrate modular control and communication approaches, consisting of a delay regulator to address problems related to variable network delay common to such systems, and a model tracking control that runs on the slave side for the compensation of uncertainties and model mismatch on the slave side. One of the configurations uses a sliding mode observer and the other one uses a modified Smith predictor scheme on the master side to ensure position transparency between the master and slave, while reference tracking of the slave is ensured by a proportional-differentiator type controller in both configurations. Experiments conducted for the networked position control of a single-link arm under system uncertainties and randomly varying network delays demonstrate significant performance improvements with both configurations over the past literature

Network Latency in Teleoperation of Connected and Autonomous Vehicles:A Review of Trends, Challenges, and Mitigation Strategies

With remarkable advancements in the development of connected and autonomous vehicles (CAVs), the integration of teleoperation has become crucial for improving safety and operational efficiency. However, teleoperation faces substantial challenges, with network latency being a critical factor influencing its performance. This survey paper explores the impact of network latency along with state-of-the-art mitigation/compensation approaches. It examines cascading effects on teleoperation communication links (i.e., uplink and downlink) and how delays in data transmission affect the real-time perception and decision-making of operators. By elucidating the challenges and available mitigation strategies, the paper offers valuable insights for researchers, engineers, and practitioners working towards the seamless integration of teleoperation in the evolving landscape of CAVs

Wave Prediction and Delay Modeling for Teleoperation via Internet

This paper propose a novel approach for modeling the end-to-end time delay dynamics of the internet using system identification, and use it for controlling real-time internet-based telerobotic operations. When a single model is used, it needs to adapt to the operating conditions before an appropriate control mechanism can be applied. Slow adaptation may result in large transient errors. As an alternative, we propose to use an adaptive multiple model framework, and determine the best model for the current operating conditions to activate the corresponding controller. We employ multivariable wave prediction method to achieve this objective

Optimal motion control and vibration suppression of flexible systems with inaccessible outputs

This work addresses the optimal control problem of dynamical systems with inaccessible outputs. A case in which dynamical system outputs cannot be measured or inaccessible. This contradicts with the nature of the optimal controllers which can be considered without any loss of generality as state feedback control laws for systems with linear dynamics. Therefore, this work attempts to estimate dynamical system states through a novel state observer that does not require injecting the dynamical system outputs onto the observer structure during its design. A linear quadratic optimal control law is then realized based on the estimated states which allows controlling motion along with active vibration suppression of this class of dynamical systems with inaccessible outputs. Validity of the proposed control framework is evaluated experimentally

Passive Control Architectures for Collaborative Virtual Haptic Interaction and Bilateral Teleoperation over Unreliable Packet-Switched Digital Network

This PhD dissertation consists of two major parts: collaborative haptic interaction (CHI) and bilateral teleoperation over the Internet. For the CHI, we propose a novel hybrid peer-to-peer (P2P) architecture including the shared virtual environment (SVE) simulation, coupling between the haptic device and VE, and P2P synchronization control among all VE copies. This framework guarantees the interaction stability for all users with general unreliable packet-switched communication network which is the most challenging problem for CHI control framework design. This is achieved by enforcing our novel \emph{passivity condition} which fully considers time-varying non-uniform communication delays, random packet loss/swapping/duplication for each communication channel. The topology optimization method based on graph algebraic connectivity is also developed to achieve optimal performance under the communication bandwidth limitation. For validation, we implement a four-user collaborative haptic system with simulated unreliable packet-switched network connections. Both the hybrid P2P architecture design and the performance improvement due to the topology optimization are verified. In the second part, two novel hybrid passive bilateral teleoperation control architectures are proposed to address the challenging stability and performance issues caused by the general Internet communication unreliability (e.g. varying time delay, packet loss, data duplication, etc.). The first method--Direct PD Coupling (DPDC)--is an extension of traditional PD control to the hybrid teleoperation system. With the assumption that the Internet communication unreliability is upper bounded, the passive gain setting condition is derived and guarantees the interaction stability for the teleoperation system which interacts with unknown/unmodeled passive human and environment. However, the performance of DPDC degrades drastically when communication unreliability is severe because its feasible gain region is limited by the device viscous damping. The second method--Virtual Proxy Based PD Coupling (VPDC)--is proposed to improve the performance while providing the same interaction stability. Experimental and quantitative comparisons between DPDC and VPDC are conducted, and both interaction stability and performance difference are validated

Sensors Allocation and Observer Design for Discrete Bilateral Teleoperation Systems with Multi-Rate Sampling

This study addresses sensor allocation by analyzing exponential stability for discrete-time teleoperation systems. Previous studies mostly concentrate on the continuous-time teleoperation systems and neglect the management of significant practical phenomena, such as data-swap, the effect of sampling rates of samplers, and refresh rates of actuators on the system’s stability. A multi-rate sampling approach is proposed in this study, given the isolation of the master and slave robots in teleoperation systems which may have different hardware restrictions. This architecture collects data through numerous sensors with various sampling rates, assuming that a continuous-time controller stabilizes a linear teleoperation system. The aim is to assign each position and velocity signals to sensors with different sampling rates and divide the state vector between sensors to guarantee the stability of the resulting multi-rate sampled-data teleoperation system. Sufficient Krasovskii-based conditions will be provided to preserve the exponential stability of the system. This problem will be transformed into a mixed-integer program with LMIs (linear matrix inequalities). These conditions are also used to design the observers for the multi-rate teleoperation systems whose estimation errors converge exponentially to the origin. The results are validated by numerical simulations which are useful in designing sensor networks for teleoperation systems

The Shape of Damping: Optimizing Damping Coefficients to Improve Transparency on Bilateral Telemanipulation

This thesis presents a novel optimization-based passivity control algorithm for hapticenabled bilateral teleoperation systems involving multiple degrees of freedom. In particular, in the context of energy-bounding control, the contribution focuses on the implementation of a passivity layer for an existing time-domain scheme, ensuring optimal transparency of the interaction along subsets of the environment space which are preponderant for the given task, while preserving the energy bounds required for passivity. The involved optimization problem is convex and amenable to real-time implementation. The effectiveness of the proposed design is validated via an experiment performed on a virtual teleoperated environment. The interplay between transparency and stability is a critical aspect in haptic-enabled bilateral teleoperation control. While it is important to present the user with the true impedance of the environment, destabilizing factors such as time delays, stiff environments, and a relaxed grasp on the master device may compromise the stability and safety of the system. Passivity has been exploited as one of the the main tools for providing sufficient conditions for stable teleoperation in several controller design approaches, such as the scattering algorithm, timedomain passivity control, energy bounding algorithm, and passive set position modulation. In this work it is presented an innovative energy-based approach, which builds upon existing time-domain passivity controllers, improving and extending their effectiveness and functionality. The set of damping coefficients are prioritized in each degree of freedom, the resulting transparency presents a realistic force feedback in comparison to the other directions. Thus, the prioritization takes effect using a quadratic programming algorithm to find the optimal values for the damping. Finally, the energy tanks approach on passivity control is a solution used to ensure stability in a system for robotics bilateral manipulation. The bilateral telemanipulation must maintain the principle of passivity in all moments to preserve the system\u2019s stability. This work presents a brief introduction to haptic devices as a master component on the telemanipulation chain; the end effector in the slave side is a representation of an interactive object within an environment having a force sensor as feedback signal. The whole interface is designed into a cross-platform framework named ROS, where the user interacts with the system. Experimental results are presented

Enhanced teleoperation performance using hybrid control and virtual fixture

To develop secure, natural and effective teleoperation, the perception of the slave plays a key role for the interaction of a human operator with the environment. By sensing slave information, the human operator can choose the correct operation in a process during the human–robot interaction.This paper develops an integrated scheme based on a hybrid control and virtual fixture approach for the telerobot. The human operator can sense the slave interaction condition and adjust the master device via the surface electromyographic signal. This hybrid control method integrates the proportional-derivative control and the variable stiffness control, and involves the muscle activation at the same time. It is proposed to quantitatively analyse the human operator’s control demand to enhance the control performance of the teleoperation system. In addition, due to unskilful operation and muscle physiological tremor of the human operator, a virtual fixture method is developed to ensure accuracy of operation and to reduce the operation pressure on the human operator. Experimental results demonstrated the effectiveness of the proposed method for the teleoperated robot