Network Latency in Teleoperation of Connected and Autonomous Vehicles:A Review of Trends, Challenges, and Mitigation Strategies

With remarkable advancements in the development of connected and autonomous vehicles (CAVs), the integration of teleoperation has become crucial for improving safety and operational efficiency. However, teleoperation faces substantial challenges, with network latency being a critical factor influencing its performance. This survey paper explores the impact of network latency along with state-of-the-art mitigation/compensation approaches. It examines cascading effects on teleoperation communication links (i.e., uplink and downlink) and how delays in data transmission affect the real-time perception and decision-making of operators. By elucidating the challenges and available mitigation strategies, the paper offers valuable insights for researchers, engineers, and practitioners working towards the seamless integration of teleoperation in the evolving landscape of CAVs

STABILITY AND PERFORMANCE OF NETWORKED CONTROL SYSTEMS

Network control systems (NCSs), as one of the most active research areas, are arousing comprehensive concerns along with the rapid development of network. This dissertation mainly discusses the stability and performance of NCSs into the following two parts. In the first part, a new approach is proposed to reduce the data transmitted in networked control systems (NCSs) via model reduction method. Up to our best knowledge, we are the first to propose this new approach in the scientific and engineering society. The "unimportant" information of system states vector is truncated by balanced truncation method (BTM) before sending to the networked controller via network based on the balance property of the remote controlled plant controllability and observability. Then, the exponential stability condition of the truncated NCSs is derived via linear matrix inequality (LMI) forms. This method of data truncation can usually reduce the time delay and further improve the performance of the NCSs. In addition, all the above results are extended to the switched NCSs. The second part presents a new robust sliding mode control (SMC) method for general uncertain time-varying delay stochastic systems with structural uncertainties and the Brownian noise (Wiener process). The key features of the proposed method are to apply singular value decomposition (SVD) to all structural uncertainties, to introduce adjustable parameters for control design along with the SMC method, and new Lyapunov-type functional. Then, a less-conservative condition for robust stability and a new robust controller for the general uncertain stochastic systems are derived via linear matrix inequality (LMI) forms. The system states are able to reach the SMC switching surface as guaranteed in probability 1 by the proposed control rule. Furthermore, the novel Lyapunov-type functional for the uncertain stochastic systems is used to design a new robust control for the general case where the derivative of time-varying delay can be any bounded value (e.g., greater than one). It is theoretically proved that the conservatism of the proposed method is less than the previous methods. All theoretical proofs are presented in the dissertation. The simulations validate the correctness of the theoretical results and have better performance than the existing results

Expert-in-the-Loop Multilateral Telerobotics for Haptics-Enabled Motor Function and Skills Development

Among medical robotics applications are Robotics-Assisted Mirror Rehabilitation Therapy (RAMRT) and Minimally-Invasive Surgical Training (RAMIST) that extensively rely on motor function development. Haptics-enabled expert-in-the-loop motor function development for such applications is made possible through multilateral telerobotic frameworks. While several studies have validated the benefits of haptic interaction with an expert in motor learning, contradictory results have also been reported. This emphasizes the need for further in-depth studies on the nature of human motor learning through haptic guidance and interaction. The objective of this study was to design and evaluate expert-in-the-loop multilateral telerobotic frameworks with stable and human-safe control loops that enable adaptive “hand-over-hand” haptic guidance for RAMRT and RAMIST. The first prerequisite for such frameworks is active involvement of the patient or trainee, which requires the closed-loop system to remain stable in the presence of an adaptable time-varying dominance factor. To this end, a wave-variable controller is proposed in this study for conventional trilateral teleoperation systems such that system stability is guaranteed in the presence of a time-varying dominance factor and communication delay. Similar to other wave-variable approaches, the controller is initially developed for the Velocity-force Domain (VD) based on the well-known passivity assumption on the human arm in VD. The controller can be applied straightforwardly to the Position-force Domain (PD), eliminating position-error accumulation and position drift, provided that passivity of the human arm in PD is addressed. However, the latter has been ignored in the literature. Therefore, in this study, passivity of the human arm in PD is investigated using mathematical analysis, experimentation as well as user studies involving 12 participants and 48 trials. The results, in conjunction with the proposed wave-variables, can be used to guarantee closed-loop PD stability of the supervised trilateral teleoperation system in its classical format. The classic dual-user teleoperation architecture does not, however, fully satisfy the requirements for properly imparting motor function (skills) in RAMRT (RAMIST). Consequently, the next part of this study focuses on designing novel supervised trilateral frameworks for providing motor learning in RAMRT and RAMIST, each customized according to the requirements of the application. The framework proposed for RAMRT includes the following features: a) therapist-in-the-loop mirror therapy; b) haptic feedback to the therapist from the patient side; c) assist-as-needed therapy realized through an adaptive Guidance Virtual Fixture (GVF); and d) real-time task-independent and patient-specific motor-function assessment. Closed-loop stability of the proposed framework is investigated using a combination of the Circle Criterion and the Small-Gain Theorem. The stability analysis addresses the instabilities caused by: a) communication delays between the therapist and the patient, facilitating haptics-enabled tele- or in-home rehabilitation; and b) the integration of the time-varying nonlinear GVF element into the delayed system. The platform is experimentally evaluated on a trilateral rehabilitation setup consisting of two Quanser rehabilitation robots and one Quanser HD2 robot. The framework proposed for RAMIST includes the following features: a) haptics-enabled expert-in-the-loop surgical training; b) adaptive expertise-oriented training, realized through a Fuzzy Interface System, which actively engages the trainees while providing them with appropriate skills-oriented levels of training; and c) task-independent skills assessment. Closed-loop stability of the architecture is analyzed using the Circle Criterion in the presence and absence of haptic feedback of tool-tissue interactions. In addition to the time-varying elements of the system, the stability analysis approach also addresses communication delays, facilitating tele-surgical training. The platform is implemented on a dual-console surgical setup consisting of the classic da Vinci surgical system (Intuitive Surgical, Inc., Sunnyvale, CA), integrated with the da Vinci Research Kit (dVRK) motor controllers, and the dV-Trainer master console (Mimic Technology Inc., Seattle, WA). In order to save on the expert\u27s (therapist\u27s) time, dual-console architectures can also be expanded to accommodate simultaneous training (rehabilitation) for multiple trainees (patients). As the first step in doing this, the last part of this thesis focuses on the development of a multi-master/single-slave telerobotic framework, along with controller design and closed-loop stability analysis in the presence of communication delays. Various parts of this study are supported with a number of experimental implementations and evaluations. The outcomes of this research include multilateral telerobotic testbeds for further studies on the nature of human motor learning and retention through haptic guidance and interaction. They also enable investigation of the impact of communication time delays on supervised haptics-enabled motor function improvement through tele-rehabilitation and mentoring

GestureMoRo: an algorithm for autonomous mobile robot teleoperation based on gesture recognition

Gestures are a common way people communicate. Gesture-based teleoperation control systems tend to be simple to operate and suitable for most people’s daily use. This paper employed a LeapMotion sensor to develop a mobile robot control system based on gesture recognition, which mainly established connections through a client/server structure. The principles of gesture recognition in the system were studied and the relevant self-investigated algorithms—GestureMoRo, for the association between gestures and mobile robots were designed. Moreover, in order to avoid the unstably fluctuated movement of the mobile robot caused by palm shaking, the Gaussian filter algorithm was used to smooth and denoise the collected gesture data, which effectively improved the robustness and stability of the mobile robot’s locomotion. Finally, the teleoperation control strategy of the gesture to the WATER2 mobile robot was realized, and the effectiveness and practicability of the designed system were verified through multiple experiments

Learning Optimal Control of Synchronization in Networks of Coupled Oscillators using Genetic Programming-based Symbolic Regression

Networks of coupled dynamical systems provide a powerful way to model systems with enormously complex dynamics, such as the human brain. Control of synchronization in such networked systems has far reaching applications in many domains, including engineering and medicine. In this paper, we formulate the synchronization control in dynamical systems as an optimization problem and present a multi-objective genetic programming-based approach to infer optimal control functions that drive the system from a synchronized to a non-synchronized state and vice-versa. The genetic programming-based controller allows learning optimal control functions in an interpretable symbolic form. The effectiveness of the proposed approach is demonstrated in controlling synchronization in coupled oscillator systems linked in networks of increasing order complexity, ranging from a simple coupled oscillator system to a hierarchical network of coupled oscillators. The results show that the proposed method can learn highly-effective and interpretable control functions for such systems.Comment: Submitted to nonlinear dynamic

Medical Images Encryption Based on Adaptive-Robust Multi-Mode Synchronization of Chen Hyper-Chaotic Systems

In this paper, a novel medical image encryption method based on multi-mode synchronization of hyper-chaotic systems is presented. The synchronization of hyper-chaotic systems is of great significance in secure communication tasks such as encryption of images. Multi-mode synchronization is a novel and highly complex issue, especially if there is uncertainty and disturbance. In this work, an adaptive-robust controller is designed for multimode synchronized chaotic systems with variable and unknown parameters, despite the bounded disturbance and uncertainty with a known function in two modes. In the first case, it is a main system with some response systems, and in the second case, it is a circular synchronization. Using theorems it is proved that the two synchronization methods are equivalent. Our results show that, we are able to obtain the convergence of synchronization error and parameter estimation error to zero using Lyapunov’s method. The new laws to update time-varying parameters, estimating disturbance and uncertainty bounds are proposed such that stability of system is guaranteed. To assess the performance of the proposed synchronization method, various statistical analyzes were carried out on the encrypted medical images and standard benchmark images. The results show effective performance of the proposed synchronization technique in the medical images encryption for telemedicine application.MINECO/ FEDER under the RTI2018-098913-B100 CV20-45250 and A-TIC- 080-UGR18 project

Optimized state feedback regulation of 3DOF helicopter system via extremum seeking

In this paper, an optimized state feedback regulation of a 3 degree of freedom (DOF) helicopter is designed via extremum seeking (ES) technique. Multi-parameter ES is applied to optimize the tracking performance via tuning State Vector Feedback with Integration of the Control Error (SVFBICE). Discrete multivariable version of ES is developed to minimize a cost function that measures the performance of the controller. The cost function is a function of the error between the actual and desired axis positions. The controller parameters are updated online as the optimization takes place. This method significantly decreases the time in obtaining optimal controller parameters. Simulations were conducted for the online optimization under both fixed and varying operating conditions. The results demonstrate the usefulness of using ES for preserving the maximum attainable performance