Network Latency in Teleoperation of Connected and Autonomous Vehicles:A Review of Trends, Challenges, and Mitigation Strategies

With remarkable advancements in the development of connected and autonomous vehicles (CAVs), the integration of teleoperation has become crucial for improving safety and operational efficiency. However, teleoperation faces substantial challenges, with network latency being a critical factor influencing its performance. This survey paper explores the impact of network latency along with state-of-the-art mitigation/compensation approaches. It examines cascading effects on teleoperation communication links (i.e., uplink and downlink) and how delays in data transmission affect the real-time perception and decision-making of operators. By elucidating the challenges and available mitigation strategies, the paper offers valuable insights for researchers, engineers, and practitioners working towards the seamless integration of teleoperation in the evolving landscape of CAVs

Remote interaction with mobile robots

This paper describes an architecture, which can be used to build remote laboratories to interact remotely via Internet with mobile robots using different interaction devices. A supervisory control strategy has been used to develop the remote laboratory in order to alleviate high communication data rates and system sensitivity to network delays. The users interact with the remote system at a more abstract level using high level commands. The local robot's autonomy has been increased by encapsulating all the robot's behaviors in different types of skills. User interfaces have been designed using visual proxy pattern to facilitate any future extension or code reuse. The developed remote laboratory has been integrated into an educational environment in the field of indoor mobile robotics. This environment is currently being used as a part of an international project to develop a distributed laboratory for autonomous and teleoperated systems (IECAT, 2003).Publicad

Digital Cognitive Companions for Marine Vessels : On the Path Towards Autonomous Ships

As for the automotive industry, industry and academia are making extensive efforts to create autonomous ships. The solutions for this are very technology-intense. Many building blocks, often relying on AI technology, need to work together to create a complete system that is safe and reliable to use. Even when the ships are fully unmanned, humans are still foreseen to guide the ships when unknown situations arise. This will be done through teleoperation systems.In this thesis, methods are presented to enhance the capability of two building blocks that are important for autonomous ships; a positioning system, and a system for teleoperation.The positioning system has been constructed to not rely on the Global Positioning System (GPS), as this system can be jammed or spoofed. Instead, it uses Bayesian calculations to compare the bottom depth and magnetic field measurements with known sea charts and magnetic field maps, in order to estimate the position. State-of-the-art techniques for this method typically use high-resolution maps. The problem is that there are hardly any high-resolution terrain maps available in the world. Hence we present a method using standard sea-charts. We compensate for the lower accuracy by using other domains, such as magnetic field intensity and bearings to landmarks. Using data from a field trial, we showed that the fusion method using multiple domains was more robust than using only one domain. In the second building block, we first investigated how 3D and VR approaches could support the remote operation of unmanned ships with a data connection with low throughput, by comparing respective graphical user interfaces (GUI) with a Baseline GUI following the currently applied interfaces in such contexts. Our findings show that both the 3D and VR approaches outperform the traditional approach significantly. We found the 3D GUI and VR GUI users to be better at reacting to potentially dangerous situations than the Baseline GUI users, and they could keep track of the surroundings more accurately. Building from this, we conducted a teleoperation user study using real-world data from a field-trial in the archipelago, where the users should assist the positioning system with bearings to landmarks. The users experienced the tool to give a good overview, and despite the connection with the low throughput, they managed through the GUI to significantly improve the positioning accuracy

Kinesthetic Haptics Sensing and Discovery with Bilateral Teleoperation Systems

In the mechanical engineering field of robotics, bilateral teleoperation is a classic but still increasing research topic. In bilateral teleoperation, a human operator moves the master manipulator, and a slave manipulator is controlled to follow the motion of the master in a remote, potentially hostile environment. This dissertation focuses on kinesthetic perception analysis in teleoperation systems. Design of the controllers of the systems is studied as the influential factor of this issue. The controllers that can provide different force tracking capability are compared using the same experimental protocol. A 6 DOF teleoperation system is configured as the system testbed. An innovative master manipulator is developed and a 7 DOF redundant manipulator is used as the slave robot. A singularity avoidance inverse kinematics algorithm is developed to resolve the redundancy of the slave manipulator. An experimental protocol is addressed and three dynamics attributes related to kineshtetic feedback are investigated: weight, center of gravity and inertia. The results support our hypothesis: the controller that can bring a better force feedback can improve the performance in the experiments

Migration from Teleoperation to Autonomy via Modular Sensor and Mobility Bricks

In this thesis, the teleoperated communications of a Remotec ANDROS robot have been reverse engineered. This research has used the information acquired through the reverse engineering process to enhance the teleoperation and add intelligence to the initially automated robot. The main contribution of this thesis is the implementation of the mobility brick paradigm, which enables autonomous operations, using the commercial teleoperated ANDROS platform. The brick paradigm is a generalized architecture for a modular approach to robotics. This architecture and the contribution of this thesis are a paradigm shift from the proprietary commercial models that exist today. The modular system of sensor bricks integrates the transformed mobility platform and defines it as a mobility brick. In the wall following application implemented in this work, the mobile robotic system acquires intelligence using the range sensor brick. This application illustrates a way to alleviate the burden on the human operator and delegate certain tasks to the robot. Wall following is one among several examples of giving a degree of autonomy to an essentially teleoperated robot through the Sensor Brick System. Indeed once the proprietary robot has been altered into a mobility brick; the possibilities for autonomy are numerous and vary with different sensor bricks. The autonomous system implemented is not a fixed-application robot but rather a non-specific autonomy capable platform. Meanwhile the native controller and the computer-interfaced teleoperation are still available when necessary. Rather than trading off by switching from teleoperation to autonomy, this system provides the flexibility to switch between the two at the operator’s command. The contributions of this thesis reside in the reverse engineering of the original robot, its upgrade to a computer-interfaced teleoperated system, the mobility brick paradigm and the addition of autonomy capabilities. The application of a robot autonomously following a wall is subsequently implemented, tested and analyzed in this work. The analysis provides the programmer with information on controlling the robot and launching the autonomous function. The results are conclusive and open up the possibilities for a variety of autonomous applications for mobility platforms using modular sensor bricks

A white paper: NASA virtual environment research, applications, and technology

Research support for Virtual Environment technology development has been a part of NASA's human factors research program since 1985. Under the auspices of the Office of Aeronautics and Space Technology (OAST), initial funding was provided to the Aerospace Human Factors Research Division, Ames Research Center, which resulted in the origination of this technology. Since 1985, other Centers have begun using and developing this technology. At each research and space flight center, NASA missions have been major drivers of the technology. This White Paper was the joint effort of all the Centers which have been involved in the development of technology and its applications to their unique missions. Appendix A is the list of those who have worked to prepare the document, directed by Dr. Cynthia H. Null, Ames Research Center, and Dr. James P. Jenkins, NASA Headquarters. This White Paper describes the technology and its applications in NASA Centers (Chapters 1, 2 and 3), the potential roles it can take in NASA (Chapters 4 and 5), and a roadmap of the next 5 years (FY 1994-1998). The audience for this White Paper consists of managers, engineers, scientists and the general public with an interest in Virtual Environment technology. Those who read the paper will determine whether this roadmap, or others, are to be followed

An Adaptive Tool-Based Telerobot Control System

Modern telerobotics concepts seek to improve the work efficiency and quality of remote operations. The unstructured nature of typical remote operational environments makes autonomous operation of telerobotic systems difficult to achieve. Thus, human operators must always remain in the control loop for safety reasons. Remote operations involve tooling interactions with task environment. These interactions can be strong enough to promote unstable operation sometimes leading to system failures. Interestingly, manipulator/tooling dynamic interactions have not been studied in detail. This dissertation introduces a human-machine cooperative telerobotic (HMCTR) system architecture that has the ability to incorporate tooling interaction control and other computer assistance functions into the overall control system. A universal tooling interaction force prediction model has been created and implemented using grey system theory. Finally, a grey prediction force/position parallel fuzzy controller has been developed that compensates for the tooling interaction forces. Detailed experiments using a full-scale telerobotics testbed indicate: (i) the feasibility of the developed methodologies, and (ii) dramatic improvements in the stability of manipulator – based on band saw cutting operations. These results are foundational toward the further enhancement and development of telerobot