Network Latency in Teleoperation of Connected and Autonomous Vehicles:A Review of Trends, Challenges, and Mitigation Strategies

With remarkable advancements in the development of connected and autonomous vehicles (CAVs), the integration of teleoperation has become crucial for improving safety and operational efficiency. However, teleoperation faces substantial challenges, with network latency being a critical factor influencing its performance. This survey paper explores the impact of network latency along with state-of-the-art mitigation/compensation approaches. It examines cascading effects on teleoperation communication links (i.e., uplink and downlink) and how delays in data transmission affect the real-time perception and decision-making of operators. By elucidating the challenges and available mitigation strategies, the paper offers valuable insights for researchers, engineers, and practitioners working towards the seamless integration of teleoperation in the evolving landscape of CAVs

Design and development of an interface for the remote control of a DC motor

Remote laboratories have been gaining popularity since their first appearance two decades ago as an alternative to hands on laboratories. They present numerous advantages for universities such as lower costs and flexibility, meanwhile, maintaining the benefits of dealing with real systems that virtual laboratories do not present. In this bachelor thesis, an interface to interact in remote with a DC motor, that can be used as a laboratory session was developed. A full stack application using HTML, CSS, JavaScript and Python languages was implemented. The front end of the application is a web page with real time graphs and a video feed with low latency. The back end includes a web server that attends the requests from the user, via web sockets, and performs the control loop for the motor, as well as managing the camera. The hardware of the platform was selected after a detailed research, advantages and disadvantages of potential elements alternatives will be explained. The platform also incorporates designed and printed 3D pieces. The system was successfully identified using Python and MATLABs code and its response in velocity and position is simulated with Simulink Tool.Ingeniería Electrónica Industrial y Automátic

Gesture Recognition and Control for Semi-Autonomous Robotic Assistant Surgeons

The next stage for robotics development is to introduce autonomy and cooperation with human agents in tasks that require high levels of precision and/or that exert considerable physical strain. To guarantee the highest possible safety standards, the best approach is to devise a deterministic automaton that performs identically for each operation. Clearly, such approach inevitably fails to adapt itself to changing environments or different human companions. In a surgical scenario, the highest variability happens for the timing of different actions performed within the same phases. This thesis explores the solutions adopted in pursuing automation in robotic minimally-invasive surgeries (R-MIS) and presents a novel cognitive control architecture that uses a multi-modal neural network trained on a cooperative task performed by human surgeons and produces an action segmentation that provides the required timing for actions while maintaining full phase execution control via a deterministic Supervisory Controller and full execution safety by a velocity-constrained Model-Predictive Controller

DSAAR: distributed software architecture for autonomous robots

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do grau de Mestre em Engenharia ElectrotécnicaThis dissertation presents a software architecture called the Distributed Software Architecture for Autonomous Robots (DSAAR), which is designed to provide the fast development and prototyping of multi-robot systems. The DSAAR building blocks allow engineers to focus on the behavioural model of robots and collectives. This architecture is of special interest in domains where several human, robot, and software agents have to interact continuously. Thus, fast prototyping and reusability is a must. DSAAR tries to cope with these requirements towards an advanced solution to the n-humans and m-robots problem with a set of design good practices and development tools. This dissertation will also focus on Human-Robot Interaction, mainly on the subject of teleoperation. In teleoperation human judgement is an integral part of the process, heavily influenced by the telemetry data received from the remote environment. So the speed in which commands are given and the telemetry data is received, is of crucial importance. Using the DSAAR architecture a teleoperation approach is proposed. This approach was designed to provide all entities present in the network a shared reality, where every entity is an information source in an approach similar to the distributed blackboard. This solution was designed to accomplish a real time response, as well as, the completest perception of the robots’ surroundings. Experimental results obtained with the physical robot suggest that the system is able to guarantee a close interaction between users and robot

Low Latency High-definition Video Streaming for Real-time Teleoperation Platform

Teleoperation is the remote controlling of machines using a real-time video stream to support the controlling decisions. The key components of a teleoperation system are video streaming through a network to enable the control, low enough latency to ensure real-time control and latency - bandwidth - resolution balance of the streaming system. In general, a low latency means high bandwidth consumption and the used resolution relates also straight to the bandwidth. Using the modern video coding method H.264/AVC allows for the reduction of bandwidth and latency by selecting a suitable H.264 profile. This thesis studies the possibility and effect of maximizing the usage of a graphics processing unit (GPU) in the streaming pipeline of a teleoperation platform and presents measurements to show the impact on the streaming latency. An open source multimedia framework GStreamer is used in the pipeline construction of the platform. The thesis presents the creation of two teleoperation platforms and examines their features. Latency measurements between an existing system and one of the developed systems are compared and results discussed. The results show that employing a GPU in the streaming pipeline greatly improves the performance of the system and allows the streaming of multiple simultaneous low latency high resolution video streams

Digital twin-enabled human-robot collaborative teaming towards sustainable and healthy built environments

Development of sustainable and healthy built environments (SHBE) is highly advocated to achieve collective societal good. Part of the pathway to SHBE is the engagement of robots to manage the ever-complex facilities for tasks such as inspection and disinfection. However, despite the increasing advancements of robot intelligence, it is still “mission impossible” for robots to independently undertake such open-ended problems as facility management, calling for a need to “team up” the robots with humans. Leveraging digital twin's ability to capture real-time data and inform decision-making via dynamic simulation, this study aims to develop a human-robot teaming framework for facility management to achieve sustainability and healthiness in the built environments. A digital twin-enabled prototype system is developed based on the framework. Case studies showed that the framework can safely and efficiently incorporate robotics into facility management tasks (e.g., patrolling, inspection, and cleaning) by allowing humans to plan, oversee, manage, and cooperate with the robot via the digital twin's bi-directional mechanism. The study lays out a high-level framework, under which purposeful efforts can be made to unlock digital twin's full potential in collaborating humans and robots in facility management towards SHBE

ROS based Teleoperation and Docking of a Low Speed Urban Vehicle

In recent years, 4G LTE technology has provided us with higher than ever transfer speeds over the cellular networks, permitting streaming of video and other high bandwidth services. On the other hand, there has been a rapid development and an explosion of interest in frameworks for robot software development, particularly ROS. Though there have been many studies which have leveraged 4G LTE network as the mode of communication when studying teleoperations, a very few studies have used 4G LTE network with ROS framework for building teleoperated systems. Therefore, this study seeks to build a teleoperated system using the ROS framework which employs the 4G LTE network for communication. For this purpose, a prototype system is built using a remote-controlled low speed urban vehicle that hosts a multimedia link between the vehicle and the control station. The operator drives the vehicle remotely primarily based on processed video feed and LIDAR data. The vehicle is also equipped with safety systems to avoid collisions. The teleoperated system built is tested by asking an experienced driver to complete certain tasks while driving the vehicle remotely. Moreover, this study also intends to build an autonomous docking procedure for the vehicle. A docking procedure based on differential GPS and video feedback is built that allows the vehicle to autonomously dock itself into a charging station. The procedure provides a proof of concept solution for the autonomous charging/fueling of self-driving cars.  M.S

Machine Learning Algorithms for Robotic Navigation and Perception and Embedded Implementation Techniques

L'abstract è presente nell'allegato / the abstract is in the attachmen