Human-in-the-Loop Teleoperation Modes for Autonomous Unmanned Aerial Vehicles

In all future scenarios of fully autonomous ground or air vehicle networks, human intervention is expected to take place in the form of remote immediate involvement/assistance. The use of telemanipulation with various ``human-in-the-loop (HITL)'' schemes is anticipated to instil the necessary degree of trust in autonomous vehicles (AVs) while operating in a highly volatile environment with other vehicles and a multitude of obstacles. According to numerous research papers, autonomous uninhabited aerial vehicles (A-UAVs) will reach higher penetration levels in mixed air traffic in the coming years. However, there hasn't been enough research in the literature on efficient A-UAV management in real-world use cases with a lot of uncertainty. This paper attempts to bridge this gap by examining the telemanipulation schemes between two smart agents: human telemanipulators (HTMs) and A-UAVs. HITL telemanipulation described in this report can i) play a key role in enabling A-UAVs to instantly handle a multitude of uncertainties and ii) expedite the integration of A-UAVs into mixed air traffic

Technical Report: Analysis of Intervention Modes in Human-In-The-Loop (HITL) Teleoperation With Autonomous Ground Vehicle Systems

Fully autonomous systems are human-out-of-the-loop systems that single-handedly determine the right course of action when given an autonomous task. In all future visions of operating networks of fully autonomous self-driving ground or aerial vehicles, humans are expected to intervene with some kind of remote instantaneous intervention role and ``Human-on-the-Loop (HOTL)'' and ``Human-in-the-Loop (HOTL)'' telemonitoring and telemanipulation is expected to establish a desired level of trust in AVs while they are interacting with a highly dynamic urban or aerial environment. Many studies envision a future with fully autonomous self-driving vehicles (FA-SDVs) with increasing penetration levels in mixed traffic. However, effective management of FA-SDVs in real-world use cases under highly uncertain conditions has not been examined sufficiently in the literature. This report, by covering the teleoperation collaboration modes between two intelligent agents — human telesupervisors (HTSs) and FA-SDVs — aims to close this gap

Conceptualisation of human-on-the-loop haptic teleoperation with fully autonomous self-driving vehicles in the urban environment

The automotive industry aims to deploy commercial level-5 fully autonomous self-driving vehicles (FA-SDVs) in a diverse range of benefit-driven concepts on city roads in the years to come. In all future visions of operating networks of FA-SDVs, humans are expected to intervene with some kind of remote supervisory role. Recent advances in cyber-physical systems (CPS) within the concept of Internet of Everything (IoE) using tactile internet (TI) teleport us to teleoperate remote objects within the cyber-world. Human-on-the-loop (HOTL) haptic teleoperation with an extension of human control and sensing capability by coupling with artificial sensors and actuators with an increased sense of real-time driving in the remote vehicle can help overcome the challenging tasks when the new driver - artificial intelligence (AI) agent - encounters an unorthodox situation that can't be addressed by the autonomous capabilities. This paper analyses HOTL real-time haptic delay-sensitive teleoperation with FA-SDVs, in the aspects of human-vehicle teamwork by establishing two similar remote parallel worlds --- real-world vehicle time-varying environment and cyber-world emulation of this environment, i.e., digital twins (DTs) --- in which a human telesupervisor (HTS), as a biological agent, can be immersed within a reasonable timescale with no cybersickness enabling omnipresence and a bidirectional flow of energy and information. The experiments conducted as a proof of concept of HOTL haptic teleoperation shows promising results and the potential of benefiting from the proposed framework

A framework for the synergistic integration of fully autonomous ground vehicles with smart city

Most of the vehicle manufacturers aim to deploy level-5 fully autonomous ground vehicles (FAGVs) on city roads in 2021 by leveraging extensive existing knowledge about sensors, actuators, telematics and Artificial Intelligence (AI) gained from the level-3 and level-4 autonomy. FAGVs by executing non-trivial sequences of events with decimetre-level accuracy live in Smart City (SC) and their integration with all the SC components and domains using real-time data analytics is urgent to establish better swarm intelligent systems and a safer and optimised harmonious smart environment enabling cooperative FAGVs-SC automation systems. The challenges of urbanisation, if unmet urgently, would entail severe economic and environmental impacts. The integration of FAGVs with SC helps improve the sustainability of a city and the functional and efficient deployment of hand over wheels on robotized city roads with behaviour coordination. SC can enable the exploitation of the full potential of FAGVs with embedded centralised systems within SC with highly distributed systems in a concept of Automation of Everything (AoE). This paper proposes a synergistic integrated FAGV-SC holistic framework - FAGVinSCF in which all the components of SC and FAGVs involving recent and impending technological advancements are moulded to make the transformation from today's driving society to future's next-generation driverless society smoother and truly make self-driving technology a harmonious part of our cities with sustainable urban development. Based on FAGVinSCF, a simulation platform is built both to model the varying penetration levels of FAGV into mixed traffic and to perform the optimal self-driving behaviours of FAGV swarms. The results show that FAGVinSCF improves the urban traffic flow significantly without huge changes to the traffic infrastructure. With this framework, the concept of Cooperative Intelligent Transportation Systems (C-ITS) is transformed into the concept of Automated ITS (A-ITS). Cities currently designed for cars can turn into cities developed for citizens using FAGVinSCF enabling more sustainable cities

A Framework for the Synergistic Integration of Fully Autonomous Ground Vehicles With Smart City

Most of the vehicle manufacturers aim to deploy level-5 fully autonomous ground vehicles (FAGVs) on city roads in 2021 by leveraging extensive existing knowledge about sensors, actuators, telematics and Artificial Intelligence (AI) gained from the level-3 and level-4 autonomy. FAGVs by executing non-trivial sequences of events with decimetre-level accuracy live in Smart City (SC) and their integration with all the SC components and domains using real-time data analytics is urgent to establish better swarm intelligent systems and a safer and optimised harmonious smart environment enabling cooperative FAGVs-SC automation systems. The challenges of urbanisation, if unmet urgently, would entail severe economic and environmental impacts. The integration of FAGVs with SC helps improve the sustainability of a city and the functional and efficient deployment of hand over wheels on robotized city roads with behaviour coordination. SC can enable the exploitation of the full potential of FAGVs with embedded centralised systems within SC with highly distributed systems in a concept of Automation of Everything (AoE). This article proposes a synergistic integrated FAGV-SC holistic framework - FAGVinSCF in which all the components of SC and FAGVs involving recent and impending technological advancements are moulded to make the transformation from today's driving society to future's next-generation driverless society smoother and truly make self-driving technology a harmonious part of our cities with sustainable urban development. Based on FAGVinSCF, a simulation platform is built both to model the varying penetration levels of FAGV into mixed traffic and to perform the optimal self-driving behaviours of FAGV swarms. The results show that FAGVinSCF improves the urban traffic flow significantly without huge changes to the traffic infrastructure. With this framework, the concept of Cooperative Intelligent Transportation Systems (C-ITS) is transformed into the concept of Automated ITS (A-ITS). Cities currently designed for cars can turn into cities developed for citizens using FAGVinSCF enabling more sustainable cities