2,930 research outputs found
Zero-shot Deep Reinforcement Learning Driving Policy Transfer for Autonomous Vehicles based on Robust Control
Although deep reinforcement learning (deep RL) methods have lots of strengths
that are favorable if applied to autonomous driving, real deep RL applications
in autonomous driving have been slowed down by the modeling gap between the
source (training) domain and the target (deployment) domain. Unlike current
policy transfer approaches, which generally limit to the usage of
uninterpretable neural network representations as the transferred features, we
propose to transfer concrete kinematic quantities in autonomous driving. The
proposed robust-control-based (RC) generic transfer architecture, which we call
RL-RC, incorporates a transferable hierarchical RL trajectory planner and a
robust tracking controller based on disturbance observer (DOB). The deep RL
policies trained with known nominal dynamics model are transfered directly to
the target domain, DOB-based robust tracking control is applied to tackle the
modeling gap including the vehicle dynamics errors and the external
disturbances such as side forces. We provide simulations validating the
capability of the proposed method to achieve zero-shot transfer across multiple
driving scenarios such as lane keeping, lane changing and obstacle avoidance.Comment: Published at IEEE ITSC 201
Exploration Without Maps via Zero-Shot Out-of-Distribution Deep Reinforcement Learning
Operation of Autonomous Mobile Robots (AMRs) of all forms that include
wheeled ground vehicles, quadrupeds and humanoids in dynamically changing GPS
denied environments without a-priori maps, exclusively using onboard sensors,
is an unsolved problem that has potential to transform the economy, and vastly
improve humanity's capabilities with improvements to agriculture,
manufacturing, disaster response, military and space exploration. Conventional
AMR automation approaches are modularized into perception, motion planning and
control which is computationally inefficient, and requires explicit feature
extraction and engineering, that inhibits generalization, and deployment at
scale. Few works have focused on real-world end-to-end approaches that directly
map sensor inputs to control outputs due to the large amount of well curated
training data required for supervised Deep Learning (DL) which is time
consuming and labor intensive to collect and label, and sample inefficiency and
challenges to bridging the simulation to reality gap using Deep Reinforcement
Learning (DRL). This paper presents a novel method to efficiently train DRL for
robust end-to-end AMR exploration, in a constrained environment at physical
limits in simulation, transferred zero-shot to the real-world. The
representation learned in a compact parameter space with 2 fully connected
layers with 64 nodes each is demonstrated to exhibit emergent behavior for
out-of-distribution generalization to navigation in new environments that
include unstructured terrain without maps, and dynamic obstacle avoidance. The
learned policy outperforms conventional navigation algorithms while consuming a
fraction of the computation resources, enabling execution on a range of AMR
forms with varying embedded computer payloads
Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin
Reinforcement learning (RL) is a promising solution for autonomous vehicles
to deal with complex and uncertain traffic environments. The RL training
process is however expensive, unsafe, and time consuming. Algorithms are often
developed first in simulation and then transferred to the real world, leading
to a common sim2real challenge that performance decreases when the domain
changes. In this paper, we propose a transfer learning process to minimize the
gap by exploiting digital twin technology, relying on a systematic and
simultaneous combination of virtual and real world data coming from vehicle
dynamics and traffic scenarios. The model and testing environment are evolved
from model, hardware to vehicle in the loop and proving ground testing stages,
similar to standard development cycle in automotive industry. In particular, we
also integrate other transfer learning techniques such as domain randomization
and adaptation in each stage. The simulation and real data are gradually
incorporated to accelerate and make the transfer learning process more robust.
The proposed RL methodology is applied to develop a path following steering
controller for an autonomous electric vehicle. After learning and deploying the
real-time RL control policy on the vehicle, we obtained satisfactory and safe
control performance already from the first deployment, demonstrating the
advantages of the proposed digital twin based learning process.Comment: This work has been submitted to IFAC for possible publicatio
Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning
In this article, we demonstrate a zero-shot transfer of an autonomous driving
policy from simulation to University of Delaware's scaled smart city with
adversarial multi-agent reinforcement learning, in which an adversary attempts
to decrease the net reward by perturbing both the inputs and outputs of the
autonomous vehicles during training. We train the autonomous vehicles to
coordinate with each other while crossing a roundabout in the presence of an
adversary in simulation. The adversarial policy successfully reproduces the
simulated behavior and incidentally outperforms, in terms of travel time, both
a human-driving baseline and adversary-free trained policies. Finally, we
demonstrate that the addition of adversarial training considerably improves the
performance \eat{stability and robustness} of the policies after transfer to
the real world compared to Gaussian noise injection.Comment: 6 pages, 4 figure
- …