5,811 research outputs found
On Offline Evaluation of Vision-based Driving Models
Autonomous driving models should ideally be evaluated by deploying them on a
fleet of physical vehicles in the real world. Unfortunately, this approach is
not practical for the vast majority of researchers. An attractive alternative
is to evaluate models offline, on a pre-collected validation dataset with
ground truth annotation. In this paper, we investigate the relation between
various online and offline metrics for evaluation of autonomous driving models.
We find that offline prediction error is not necessarily correlated with
driving quality, and two models with identical prediction error can differ
dramatically in their driving performance. We show that the correlation of
offline evaluation with driving quality can be significantly improved by
selecting an appropriate validation dataset and suitable offline metrics. The
supplementary video can be viewed at
https://www.youtube.com/watch?v=P8K8Z-iF0cYComment: Published at the ECCV 2018 conferenc
Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions
We consider the paradigm of a black box AI system that makes life-critical
decisions. We propose an "arguing machines" framework that pairs the primary AI
system with a secondary one that is independently trained to perform the same
task. We show that disagreement between the two systems, without any knowledge
of underlying system design or operation, is sufficient to arbitrarily improve
the accuracy of the overall decision pipeline given human supervision over
disagreements. We demonstrate this system in two applications: (1) an
illustrative example of image classification and (2) on large-scale real-world
semi-autonomous driving data. For the first application, we apply this
framework to image classification achieving a reduction from 8.0% to 2.8% top-5
error on ImageNet. For the second application, we apply this framework to Tesla
Autopilot and demonstrate the ability to predict 90.4% of system disengagements
that were labeled by human annotators as challenging and needing human
supervision
WiseMove: A Framework for Safe Deep Reinforcement Learning for Autonomous Driving
Machine learning can provide efficient solutions to the complex problems
encountered in autonomous driving, but ensuring their safety remains a
challenge. A number of authors have attempted to address this issue, but there
are few publicly-available tools to adequately explore the trade-offs between
functionality, scalability, and safety.
We thus present WiseMove, a software framework to investigate safe deep
reinforcement learning in the context of motion planning for autonomous
driving. WiseMove adopts a modular learning architecture that suits our current
research questions and can be adapted to new technologies and new questions. We
present the details of WiseMove, demonstrate its use on a common traffic
scenario, and describe how we use it in our ongoing safe learning research
Evolving controllers for simulated car racing
This paper describes the evolution of controllers for racing a simulated radio-controlled car around a track, modelled on a real physical track. Five different controller architectures were compared, based on neural networks, force fields and action sequences. The controllers use either egocentric (first person), Newtonian (third person) or no information about the state of the car (open-loop controller). The only controller that is able to evolve good racing behaviour is based on a neural network acting on egocentric inputs
- …