11 research outputs found
A General Framework for Uncertainty Estimation in Deep Learning
Neural networks predictions are unreliable when the input sample is out of
the training distribution or corrupted by noise. Being able to detect such
failures automatically is fundamental to integrate deep learning algorithms
into robotics. Current approaches for uncertainty estimation of neural networks
require changes to the network and optimization process, typically ignore prior
knowledge about the data, and tend to make over-simplifying assumptions which
underestimate uncertainty. To address these limitations, we propose a novel
framework for uncertainty estimation. Based on Bayesian belief networks and
Monte-Carlo sampling, our framework not only fully models the different sources
of prediction uncertainty, but also incorporates prior data information, e.g.
sensor noise. We show theoretically that this gives us the ability to capture
uncertainty better than existing methods. In addition, our framework has
several desirable properties: (i) it is agnostic to the network architecture
and task; (ii) it does not require changes in the optimization process; (iii)
it can be applied to already trained architectures. We thoroughly validate the
proposed framework through extensive experiments on both computer vision and
control tasks, where we outperform previous methods by up to 23% in accuracy.Comment: Accepted for publication in the Robotics and Automation Letters 2020,
and for presentation at the International Conference on Robotics and
Automation (ICRA) 202
Lightweight Probabilistic Deep Networks
Even though probabilistic treatments of neural networks have a long history,
they have not found widespread use in practice. Sampling approaches are often
too slow already for simple networks. The size of the inputs and the depth of
typical CNN architectures in computer vision only compound this problem.
Uncertainty in neural networks has thus been largely ignored in practice,
despite the fact that it may provide important information about the
reliability of predictions and the inner workings of the network. In this
paper, we introduce two lightweight approaches to making supervised learning
with probabilistic deep networks practical: First, we suggest probabilistic
output layers for classification and regression that require only minimal
changes to existing networks. Second, we employ assumed density filtering and
show that activation uncertainties can be propagated in a practical fashion
through the entire network, again with minor changes. Both probabilistic
networks retain the predictive power of the deterministic counterpart, but
yield uncertainties that correlate well with the empirical error induced by
their predictions. Moreover, the robustness to adversarial examples is
significantly increased.Comment: To appear at CVPR 201