Single image pose estimation is a fundamental problem in many vision and
robotics tasks, and existing deep learning approaches suffer by not completely
modeling and handling: i) uncertainty about the predictions, and ii) symmetric
objects with multiple (sometimes infinite) correct poses. To this end, we
introduce a method to estimate arbitrary, non-parametric distributions on
SO(3). Our key idea is to represent the distributions implicitly, with a neural
network that estimates the probability given the input image and a candidate
pose. Grid sampling or gradient ascent can be used to find the most likely
pose, but it is also possible to evaluate the probability at any pose, enabling
reasoning about symmetries and uncertainty. This is the most general way of
representing distributions on manifolds, and to showcase the rich expressive
power, we introduce a dataset of challenging symmetric and nearly-symmetric
objects. We require no supervision on pose uncertainty -- the model trains only
with a single pose per example. Nonetheless, our implicit model is highly
expressive to handle complex distributions over 3D poses, while still obtaining
accurate pose estimation on standard non-ambiguous environments, achieving
state-of-the-art performance on Pascal3D+ and ModelNet10-SO(3) benchmarks