32 research outputs found
Mono3D++: Monocular 3D Vehicle Detection with Two-Scale 3D Hypotheses and Task Priors
We present a method to infer 3D pose and shape of vehicles from a single
image. To tackle this ill-posed problem, we optimize two-scale projection
consistency between the generated 3D hypotheses and their 2D
pseudo-measurements. Specifically, we use a morphable wireframe model to
generate a fine-scaled representation of vehicle shape and pose. To reduce its
sensitivity to 2D landmarks, we jointly model the 3D bounding box as a coarse
representation which improves robustness. We also integrate three task priors,
including unsupervised monocular depth, a ground plane constraint as well as
vehicle shape priors, with forward projection errors into an overall energy
function.Comment: Proc. of the AAAI, September 201
Towards Generalization Across Depth for Monocular 3D Object Detection
While expensive LiDAR and stereo camera rigs have enabled the development of
successful 3D object detection methods, monocular RGB-only approaches lag much
behind. This work advances the state of the art by introducing MoVi-3D, a
novel, single-stage deep architecture for monocular 3D object detection.
MoVi-3D builds upon a novel approach which leverages geometrical information to
generate, both at training and test time, virtual views where the object
appearance is normalized with respect to distance. These virtually generated
views facilitate the detection task as they significantly reduce the visual
appearance variability associated to objects placed at different distances from
the camera. As a consequence, the deep model is relieved from learning
depth-specific representations and its complexity can be significantly reduced.
In particular, in this work we show that, thanks to our virtual views
generation process, a lightweight, single-stage architecture suffices to set
new state-of-the-art results on the popular KITTI3D benchmark
Polygon Intersection-over-Union Loss for Viewpoint-Agnostic Monocular 3D Vehicle Detection
Monocular 3D object detection is a challenging task because depth information
is difficult to obtain from 2D images. A subset of viewpoint-agnostic monocular
3D detection methods also do not explicitly leverage scene homography or
geometry during training, meaning that a model trained thusly can detect
objects in images from arbitrary viewpoints. Such works predict the projections
of the 3D bounding boxes on the image plane to estimate the location of the 3D
boxes, but these projections are not rectangular so the calculation of IoU
between these projected polygons is not straightforward. This work proposes an
efficient, fully differentiable algorithm for the calculation of IoU between
two convex polygons, which can be utilized to compute the IoU between two 3D
bounding box footprints viewed from an arbitrary angle. We test the performance
of the proposed polygon IoU loss (PIoU loss) on three state-of-the-art
viewpoint-agnostic 3D detection models. Experiments demonstrate that the
proposed PIoU loss converges faster than L1 loss and that in 3D detection
models, a combination of PIoU loss and L1 loss gives better results than L1
loss alone (+1.64% AP70 for MonoCon on cars, +0.18% AP70 for RTM3D on cars, and
+0.83%/+2.46% AP50/AP25 for MonoRCNN on cyclists)