1,321 research outputs found
Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer
Semantic annotations are vital for training models for object recognition,
semantic segmentation or scene understanding. Unfortunately, pixelwise
annotation of images at very large scale is labor-intensive and only little
labeled data is available, particularly at instance level and for street
scenes. In this paper, we propose to tackle this problem by lifting the
semantic instance labeling task from 2D into 3D. Given reconstructions from
stereo or laser data, we annotate static 3D scene elements with rough bounding
primitives and develop a model which transfers this information into the image
domain. We leverage our method to obtain 2D labels for a novel suburban video
dataset which we have collected, resulting in 400k semantic and instance image
annotations. A comparison of our method to state-of-the-art label transfer
baselines reveals that 3D information enables more efficient annotation while
at the same time resulting in improved accuracy and time-coherent labels.Comment: 10 pages in Conference on Computer Vision and Pattern Recognition
(CVPR), 201
Recommended from our members
Particle-Based Sampling and Meshing of Surfaces in Multimaterial Volumes
Methods that faithfully and robustly capture the geometry of complex material interfaces in labeled volume data are important for generating realistic and accurate visualizations and simulations of real-world objects. The generation of such multimaterial models from measured data poses two unique challenges: first, the surfaces must be well-sampled with regular, efficient tessellations that are consistent across material boundaries; and second, the resulting meshes must respect the nonmanifold geometry of the multimaterial interfaces. This paper proposes a strategy for sampling and meshing multimaterial volumes using dynamic particle systems, including a novel, differentiable representation of the material junctions that allows the particle system to explicitly sample corners, edges, and surfaces of material intersections. The distributions of particles are controlled by fundamental sampling constraints, allowing Delaunay-based meshing algorithms to reliably extract watertight meshes of consistently high-quality.Engineering and Applied Science
Semantic 3D Reconstruction with Finite Element Bases
We propose a novel framework for the discretisation of multi-label problems
on arbitrary, continuous domains. Our work bridges the gap between general FEM
discretisations, and labeling problems that arise in a variety of computer
vision tasks, including for instance those derived from the generalised Potts
model. Starting from the popular formulation of labeling as a convex relaxation
by functional lifting, we show that FEM discretisation is valid for the most
general case, where the regulariser is anisotropic and non-metric. While our
findings are generic and applicable to different vision problems, we
demonstrate their practical implementation in the context of semantic 3D
reconstruction, where such regularisers have proved particularly beneficial.
The proposed FEM approach leads to a smaller memory footprint as well as faster
computation, and it constitutes a very simple way to enable variable, adaptive
resolution within the same model
Doctor of Philosophy
dissertationOne of the fundamental building blocks of many computational sciences is the construction and use of a discretized, geometric representation of a problem domain, often referred to as a mesh. Such a discretization enables an otherwise complex domain to be represented simply, and computation to be performed over that domain with a finite number of basis elements. As mesh generation techniques have become more sophisticated over the years, focus has largely shifted to quality mesh generation techniques that guarantee or empirically generate numerically well-behaved elements. In this dissertation, the two complementary meshing subproblems of vertex placement and element creation are analyzed, both separately and together. First, a dynamic particle system achieves adaptivity over domains by inferring feature size through a new information passing algorithm. Second, a new tetrahedral algorithm is constructed that carefully combines lattice-based stenciling and mesh warping to produce guaranteed quality meshes on multimaterial volumetric domains. Finally, the ideas of lattice cleaving and dynamic particle systems are merged into a unified framework for producing guaranteed quality, unstructured and adaptive meshing of multimaterial volumetric domains
Multi-Material Mesh Representation of Anatomical Structures for Deep Brain Stimulation Planning
The Dual Contouring algorithm (DC) is a grid-based process used to generate surface meshes from volumetric data. However, DC is unable to guarantee 2-manifold and watertight meshes due to the fact that it produces only one vertex for each grid cube. We present a modified Dual Contouring algorithm that is capable of overcoming this limitation. The proposed method decomposes an ambiguous grid cube into a set of tetrahedral cells and uses novel polygon generation rules that produce 2-manifold and watertight surface meshes with good-quality triangles. These meshes, being watertight and 2-manifold, are geometrically correct, and therefore can be used to initialize tetrahedral meshes.
The 2-manifold DC method has been extended into the multi-material domain. Due to its multi-material nature, multi-material surface meshes will contain non-manifold elements along material interfaces or shared boundaries. The proposed multi-material DC algorithm can (1) generate multi-material surface meshes where each material sub-mesh is a 2-manifold and watertight mesh, (2) preserve the non-manifold elements along the material interfaces, and (3) ensure that the material interface or shared boundary between materials is consistent. The proposed method is used to generate multi-material surface meshes of deep brain anatomical structures from a digital atlas of the basal ganglia and thalamus. Although deep brain anatomical structures can be labeled as functionally separate, they are in fact continuous tracts of soft tissue in close proximity to each other. The multi-material meshes generated by the proposed DC algorithm can accurately represent the closely-packed deep brain structures as a single mesh consisting of multiple material sub-meshes. Each sub-mesh represents a distinct functional structure of the brain.
Printed and/or digital atlases are important tools for medical research and surgical intervention. While these atlases can provide guidance in identifying anatomical structures, they do not take into account the wide variations in the shape and size of anatomical structures that occur from patient to patient. Accurate, patient-specific representations are especially important for surgical interventions like deep brain stimulation, where even small inaccuracies can result in dangerous complications. The last part of this research effort extends the discrete deformable 2-simplex mesh into the multi-material domain where geometry-based internal forces and image-based external forces are used in the deformation process. This multi-material deformable framework is used to segment anatomical structures of the deep brain region from Magnetic Resonance (MR) data
Building with Drones: Accurate 3D Facade Reconstruction using MAVs
Automatic reconstruction of 3D models from images using multi-view
Structure-from-Motion methods has been one of the most fruitful outcomes of
computer vision. These advances combined with the growing popularity of Micro
Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools
ubiquitous for large number of Architecture, Engineering and Construction
applications among audiences, mostly unskilled in computer vision. However, to
obtain high-resolution and accurate reconstructions from a large-scale object
using SfM, there are many critical constraints on the quality of image data,
which often become sources of inaccuracy as the current 3D reconstruction
pipelines do not facilitate the users to determine the fidelity of input data
during the image acquisition. In this paper, we present and advocate a
closed-loop interactive approach that performs incremental reconstruction in
real-time and gives users an online feedback about the quality parameters like
Ground Sampling Distance (GSD), image redundancy, etc on a surface mesh. We
also propose a novel multi-scale camera network design to prevent scene drift
caused by incremental map building, and release the first multi-scale image
sequence dataset as a benchmark. Further, we evaluate our system on real
outdoor scenes, and show that our interactive pipeline combined with a
multi-scale camera network approach provides compelling accuracy in multi-view
reconstruction tasks when compared against the state-of-the-art methods.Comment: 8 Pages, 2015 IEEE International Conference on Robotics and
Automation (ICRA '15), Seattle, WA, US
Tetrahedral Image-to-Mesh Conversion Software for Anatomic Modeling of Arteriovenous Malformations
We describe a new implementation of an adaptive multi-tissue tetrahedral mesh generator targeting anatomic modeling of Arteriovenous Malformation (AVM) for surgical simulations. Our method, initially constructs an adaptive Body-Centered Cubic (BCC) mesh of high quality elements. Then, it deforms the mesh surfaces to their corresponding physical image boundaries, hence, improving the mesh fidelity and smoothness. Our deformation scheme, which builds upon the ITK toolkit, is based on the concept of energy minimization, and relies on a multi-material point-based registration. It uses non-connectivity patterns to implicitly control the number of the extracted feature points needed for the registration, and thus, adjusts the trade-off between the achieved mesh fidelity and the deformation speed. While many medical imaging applications require robust mesh generation, there are few codes available to the public. We compare our implementation with two similar open-source image-to-mesh conversion codes: (1) Cleaver from US, and (2) CGAL from EU. Our evaluation is based on five isotropic/anisotropic segmented images, and relies on metrics like geometric & topologic fidelity, mesh quality, gradation and smoothness. The implementation we describe is open- source and it will be available within: (i) the 3D Slicer package for visualization and image analysis from Harvard Medical School, and (ii) an interactive simulator for neurosurgical procedures involving vasculature using SOFA, a framework for real-time medical simulation developed by INRIA
- …