3 research outputs found
Interactive Annotation of 3D Object Geometry using 2D Scribbles
Inferring detailed 3D geometry of the scene is crucial for robotics
applications, simulation, and 3D content creation. However, such information is
hard to obtain, and thus very few datasets support it. In this paper, we
propose an interactive framework for annotating 3D object geometry from both
point cloud data and RGB imagery. The key idea behind our approach is to
exploit strong priors that humans have about the 3D world in order to
interactively annotate complete 3D shapes. Our framework targets naive users
without artistic or graphics expertise. We introduce two simple-to-use
interaction modules. First, we make an automatic guess of the 3D shape and
allow the user to provide feedback about large errors by drawing scribbles in
desired 2D views. Next, we aim to correct minor errors, in which users drag and
drop mesh vertices, assisted by a neural interactive module implemented as a
Graph Convolutional Network. Experimentally, we show that only a few user
interactions are needed to produce good quality 3D shapes on popular benchmarks
such as ShapeNet, Pix3D and ScanNet. We implement our framework as a web
service and conduct a user study, where we show that user annotated data using
our method effectively facilitates real-world learning tasks. Web service:
http://www.cs.toronto.edu/~shenti11/scribble3d.Comment: Accepted to ECCV 202
Extracting Triangular 3D Models, Materials, and Lighting From Images
We present an efficient method for joint optimization of topology, materials
and lighting from multi-view image observations. Unlike recent multi-view
reconstruction approaches, which typically produce entangled 3D representations
encoded in neural networks, we output triangle meshes with spatially-varying
materials and environment lighting that can be deployed in any traditional
graphics engine unmodified. We leverage recent work in differentiable
rendering, coordinate-based networks to compactly represent volumetric
texturing, alongside differentiable marching tetrahedrons to enable
gradient-based optimization directly on the surface mesh. Finally, we introduce
a differentiable formulation of the split sum approximation of environment
lighting to efficiently recover all-frequency lighting. Experiments show our
extracted models used in advanced scene editing, material decomposition, and
high quality view interpolation, all running at interactive rates in
triangle-based renderers (rasterizers and path tracers). Project website:
https://nvlabs.github.io/nvdiffrec/ .Comment: Project website: https://nvlabs.github.io/nvdiffrec
Zeus: A System Description of the Two-Time Winner of the Collegiate SAE AutoDrive Competition
The SAE AutoDrive Challenge is a three-year collegiate competition to develop
a self-driving car by 2020. The second year of the competition was held in June
2019 at MCity, a mock town built for self-driving car testing at the University
of Michigan. Teams were required to autonomously navigate a series of
intersections while handling pedestrians, traffic lights, and traffic signs.
Zeus is aUToronto's winning entry in the AutoDrive Challenge. This article
describes the system design and development of Zeus as well as many of the
lessons learned along the way. This includes details on the team's
organizational structure, sensor suite, software components, and performance at
the Year 2 competition. With a team of mostly undergraduates and minimal
resources, aUToronto has made progress towards a functioning self-driving
vehicle, in just two years. This article may prove valuable to researchers
looking to develop their own self-driving platform.Comment: Submitted to the Journal of Field Robotic