526 research outputs found
TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation
Effective use of camera-based vision systems is essential for robust
performance in autonomous off-road driving, particularly in the high-speed
regime. Despite success in structured, on-road settings, current end-to-end
approaches for scene prediction have yet to be successfully adapted for complex
outdoor terrain. To this end, we present TerrainNet, a vision-based terrain
perception system for semantic and geometric terrain prediction for aggressive,
off-road navigation. The approach relies on several key insights and practical
considerations for achieving reliable terrain modeling. The network includes a
multi-headed output representation to capture fine- and coarse-grained terrain
features necessary for estimating traversability. Accurate depth estimation is
achieved using self-supervised depth completion with multi-view RGB and stereo
inputs. Requirements for real-time performance and fast inference speeds are
met using efficient, learned image feature projections. Furthermore, the model
is trained on a large-scale, real-world off-road dataset collected across a
variety of diverse outdoor environments. We show how TerrainNet can also be
used for costmap prediction and provide a detailed framework for integration
into a planning module. We demonstrate the performance of TerrainNet through
extensive comparison to current state-of-the-art baselines for camera-only
scene prediction. Finally, we showcase the effectiveness of integrating
TerrainNet within a complete autonomous-driving stack by conducting a
real-world vehicle test in a challenging off-road scenario
Occupancy Anticipation for Efficient Exploration and Navigation
State-of-the-art navigation methods leverage a spatial memory to generalize
to new environments, but their occupancy maps are limited to capturing the
geometric structures directly observed by the agent. We propose occupancy
anticipation, where the agent uses its egocentric RGB-D observations to infer
the occupancy state beyond the visible regions. In doing so, the agent builds
its spatial awareness more rapidly, which facilitates efficient exploration and
navigation in 3D environments. By exploiting context in both the egocentric
views and top-down maps our model successfully anticipates a broader map of the
environment, with performance significantly better than strong baselines.
Furthermore, when deployed for the sequential decision-making tasks of
exploration and navigation, our model outperforms state-of-the-art methods on
the Gibson and Matterport3D datasets. Our approach is the winning entry in the
2020 Habitat PointNav Challenge. Project page:
http://vision.cs.utexas.edu/projects/occupancy_anticipation/Comment: Accepted in ECCV 2020. 19 pages, 6 figures, appendix at en
- …