69 research outputs found
MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction
We introduce MIPS-Fusion, a robust and scalable online RGB-D reconstruction
method based on a novel neural implicit representation --
multi-implicit-submap. Different from existing neural RGB-D reconstruction
methods lacking either flexibility with a single neural map or scalability due
to extra storage of feature grids, we propose a pure neural representation
tackling both difficulties with a divide-and-conquer design. In our method,
neural submaps are incrementally allocated alongside the scanning trajectory
and efficiently learned with local neural bundle adjustments. The submaps can
be refined individually in a back-end optimization and optimized jointly to
realize submap-level loop closure. Meanwhile, we propose a hybrid tracking
approach combining randomized and gradient-based pose optimizations. For the
first time, randomized optimization is made possible in neural tracking with
several key designs to the learning process, enabling efficient and robust
tracking even under fast camera motions. The extensive evaluation demonstrates
that our method attains higher reconstruction quality than the state of the
arts for large-scale scenes and under fast camera motions
Panoptic Mapping with Fruit Completion and Pose Estimation for Horticultural Robots
Monitoring plants and fruits at high resolution play a key role in the future
of agriculture. Accurate 3D information can pave the way to a diverse number of
robotic applications in agriculture ranging from autonomous harvesting to
precise yield estimation. Obtaining such 3D information is non-trivial as
agricultural environments are often repetitive and cluttered, and one has to
account for the partial observability of fruit and plants. In this paper, we
address the problem of jointly estimating complete 3D shapes of fruit and their
pose in a 3D multi-resolution map built by a mobile robot. To this end, we
propose an online multi-resolution panoptic mapping system where regions of
interest are represented with a higher resolution. We exploit data to learn a
general fruit shape representation that we use at inference time together with
an occlusion-aware differentiable rendering pipeline to complete partial fruit
observations and estimate the 7 DoF pose of each fruit in the map. The
experiments presented in this paper evaluated both in the controlled
environment and in a commercial greenhouse, show that our novel algorithm
yields higher completion and pose estimation accuracy than existing methods,
with an improvement of 41% in completion accuracy and 52% in pose estimation
accuracy while keeping a low inference time of 0.6s in average. Codes are
available at: https://github.com/PRBonn/HortiMapping.Comment: 8 pages, IROS 202
A submap joining algorithm for 3D reconstruction using an RGB-D camera based on point and plane features
© 2019 Elsevier B.V. In standard point-based methods, the depth measurements of the point features suffer from noise, which will lead to incorrect global structure of the environment. This paper presents a submap joining based SLAM with an RGB-D camera by introducing planes as well as points as features.This work is consisted of two steps: submap building and submap joining. Several adjacent keyframes, with the corresponding small patches, visual feature points, and planes observed from these keyframes, are used to build a submap. We fuse the submaps into a global map in a sequential fashion, such that, the global structure is recovered gradually through plane feature associations and optimization. We also show that the proposed algorithm can handle plane association problem incrementally in submap level, as the plane covariance can be obtained in each submap. The use of submap significantly reduces the computational cost during the optimization process, while keeping all information about planes. The proposed method is validated using both publicly available RGB-D benchmarks and datasets collected by authors. The algorithm can produce accurate trajectories and high quality 3D models on these challenging datasets, which are difficult for existing RGB-D SLAM or SFM algorithms
Single-pass inline pipeline 3D reconstruction using depth camera array
A novel inline inspection (ILI) approach using depth cameras array (DCA) is introduced to create high-fidelity, dense 3D pipeline models. A new camera calibration method is introduced to register the color and the depth information of the cameras into a unified pipe model. By incorporating the calibration outcomes into a robust camera motion estimation approach, dense and complete 3D pipe surface reconstruction is achieved by using only the inline image data collected by a self-powered ILI rover in a single pass through a straight pipeline. The outcomes of the laboratory experiments demonstrate one-millimeter geometrical accuracy and 0.1-pixel photometric accuracy. In the reconstructed model of a longer pipeline, the proposed method generates the dense 3D surface reconstruction model at the millimeter level accuracy with less than 0.5% distance error. The achieved performance highlights its potential as a useful tool for efficient in-line, non-destructive evaluation of pipeline assets
Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction
State-of-the-art methods for large-scale 3D reconstruction from RGB-D sensors
usually reduce drift in camera tracking by globally optimizing the estimated
camera poses in real-time without simultaneously updating the reconstructed
surface on pose changes. We propose an efficient on-the-fly surface correction
method for globally consistent dense 3D reconstruction of large-scale scenes.
Our approach uses a dense Visual RGB-D SLAM system that estimates the camera
motion in real-time on a CPU and refines it in a global pose graph
optimization. Consecutive RGB-D frames are locally fused into keyframes, which
are incorporated into a sparse voxel hashed Signed Distance Field (SDF) on the
GPU. On pose graph updates, the SDF volume is corrected on-the-fly using a
novel keyframe re-integration strategy with reduced GPU-host streaming. We
demonstrate in an extensive quantitative evaluation that our method is up to
93% more runtime efficient compared to the state-of-the-art and requires
significantly less memory, with only negligible loss of surface quality.
Overall, our system requires only a single GPU and allows for real-time surface
correction of large environments.Comment: British Machine Vision Conference (BMVC), London, September 201
Efficient Online Surface Correction for Real-time Large-Scale 3D Reconstruction
State-of-the-art methods for large-scale 3D reconstruction from RGB-D sensors
usually reduce drift in camera tracking by globally optimizing the estimated
camera poses in real-time without simultaneously updating the reconstructed
surface on pose changes. We propose an efficient on-the-fly surface correction
method for globally consistent dense 3D reconstruction of large-scale scenes.
Our approach uses a dense Visual RGB-D SLAM system that estimates the camera
motion in real-time on a CPU and refines it in a global pose graph
optimization. Consecutive RGB-D frames are locally fused into keyframes, which
are incorporated into a sparse voxel hashed Signed Distance Field (SDF) on the
GPU. On pose graph updates, the SDF volume is corrected on-the-fly using a
novel keyframe re-integration strategy with reduced GPU-host streaming. We
demonstrate in an extensive quantitative evaluation that our method is up to
93% more runtime efficient compared to the state-of-the-art and requires
significantly less memory, with only negligible loss of surface quality.
Overall, our system requires only a single GPU and allows for real-time surface
correction of large environments.Comment: British Machine Vision Conference (BMVC), London, September 201
Nonrigid reconstruction of 3D breast surfaces with a low-cost RGBD camera for surgical planning and aesthetic evaluation
Accounting for 26% of all new cancer cases worldwide, breast cancer remains
the most common form of cancer in women. Although early breast cancer has a
favourable long-term prognosis, roughly a third of patients suffer from a
suboptimal aesthetic outcome despite breast conserving cancer treatment.
Clinical-quality 3D modelling of the breast surface therefore assumes an
increasingly important role in advancing treatment planning, prediction and
evaluation of breast cosmesis. Yet, existing 3D torso scanners are expensive
and either infrastructure-heavy or subject to motion artefacts. In this paper
we employ a single consumer-grade RGBD camera with an ICP-based registration
approach to jointly align all points from a sequence of depth images
non-rigidly. Subtle body deformation due to postural sway and respiration is
successfully mitigated leading to a higher geometric accuracy through
regularised locally affine transformations. We present results from 6 clinical
cases where our method compares well with the gold standard and outperforms a
previous approach. We show that our method produces better reconstructions
qualitatively by visual assessment and quantitatively by consistently obtaining
lower landmark error scores and yielding more accurate breast volume estimates
- …