799 research outputs found
Reliable fusion of ToF and stereo depth driven by confidence measures
In this paper we propose a framework for the fusion of depth data produced by a Time-of-Flight (ToF) camera and stereo vision system. Initially, depth data acquired by the ToF camera are upsampled by an ad-hoc algorithm based on image segmentation and bilateral filtering. In parallel a dense disparity map is obtained using the Semi- Global Matching stereo algorithm. Reliable confidence measures are extracted for both the ToF and stereo depth data. In particular, ToF confidence also accounts for the mixed-pixel effect and the stereo confidence accounts for the relationship between the pointwise matching costs and the cost obtained by the semi-global optimization. Finally, the two depth maps are synergically fused by enforcing the local consistency of depth data accounting for the confidence of the two data sources at each location. Experimental results clearly show that the proposed method produces accurate high resolution depth maps and outperforms the compared fusion algorithms
Computer vision and optimization methods applied to the measurements of in-plane deformations
fi=vertaisarvioitu|en=peerReviewed
ReS²tAC—UAV-borne real-time SGM stereo optimized for embedded ARM and CUDA devices
With the emergence of low-cost robotic systems, such as unmanned aerial vehicle, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the recently increasing availability of embedded GPU-based systems, such as the NVIDIA Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for massively parallel embedded computing on graphics hardware. With this in mind, we propose an approach for real-time embedded stereo processing on ARM and CUDA-enabled devices, which is based on the popular and widely used Semi-Global Matching algorithm. In this, we propose an optimization of the algorithm for embedded CUDA GPUs, by using massively parallel computing, as well as using the NEON intrinsics to optimize the algorithm for vectorized SIMD processing on embedded ARM CPUs. We have evaluated our approach with different configurations on two public stereo benchmark datasets to demonstrate that they can reach an error rate as low as 3.3%. Furthermore, our experiments show that the fastest configuration of our approach reaches up to 46 FPS on VGA image resolution. Finally, in a use-case specific qualitative evaluation, we have evaluated the power consumption of our approach and deployed it on the DJI Manifold 2-G attached to a DJI Matrix 210v2 RTK unmanned aerial vehicle (UAV), demonstrating its suitability for real-time stereo processing onboard a UAV
ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices
With the emergence of low-cost robotic systems, such as unmanned aerial
vehicle, the importance of embedded high-performance image processing has
increased. For a long time, FPGAs were the only processing hardware that were
capable of high-performance computing, while at the same time preserving a low
power consumption, essential for embedded systems. However, the recently
increasing availability of embedded GPU-based systems, such as the NVIDIA
Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for
massively parallel embedded computing on graphics hardware. With this in mind,
we propose an approach for real-time embedded stereo processing on ARM and
CUDA-enabled devices, which is based on the popular and widely used Semi-Global
Matching algorithm. In this, we propose an optimization of the algorithm for
embedded CUDA GPUs, by using massively parallel computing, as well as using the
NEON intrinsics to optimize the algorithm for vectorized SIMD processing on
embedded ARM CPUs. We have evaluated our approach with different configurations
on two public stereo benchmark datasets to demonstrate that they can reach an
error rate as low as 3.3%. Furthermore, our experiments show that the fastest
configuration of our approach reaches up to 46 FPS on VGA image resolution.
Finally, in a use-case specific qualitative evaluation, we have evaluated the
power consumption of our approach and deployed it on the DJI Manifold 2-G
attached to a DJI Matrix 210v2 RTK unmanned aerial vehicle (UAV), demonstrating
its suitability for real-time stereo processing onboard a UAV
Road surface 3D reconstruction based on dense subpixel disparity map estimation
Various 3D reconstruction methods have enabled civil engineers to detect
damage on a road surface. To achieve the millimetre accuracy required for road
condition assessment, a disparity map with subpixel resolution needs to be
used. However, none of the existing stereo matching algorithms are specially
suitable for the reconstruction of the road surface. Hence in this paper, we
propose a novel dense subpixel disparity estimation algorithm with high
computational efficiency and robustness. This is achieved by first transforming
the perspective view of the target frame into the reference view, which not
only increases the accuracy of the block matching for the road surface but also
improves the processing speed. The disparities are then estimated iteratively
using our previously published algorithm where the search range is propagated
from three estimated neighbouring disparities. Since the search range is
obtained from the previous iteration, errors may occur when the propagated
search range is not sufficient. Therefore, a correlation maxima verification is
performed to rectify this issue, and the subpixel resolution is achieved by
conducting a parabola interpolation enhancement. Furthermore, a novel disparity
global refinement approach developed from the Markov Random Fields and Fast
Bilateral Stereo is introduced to further improve the accuracy of the estimated
disparity map, where disparities are updated iteratively by minimising the
energy function that is related to their interpolated correlation polynomials.
The algorithm is implemented in C language with a near real-time performance.
The experimental results illustrate that the absolute error of the
reconstruction varies from 0.1 mm to 3 mm.Comment: 11 pages, 16 figures, IEEE Transactions on Image Processin
HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
This paper presents HITNet, a novel neural network architecture for real-time
stereo matching. Contrary to many recent neural network approaches that operate
on a full cost volume and rely on 3D convolutions, our approach does not
explicitly build a volume and instead relies on a fast multi-resolution
initialization step, differentiable 2D geometric propagation and warping
mechanisms to infer disparity hypotheses. To achieve a high level of accuracy,
our network not only geometrically reasons about disparities but also infers
slanted plane hypotheses allowing to more accurately perform geometric warping
and upsampling operations. Our architecture is inherently multi-resolution
allowing the propagation of information across different levels. Multiple
experiments prove the effectiveness of the proposed approach at a fraction of
the computation required by state-of-the-art methods. At the time of writing,
HITNet ranks 1st-3rd on all the metrics published on the ETH3D website for two
view stereo, ranks 1st on most of the metrics among all the end-to-end learning
approaches on Middlebury-v3, ranks 1st on the popular KITTI 2012 and 2015
benchmarks among the published methods faster than 100ms.Comment: The pretrained models used for submission to benchmarks and sample
evaluation scripts can be found at
https://github.com/google-research/google-research/tree/master/hitne
- …