24 research outputs found
Visual-Inertial Mapping with Non-Linear Factor Recovery
Cameras and inertial measurement units are complementary sensors for
ego-motion estimation and environment mapping. Their combination makes
visual-inertial odometry (VIO) systems more accurate and robust. For globally
consistent mapping, however, combining visual and inertial information is not
straightforward. To estimate the motion and geometry with a set of images large
baselines are required. Because of that, most systems operate on keyframes that
have large time intervals between each other. Inertial data on the other hand
quickly degrades with the duration of the intervals and after several seconds
of integration, it typically contains only little useful information.
In this paper, we propose to extract relevant information for visual-inertial
mapping from visual-inertial odometry using non-linear factor recovery. We
reconstruct a set of non-linear factors that make an optimal approximation of
the information on the trajectory accumulated by VIO. To obtain a globally
consistent map we combine these factors with loop-closing constraints using
bundle adjustment. The VIO factors make the roll and pitch angles of the global
map observable, and improve the robustness and the accuracy of the mapping. In
experiments on a public benchmark, we demonstrate superior performance of our
method over the state-of-the-art approaches
Visual-Inertial and Leg Odometry Fusion for Dynamic Locomotion
Implementing dynamic locomotion behaviors on legged robots requires a
high-quality state estimation module. Especially when the motion includes
flight phases, state-of-the-art approaches fail to produce reliable estimation
of the robot posture, in particular base height. In this paper, we propose a
novel approach for combining visual-inertial odometry (VIO) with leg odometry
in an extended Kalman filter (EKF) based state estimator. The VIO module uses a
stereo camera and IMU to yield low-drift 3D position and yaw orientation and
drift-free pitch and roll orientation of the robot base link in the inertial
frame. However, these values have a considerable amount of latency due to image
processing and optimization, while the rate of update is quite low which is not
suitable for low-level control. To reduce the latency, we predict the VIO state
estimate at the rate of the IMU measurements of the VIO sensor. The EKF module
uses the base pose and linear velocity predicted by VIO, fuses them further
with a second high-rate IMU and leg odometry measurements, and produces robot
state estimates with a high frequency and small latency suitable for control.
We integrate this lightweight estimation framework with a nonlinear model
predictive controller and show successful implementation of a set of agile
locomotion behaviors, including trotting and jumping at varying horizontal
speeds, on a torque-controlled quadruped robot.Comment: Submitted to IEEE International Conference on Robotics and Automation
(ICRA), 202
GPS-VIO Fusion with Online Rotational Calibration
Accurate global localization is crucial for autonomous navigation and
planning. To this end, various GPS-aided Visual-Inertial Odometry (GPS-VIO)
fusion algorithms are proposed in the literature. This paper presents a novel
GPS-VIO system that is able to significantly benefit from the online
calibration of the rotational extrinsic parameter between the GPS reference
frame and the VIO reference frame. The behind reason is this parameter is
observable. This paper provides novel proof through nonlinear observability
analysis. We also evaluate the proposed algorithm extensively on diverse
platforms, including flying UAV and driving vehicle. The experimental results
support the observability analysis and show increased localization accuracy in
comparison to state-of-the-art (SOTA) tightly-coupled algorithms.Comment: Accepted by ICRA 202
BAMF-SLAM: Bundle Adjusted Multi-Fisheye Visual-Inertial SLAM Using Recurrent Field Transforms
In this paper, we present BAMF-SLAM, a novel multi-fisheye visual-inertial
SLAM system that utilizes Bundle Adjustment (BA) and recurrent field transforms
(RFT) to achieve accurate and robust state estimation in challenging scenarios.
First, our system directly operates on raw fisheye images, enabling us to fully
exploit the wide Field-of-View (FoV) of fisheye cameras. Second, to overcome
the low-texture challenge, we explore the tightly-coupled integration of
multi-camera inputs and complementary inertial measurements via a unified
factor graph and jointly optimize the poses and dense depth maps. Third, for
global consistency, the wide FoV of the fisheye camera allows the system to
find more potential loop closures, and powered by the broad convergence basin
of RFT, our system can perform very wide baseline loop closing with little
overlap. Furthermore, we introduce a semi-pose-graph BA method to avoid the
expensive full global BA. By combining relative pose factors with loop closure
factors, the global states can be adjusted efficiently with modest memory
footprint while maintaining high accuracy. Evaluations on TUM-VI, Hilti-Oxford
and Newer College datasets show the superior performance of the proposed system
over prior works. In the Hilti SLAM Challenge 2022, our VIO version achieves
second place. In a subsequent submission, our complete system, including the
global BA backend, outperforms the winning approach.Comment: Accepted to ICRA202
MAVIS: Multi-Camera Augmented Visual-Inertial SLAM using SE2(3) Based Exact IMU Pre-integration
We present a novel optimization-based Visual-Inertial SLAM system designed
for multiple partially overlapped camera systems, named MAVIS. Our framework
fully exploits the benefits of wide field-of-view from multi-camera systems,
and the metric scale measurements provided by an inertial measurement unit
(IMU). We introduce an improved IMU pre-integration formulation based on the
exponential function of an automorphism of SE_2(3), which can effectively
enhance tracking performance under fast rotational motion and extended
integration time. Furthermore, we extend conventional front-end tracking and
back-end optimization module designed for monocular or stereo setup towards
multi-camera systems, and introduce implementation details that contribute to
the performance of our system in challenging scenarios. The practical validity
of our approach is supported by our experiments on public datasets. Our MAVIS
won the first place in all the vision-IMU tracks (single and multi-session
SLAM) on Hilti SLAM Challenge 2023 with 1.7 times the score compared to the
second place.Comment: video link: https://youtu.be/Q_jZSjhNFf
CARLA-Loc: Synthetic SLAM Dataset with Full-stack Sensor Setup in Challenging Weather and Dynamic Environments
The robustness of SLAM algorithms in challenging environmental conditions is
crucial for autonomous driving, but the impact of these conditions are unknown
while given the difficulty of arbitrarily changing the relevant environmental
parameters of the same environment in the real world. Therefore, we propose
CARLA-Loc, a synthetic dataset of challenging and dynamic environments built on
CARLA simulator. We integrate multiple sensors into the dataset with strict
calibration, synchronization and precise timestamping. 7 maps and 42 sequences
are posed in our dataset with different dynamic levels and weather conditions.
Objects in both stereo images and point clouds are well-segmented with their
class labels. We evaluate 5 visual-based and 4 LiDAR-based approaches on varies
sequences and analyze the effect of challenging environmental factors on the
localization accuracy, showing the applicability of proposed dataset for
validating SLAM algorithms
DH-PTAM: A Deep Hybrid Stereo Events-Frames Parallel Tracking And Mapping System
This paper presents a robust approach for a visual parallel tracking and
mapping (PTAM) system that excels in challenging environments. Our proposed
method combines the strengths of heterogeneous multi-modal visual sensors,
including stereo event-based and frame-based sensors, in a unified reference
frame through a novel spatio-temporal synchronization of stereo visual frames
and stereo event streams. We employ deep learning-based feature extraction and
description for estimation to enhance robustness further. We also introduce an
end-to-end parallel tracking and mapping optimization layer complemented by a
simple loop-closure algorithm for efficient SLAM behavior. Through
comprehensive experiments on both small-scale and large-scale real-world
sequences of VECtor and TUM-VIE benchmarks, our proposed method (DH-PTAM)
demonstrates superior performance compared to state-of-the-art methods in terms
of robustness and accuracy in adverse conditions. Our implementation's
research-based Python API is publicly available on GitHub for further research
and development: https://github.com/AbanobSoliman/DH-PTAM.Comment: Submitted for publication in IEEE RA-