460 research outputs found
Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects
In this paper we introduce Co-Fusion, a dense SLAM system that takes a live
stream of RGB-D images as input and segments the scene into different objects
(using either motion or semantic cues) while simultaneously tracking and
reconstructing their 3D shape in real time. We use a multiple model fitting
approach where each object can move independently from the background and still
be effectively tracked and its shape fused over time using only the information
from pixels associated with that object label. Previous attempts to deal with
dynamic scenes have typically considered moving regions as outliers, and
consequently do not model their shape or track their motion over time. In
contrast, we enable the robot to maintain 3D models for each of the segmented
objects and to improve them over time through fusion. As a result, our system
can enable a robot to maintain a scene description at the object level which
has the potential to allow interactions with its working environment; even in
the case of dynamic scenes.Comment: International Conference on Robotics and Automation (ICRA) 2017,
http://visual.cs.ucl.ac.uk/pubs/cofusion,
https://github.com/martinruenz/co-fusio
A Factor Graph Approach to Multi-Camera Extrinsic Calibration on Legged Robots
Legged robots are becoming popular not only in research, but also in
industry, where they can demonstrate their superiority over wheeled machines in
a variety of applications. Either when acting as mobile manipulators or just as
all-terrain ground vehicles, these machines need to precisely track the desired
base and end-effector trajectories, perform Simultaneous Localization and
Mapping (SLAM), and move in challenging environments, all while keeping
balance. A crucial aspect for these tasks is that all onboard sensors must be
properly calibrated and synchronized to provide consistent signals for all the
software modules they feed. In this paper, we focus on the problem of
calibrating the relative pose between a set of cameras and the base link of a
quadruped robot. This pose is fundamental to successfully perform sensor
fusion, state estimation, mapping, and any other task requiring visual
feedback. To solve this problem, we propose an approach based on factor graphs
that jointly optimizes the mutual position of the cameras and the robot base
using kinematics and fiducial markers. We also quantitatively compare its
performance with other state-of-the-art methods on the hydraulic quadruped
robot HyQ. The proposed approach is simple, modular, and independent from
external devices other than the fiducial marker.Comment: To appear on "The Third IEEE International Conference on Robotic
Computing (IEEE IRC 2019)
Medical SLAM in an autonomous robotic system
One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-operative morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilities by observing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted instruments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This thesis addresses the ambitious goal of achieving surgical autonomy, through the study of the anatomical environment by Initially studying the technology present and what is needed to analyze the scene: vision sensors. A novel endoscope for autonomous surgical task execution is presented in the first part of this thesis. Which combines a standard stereo camera with a depth sensor. This solution introduces several key advantages, such as the possibility of reconstructing the 3D at a greater distance than traditional endoscopes. Then the problem of hand-eye calibration is tackled, which unites the vision system and the robot in a single reference system. Increasing the accuracy in the surgical work plan. In the second part of the thesis the problem of the 3D reconstruction and the algorithms currently in use were addressed. In MIS, simultaneous localization and mapping (SLAM) can be used to localize the pose of the endoscopic camera and build ta 3D model of the tissue surface. Another key element for MIS is to have real-time knowledge of the pose of surgical tools with respect to the surgical camera and underlying anatomy. Starting from the ORB-SLAM algorithm we have modified the architecture to make it usable in an anatomical environment by adding the registration of the pre-operative information of the intervention to the map obtained from the SLAM. Once it has been proven that the slam algorithm is usable in an anatomical environment, it has been improved by adding semantic segmentation to be able to distinguish dynamic features from static ones. All the results in this thesis are validated on training setups, which mimics some of the challenges of real surgery and on setups that simulate the human body within Autonomous Robotic Surgery (ARS) and Smart Autonomous Robotic Assistant Surgeon (SARAS) projects
- …