Menegatti, Emanuele

Munaro, Matteo

Rusu, Radu B.

English

MUNARO, MATTEO

MENEGATTI, EMANUELE

Archivio istituzionale della ricerca - Università di Padova

Original Citation:3D robot perception with Point Cloud LibraryElsevierPublisher:Published version:DOI:Terms of use:Open Access(Article begins on next page)This article is made available under terms and conditions applicable to Open Access Guidelines, as described athttp://www.unipd.it/download/file/fid/55401 (Italian only)Availability:This version is available at: 11577/3185204 since: 2016-05-11T14:31:44Z10.1016/j.robot.2015.12.008Università degli Studi di PadovaPadua Research Archive - Institutional Repository3D Robot Perception with Point Cloud Library1. Introduction3D perception allows robot to reach levels of auton-omy and safety that are not possible with 2D visionalone. Since 2010, 3D depth data became readily avail-able thanks to the introduction of new sensors, suchas the Microsoft Kinect, that could densely estimatedepth at low cost and couple it with color or infrareddata. The robotics community – already used to 3D in-formation estimated with lasers and stereo cameras –adopted the new RGB-D sensors very quickly and pro-moted a big part of research on exploiting combineddepth and color information for improving the perfor-mance of many robotic perception tasks. Together withthe diffusion of these novel sensors, the need for newtools and software libraries that could help in handlingthree-dimensional data rapidly emerged. It was also im-portant to standardize the representation of depth dataand depth+color data, in order, for example, to easilycompare data coming from different sensors.For this purpose, in 2011, the Point Cloud Library (orPCL) was created. PCL is a large scale, open project for2D/3D image and point cloud processing. Since 2011,PCL grew with an exceptional trend thanks to the con-tributions of more than 1,000 contributors from morethan 100 institutions all over the world. This growthled PCL to become a reference library for 3D process-ing, providing numerous state-of-the art algorithms forfiltering, feature estimation, surface reconstruction, reg-istration, model fitting and segmentation and driversfor the most diffused 3D sensors. Moreover, completepipelines needed for solving a number of complex per-ception problems are also present, such as object recog-nition and tracking, people detection and large-scale3D reconstruction. PCL’s open BSD license, free forcommercial and research use, favored its quick spreadamong both academia and industry. The tools and algo-rithms within PCL provide a wide basic infrastructurewhich allows researchers and companies to better con-centrate on their specific areas of expertise and prod-uct development, without the need of re-implementingall the parts of a complex system. Since the very be-ginning, PCL has been directly integrated as part of theRobot Operating System, being the backbone of most ofperception algorithms for robots equipped with RGB-Dsensors and thus boosting research and development inthis fundamental field.2. In this issueThis Special Issue on 3D Robot Perception with PointCloud Library stems from a workshop that was orga-nized in conjunction with the 13th International Con-ference on Autonomous Intelligent Systems (IAS-13),held in Padova (Italy), on July 15th, 2014. The purposeof this workshop was to bring together researchers pre-senting novel 3D perception algorithms and 3D systemsfor mobile robots built on the Point Cloud Library, inorder to create a forum to discuss advances in 3D robotperception and the role that the Point Cloud Library had,has and will have, in robotics research and industrial ap-plications. Due to the success of the workshop and thestrong interest in the topic of robot perception, we pro-moted this Special Issue, which was open not only topapers presented at the workshop, but also to contribu-tions not previously published. All papers were thencarefully peer-reviewed and a brief introduction to theaccepted papers follows.This special issue starts with a metrological charac-terization process for time-of-flight (TOF) cameras. Inparticular, Corti et al. apply this process to the Mi-crosoft Kinect V2, one of the most recently released andpromising RGB-D sensors that can be used for roboticperception. This work points out that performances arehighly influenced by measuring conditions and environ-mental parameters of the scene.When dealing with a mobile robot equipped withsuch sensors, the huge amount of 3D information thatis collected is usually difficult to manage due to the factthat the robot storage system and computing capabilitiesare insufficient. Therefore, a data compression methodis necessary to store and process this information whilepreserving as much information as possible. Data com-pression is also useful to efficiently transmit these datato other robots or to a central server for further process-Preprint submitted to Elsevier October 1, 2015ing. In the work of Navarrete et al., the authors proposea dataset composed of a set of 3D point clouds with dif-ferent structure and texture variability to evaluate the re-sults obtained from 3D data compression methods andprovide useful tools for comparing different methods.One fundamental task in 3D robot perception is theability to model the raw data, for example for the pur-pose of object reconstruction, recognition and grasping.Mo¨rwald et al. present an algorithm for robustly ap-proximating the boundary of a domain, latent in a pla-nar set of scattered points, by a B-spline curve. Addi-tional to the scientific contribution, they integrated theircode into PCL and created a tutorial that guides throughthe steps of the algorithm. Bu¨ttner et al., instead, pro-pose to model objects by means of primitive 3D shapes(cylinders, spheres, cones, etc.), thus creating an esti-mate of the invisible back sides. Their method detectssymmetrical parts of an object, model these symmetri-cal parts as primitive shapes and fit them to noisy sensordata using sample consensus methods. For the purposeof object grasping, this has the advantage that feasiblegrasps can be chosen from a precomputed set based onthe estimated model, instead of a time-consuming ran-dom sampling approach. The authors extended state ofthe art methods from PCL to include additional rele-vant shapes (e.g. boxes), constraints (e.g. on size andorientation), and to consider additional information likeknowledge about free space or proprioceptive informa-tion.For recognizing an object and its pose, an alternativeapproach to fitting primitive shapes consists in extract-ing and matching local feature descriptors. However,the research on 3D local descriptors is still at an earlystage. Logoglu et al. describe a new local descriptor,Colored Histograms of Spatial Concentric Surflet-Pairs(CoSPAIR), that encodes both depth and color informa-tion and outperforms the state-of-the-art descriptors inboth category-level and instance-level recognition task.The proposed descriptor is compared against the state-of-the-art local 3D descriptors that are available in PCLand their object recognition performances are evaluatedon several publicly available datasets. We conclude thegroup of papers on object recognition and pose esti-mation with the work by Choi and Christensen, whopresent an object pose estimation approach exploitingboth geometric depth and photometric color informationavailable from an RGB-D sensor. In contrast to vari-ous efforts relying on object segmentation with a knownbackground structure, their approach does not dependon the segmentation and thus exhibits superior perfor-mance in unstructured environments. Inspired by avoting-based approach employing an oriented point pairfeature, they present a voting-based approach which fur-ther incorporates color information from the RGB-Dsensor and which exploits parallel power of the modernparallel computing architecture.The availability of dense 3D data allowed to consider-ably improve also mapping and localization algorithms.Moral et al. present a flexible strategy to register scenesbased on their planar structure. The proposed strategyis based on the segmentation of the planar surfaces fromthe scene, and its representation using a graph whichstores the geometric relationships between neighboringplanar patches. Quick registration is achieved in indoorstructured scenarios, offering advantages like a compactrepresentation, and flexibility to adapt to different en-vironments and sensors. The authors provided also animplementation of their method as a module of PCL.For the purpose of visual odometry, Gutierrez-Gomezet al. propose to parametrize the geometric error by theinverse depth and switch keyframes based on a visibilitycriteria between frames. They implemented this methodwithin the scope of PCL as a branch of the code forKinFu Large Scale, an open source implementation ofKinectFusion, where the original ICP system for odom-etry estimation has been completely substituted by theirmethod. Moreover, Correia da Costa et al. describe amodular localization system suitable for a wide rangeof mobile robots and based on LIDAR / RGB-D data.It demonstrated high accuracy while performing posetracking with point cloud registration algorithms andhigh reliability when estimating the initial pose usingfeature matching techniques. Also this algorithm hasbeen made publicly available.The semantic localization problem in robotics con-sists in determining the place where a robot is locatedby means of semantic categories. In the paper of Mar-tinez et al., the authors propose a framework, imple-mented within PCL, which provides a set of valuabletools to easily develop and evaluate semantic localiza-tion systems. The implementation includes the gen-eration of 3D global descriptors following a Bag-of-Words approach. This allows the generation of fixed-dimensionality descriptors from any type of keypointdetector and feature extractor combinations.When a user and a robot share the same physicalworkspace the robot may need to keep an updated 3Drepresentation of the environment and focus on the partswhere the user executes manipulation tasks. Monica etal. propose to drive the robot behavior by a next-bestview algorithm that computes the most promising rangesensor viewpoints to observe the detected salient re-gions, where potential changes in the environment haveoccurred. The environment representation is built upon2the PCL KinFu Large Scale project, that has been par-allelized to exploit processing on GPUs.For robots having to interact with humans, in domes-tic or industrial environments, it is fundamental to beable to detect, track and distinguish people. On top ofPCL, OpenPTrack was specifically built for this, as de-scribed in the paper of Munaro et al.. OpenPTrack is anopen source software for multi-camera calibration andpeople tracking in RGB-D camera networks. It allowsto track people in big volumes at sensor frame rate andcurrently supports a heterogeneous set of 3D sensors.In the paper of this issue, the authors describe its user-friendly calibration procedure, which consists of simplesteps with real-time feedback that allow to obtain accu-rate results in estimating the camera poses that are thenused for tracking people. Moreover, they detail how acascade of algorithms working on depth point cloudsand color, infrared and disparity images is used to per-form people detection from different types of sensorsand in any indoor light condition.For what concerns the task of modeling and recogniz-ing people from 3D data, Tombari and Ichim proposean original approach for fitting a pre-defined parametricshape model to depth data by exploiting the 3D bodypose tracked through a sequence of range maps. To thisgoal, they make use of multiple types of constraints andcues embedded into a unique cost function, which isthen efficiently minimized. Their algorithm relies on thetools and algorithms provided by PCL, representing agood integration of the functionalities available therein.We chose to conclude this special issue with a casestudy on 3D robot perception that heavily relies on thePCL library. The paper of Oliveira et al. describes a 3Dobject perception and perceptual learning system devel-oped for a complex artificial cognitive agent workingin a restaurant scenario. This system, developed withinthe scope of the European project RACE, integrates de-tection, tracking, learning and recognition of tabletopobjects. This work shows the current importance ofPCL in building sophisticated 3D perception systems inrobotics and other domains.We hope that the ideas, tools and open source soft-ware presented by the papers collected in this specialissue can seed new research ideas and serve as a basisfor solving more and more complex 3D robot percep-tion problems. We also hope that the readers of thisspecial issue could consider to contribute back part oftheir work to PCL, in order to foster the growth of thisimportant open source ecosystem.AcknowledgementsWe wish to thank all the authors and reviewers of thepapers in this special issue for their great work in im-proving, polishing, and commenting the papers you canfinally find in the next pages. We also wish to thankProf. Ru¨diger Dillmann, the Editor in Chief of Roboticsand Autonomous Systems, for providing us the oppor-tunity to publish this special issue and Christine Brand,Hilda Hu and Mathangi Venkatesan for the help pro-vided during our editorial work.Matteo Munaro1Intelligent Autonomous Systems Laboratory,Department of Information Engineering (DEI), Schoolof Engineering, Universita’ degli Studi di Padova, ViaGradenigo 6/a, I-35131 Padua, ItalyE-mail address: matteo.munaro@dei.unipd.it.Radu B. RusuOpen Perception, Inc., 68 Willow Road, Menlo Park,CA 94025, USAE-mail address: rusu@openperception.org.Emanuele MenegattiIntelligent Autonomous Systems Laboratory,Department of Information Engineering (DEI), Schoolof Engineering, Universita’ degli Studi di Padova, ViaGradenigo 6/a, I-35131 Padua, ItalyE-mail address: emg@dei.unipd.it.1Corresponding editor.3

3D robot perception with Point Cloud Library

https://www.research.unipd.it/bitstream/11577/3185204/1/editorial.pdf

3D robot perception with Point Cloud Library

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Padova