We present an open-source marker-based localization system intended as a low-cost easy-to-deploy solution for aerial and swarm robotics. The main advantage of the presented method is its high computational efficiency, which allows its deployment on small robots with limited computational resources. Even on low-end computers, the core component of the system can detect and estimate 3D positions of hundreds of black and white markers at the maximum frame-rate of standard cameras. The method is robust to changing lighting conditions and achieves accuracy in the order of millimeters to centimeters. Due to its reliability, simplicity of use and availability as an open-source ROS module (http://purl.org/robotics/whycon), the system is now used in a number of aerial robotics projects where fast and precise relative localization is required

Cizek, Petr

Duckett, Tom

Krajnik, Tomas

Mejail, Marta

Nitsche, Matias

University of Lincoln Institutional Repository

WhyCon: An Efficent, Marker-based Localization SystemMatı´as Nitsche1 Toma´sˇ Krajnı´k2 Petr Cˇı´zˇek3 Marta Mejail1 Tom Duckett2Abstract— We present an open-source marker-based localiza-tion system intended as a low-cost easy-to-deploy solution foraerial and swarm robotics. The main advantage of the presentedmethod is its high computational efficiency, which allows its de-ployment on small robots with limited computational resources.Even on low-end computers, the core component of the systemcan detect and estimate 3D positions of hundreds of blackand white markers at the maximum frame-rate of standardcameras. The method is robust to changing lighting conditionsand achieves accuracy in the order of millimeters to centimeters.Due to its reliability, simplicity of use and availability as anopen-source ROS module (http://purl.org/robotics/whycon), thesystem is now used in a number of aerial robotics projectswhere fast and precise relative localization is required.I. INTRODUCTIONPrecise and reliable pose estimation is one of the mainproblems of mobile robotics. Robots not only need to es-timate their own pose, but, in the case of multi-robot andswarm systems, they also need to determine the relative posesof their peers. While both the self-localization and pose-estimation problems can be tackled by several methods, thesemight still be too computationally intensive for deploymenton robots with constrained processing power. Thus, a popularsolution to the problem of localization is the use of artifi-cial markers, which are tailored for reliable detection andaccurate localization.This paper presents a precise, fast, reliable, easy to use,and open-source vision-based localization system based onoff-the-shelf components [1]. Unlike widely used commer-cial systems, the presented system does not require anyspecial equipment and is significantly faster than the ArTag-[2] and OpenCV-based solutions. A thorough description ofthe method is presented in [3], [4] and a complete hardwareand software solution comprising a lightweight embeddedmodule that runs an early version of the described algorithmis presented in [5].II. DESCRIPTIONThe core of the presented system is based on efficientdetection of a black and white roundel (see Figure 1)with known dimensions. The algorithm combines on-demandthresholding, a flood-fill technique, and on-the-fly calculationof the detected pattern statistics.1Laboratory of Robotics and Embedded Systems, University of BuenosAires, Argentina Email: mnitsche@dc.uba.ar2Lincoln Centre for Autonomous Systems, University of Lincoln, UK3Dept. of Computer Science, FEE, Czech Technical University, CZThe work is supported by the EU ICT project 600623 ‘STRANDS’. Thework of P. Cˇı´zˇek has been supported by the Czech Science Foundation(GACˇR) under research project No. 13-18316P.Fig. 1. Decentralized formation flight based on relative localizationperformed by the presented method. Courtesy of the GRASP laboratory,UPENN.In the first stage, the algorithm employs a fast flood-filltechnique to search for contiguous segments of dark pixels.The ratio of each segment’s pixels to its bounding box areais then compared to a value calculated from the knowndimensions of the searched pattern. If the test is passed, themethod starts to search around the segment’s centroid foranother continuous area of bright pixels and again verifiesif the number of pixels conforms to the value calculatedfrom its bounding box dimensions. Successful pass of thesecond test indicates that the method found two elliptical,roughly concentric patterns and the subsequent stage veri-fies if the ratio of their areas, concentricity and semi-axesdimensions conform with the known shape of the searchedpattern. Once the segments found pass this test, the pattern’s3D position is calculated based on its known dimensions,camera parameters, and the user-defined coordinate system.The aforementioned consecutive steps allow for a quickrejection of outliers in the early stages of the detection, whichimproves the algorithm’s performance. More importantly, theflood-fill technique can be initiated from any position inthe processed image without a performance penalty, whichallows to start the search at a position predicted from theprevious detection. This typically results in a quick searchthat processes only pixels that belong to the actual patternand its immediate vicinity, which significantly reduces thecomputational cost.The localization system is available as a ROS package [1],which includes both a shared library containing the mainfunctionality and a series of nodes. A base node performstarget detection and 3D position estimation in camera co-ordinates. For the case of targets moving on the plane, asimple calibration can be performed resulting in increasedlocalization precision. For the full 3D motion case, anothernode can estimate the position in user-defined coordinates.Finally, by using a pattern composed of three circles formingan “L” shape, orientation can be directly obtained.III. PERFORMANCEThe accuracy of the position estimation and computationalefficiency of the detection were thoroughly examined in [4],which presents comparisons of the method to the popularArUco [2] system and an OpenCV-based solution. Theresults presented in [4] show that the typical search timeof a single pattern in an 1-10Mpix image is about 40 µs ona standard desktop and 1-5 ms on the Raspberry Pi, whichis two orders of magnitude faster than ArUco or OpenCV.Furthermore, [4] reports that the system accuracy is about1% for the full 3D and 0.2% for 2D position estimation,which outperforms slightly the other two baseline methods.The paper [4] also presents a mathematical model whichallows to determine the system accuracy, processing timeand coverage based on the camera and processing hardwareparameters.IV. APPLICATIONSThe presented system is used by several research groupsinternationally, both ‘off-board’ as a low-cost alternative toprecise motion-capture-based localization systems and ‘on-board’ for detecting other robots and important environmentcues. A typical example of its ‘off-board’ application isits use as an external reference to evaluate the precisionand efficiency of visual-based UAV navigation [6], [7] andlocalization methods [8]. Other research teams used oursoftware on-board UAVs as a computationally efficient andprecise module for relative localization of other quadro-tors [5], [9], which allowed to deploy quadrotor swarmsin surveillance [10] and plume tracking [11]. The abilityto accurately determine relative positions of other robotsalso allows for precise formation control of both UAV [12]and heterogenous UAV-UGV teams [13], [14]. Moreover, thesystem is precise and robust enough to allow autonomouslanding of the UAV’s on a slowly moving UGV [15].The system was also used outside of the aerial roboticsdomain, e.g., to evaluate the accuracy of ground robotnavigation [16], [17] and efficiency of motion planning inevolutionary robotics [18], [19]. Due to its computationalefficiency and ease of use, it has found applications in swarmrobotics, e.g., for visual-based docking to power sources [20],as a component of artificial pheromone systems [21], and inautomated analysis of the swarm behaviour [22].V. CONCLUSIONSWe present a computationally efficient and precise vision-based system intended for marker-based localization. Thesystem allows real-time detection and precise localiza-tion of large numbers of black-and-white markers oncomputationally-constrained hardware. Source code for thepresented method is publicly available at [1], either as astandalone application or as a component of the RobotOperating System (ROS) framework. The system reliability,ease of use, and the fact that it utilizes only low-cost, off-the-shelf components makes it a popular choice in aerial andswarm robotics research.REFERENCES[1] M. Nitsche and T. Krajnı´k, “WhyCon localization system,” [cited 11Aug 2015]. [Online]. Available: http://purl.org/robotics/whycon[2] “ArUco: a minimal library for augmented reality applicationsbased on OpenCV,” [cited 8 Jan 2014]. [Online]. Available:http://www.uco.es/investiga/grupos/ava/node/26[3] T. Krajnı´k, M. Nitsche, J. Faigl, T. Duckett, M. Mejail, and L. Prˇeucˇil,“External localization system for mobile robotics,” in 16th Interna-tional Conference on Advanced Robotics (ICAR). IEEE, 2013.[4] T. Krajnı´k, M. Nitsche, J. Faigl, P. Vaneˇk, M. Saska, L. Prˇeucˇil,T. Duckett, and M. Mejail, “A practical multirobot localization sys-tem,” Journal of Intelligent & Robotic Systems, vol. 76, no. 3-4, pp.539–562, 2014.[5] J. Faigl et al., “Low-cost embedded system for relative localization inrobotic swarms,” in Int. Conf. on Robotics and Automation (ICRA),2013. [Online]. Available: http://purl.org/faigl/caspa[6] T. Krajnı´k, M. Nitsche, S. Pedre, L. Prˇeucˇil, and M. E. Mejail, “Asimple visual navigation system for an UAV,” in International Multi-Conference on Systems, Signals and Devices (SSD). IEEE, 2012.[7] M. Nitsche, T. Pire, T. Krajnı´k, M. Kulich, and M. Mejail, “Montecarlo localization for teach-and-repeat feature-based navigation,” inAdvances in Autonomous Robotics Systems. Springer, 2014.[8] A. Basit, W. S. Qureshi, M. N. Dailey, and T. Krajnı´k, “Jointlocalization of pursuit quadcopters and target using monocular cues,”Journal of Intelligent & Robotic Systems, 2014.[9] M. Saska, “MAV-swarms: Unmanned aerial vehicles stabilized alonga given path using onboard relative localization,” in InternationalConference on Unmanned Aircraft Systems (ICUAS). IEEE, 2015.[10] M. Saska et al., “Autonomous deployment of swarms of micro-aerialvehicles in cooperative surveillance,” in International Conference onUnmanned Aircraft Systems (ICUAS). IEEE, 2014.[11] M. Saska, J. Langr, and L. Prˇeucˇil, “Plume tracking by a self-stabilizedgroup of micro aerial vehicles,” in Modelling and Simulation forAutonomous Systems: First International Workshop, MESAS, RevisedSelected Papers, vol. 8906. Springer, 2014, p. 44.[12] Z. Kasl, M. Saska, and L. Prˇeucˇil, “Rapidly exploring random trees-based initialization of MPC technique designed for formations ofMAVs,” in 11th International Conference on Informatics in Control,Automation and Robotics (ICINCO). IEEE, 2014.[13] M. Saska, V. Vona´sek, T. Krajnı´k, and L. Prˇeucˇil, “Coordination andnavigation of heterogeneous MAV–UGV formations localized by a“hawk-eye”-like approach under a model predictive control scheme,”International Journal of Robotics Research, 2014.[14] M. Saska et al., “Fault-tolerant formation driving mechanism designedfor heterogeneous MAVs-UGVs groups,” Journal of Intelligent &Robotic Systems, vol. 73, no. 1-4, pp. 603–622, 2014.[15] M. Saska, T. Krajnı´k, and L. Prˇeucˇil, “Cooperative µUAV-UGV au-tonomous indoor surveillance,” in 9th International Multi-Conferenceon Systems, Signals and Devices (SSD). IEEE, 2012, pp. 1–6.[16] P. De Cristo´foris, M. Nitsche, T. Krajnı´k, T. Pire, and M. Mejail, “Hy-brid vision-based navigation for mobile robots in mixed indoor/outdoorenvironments,” Pattern Recognition Letters, 2015.[17] S. Gonzalez, E. Gonzlez, M. Nitsche, and P. De Cristforis, “Odometrı´avisual para robots mo´viles utilizando smartphones como unidad desensado y procesamiento,” in Jornadas Argentinas de Robo´tica, 2014.[18] V. Vona´sek, M. Saska, L. Winkler, and L. Prˇeucˇil, “High-level motionplanning for CPG-driven modular robots,” Robotics and AutonomousSystems, vol. 68, pp. 116–128, 2015.[19] V. Vona´sek, L. Winkler, J. Liedke, M. Saska, K. Kosˇnar, and L. Prˇeucˇil,“Fast on-board motion planning for modular robots,” in ICRA, 2014.[20] P. Levi et al., “A cognitive architecture for modular and self-reconfigurable robots,” in 8th Annual IEEE Systems Conference(SysCon). IEEE, 2014, pp. 465–472.[21] F. Arvin, T. Krajnik, A. E. Turgut, and S. Yue, “COSΦ: artificialpheromone system for robotic swarms research,” in IROS, 2015.[22] F. Arvin, A. Attar, A. E. Turgut, and S. Yue, “Power-law distributionof long-term experimental data in swarm robotics,” in Advances inSwarm and Computational Intelligence. Springer, 2015.

WhyCon: an efficient, marker-based localization system

Abstract

Similar works

Full text

Available Versions

University of Lincoln Institutional Repository