A Collaborative Visual Localization Scheme for a Low-Cost Heterogeneous Robotic Team with Non-Overlapping Perspectives

This paper presents and evaluates a relative localization scheme for a heterogeneous team of low-cost mobile robots. An error-state, complementary Kalman Filter was developed to fuse analytically-derived uncertainty of stereoscopic pose measurements of an aerial robot, made by a ground robot, with the inertial/visual proprioceptive measurements of both robots. Results show that the sources of error, image quantization, asynchronous sensors, and a non-stationary bias, were sufficiently modeled to estimate the pose of the aerial robot. In both simulation and experiments, we demonstrate the proposed methodology with a heterogeneous robot team, consisting of a UAV and a UGV tasked with collaboratively localizing themselves while avoiding obstacles in an unknown environment. The team is able to identify a goal location and obstacles in the environment and plan a path for the UGV to the goal location. The results demonstrate localization accuracies of 2cm to 4cm, on average, while the robots operate at a distance from each-other between 1m and 4m

On-Board Vision-Based 3D Relative Localization System for Multiple Quadrotors

This work proposes a novel relative localization system, based on active markers and an on-board camera, for tracking multiple quadrotors in a limited field of view. The system extracts the 3D poses of the markers including one that, by pulsating at a predefined frequency, provides an unique platform ID. We discuss how the camera field of view can be explored in presence of multiple targets, and what are the conditions on the system visibility that lead to the establishment of bidirectional sensing between robots with similar sensing capabilities. A visibility analysis is conducted to show that the developed relative localization system meets such requirements, and a closed-loop experiment is used to validate its performance under these conditions. Finally, its performance is compared with other results from the literature, and a metric is established with the intent of mapping different design solutions, facilitating design choices in presence of different requirements

Lisätyn todellisuuden käyttöliittymä puoliautonomisiin työkoneisiin

Forest machines are being automated today. However, the challenging environment and complexity of the work makes the task difficult. A forest machine operator needs easily interpretable input from the machine in order to supervise and control it. Hence, a device that would show the digital information as a part of the real environment is desired. The goal of the thesis is to implement a real time augmented reality display for forest machines. The main task is to estimate the pose of the user’s head because the virtual data should be aligned with real objects. Also, the digital content and how it is visualized has to be considered. A machine vision camera and inertial measurements are used in the pose estimation. Visual markers are utilized to get pose estimate of the camera. And, orientation from inertial measurements is estimated using an extended Kalman filter. To get the final estimate, the orientations of the two devices are sensor fused. Furthermore, the virtual data comes mainly from an on-board lidar. A 3D point cloud and a wire frame model of a forestry crane are augmented to a live video on a PC. The implemented system proved to work outdoors with actual hardware in real time. Although there are some identifiable errors in the pose estimate, the initial results are encouraging. Further improvements should be targeted to the accuracy of marker detection and to the development of a comprehensive sensor fusion algorithm.Haastava ympäristö ja monimutkaiset työtehtävät tekevät metsäkoneiden toimintojen automatisoimisesta vaikeaa. Olisikin toivottavaa, että metsäkoneenkuljettaja pystyisi tulkitsemaan koneelta tulevaa tietoa helposti ja nopeasti. Ratkaisuksi ehdotetaan järjestelmää, joka sulauttaa digitaalisen tiedon osaksi käyttöympäristöä. Tämä mahdollistaisi puoliautonomisen työkoneen sujuvamman valvomisen ja ohjaamisen. Tämän työn tavoitteena on toteuttaa lisätyn todellisuuden näyttö metsäkoneisiin. Tärkeimpänä tehtävänä on estimoida käyttäjän pään sijainti ja asento, sillä digitaalisen datan pitäisi limittyä todellisuuden kanssa. Lisäksi on pohdittava virtuaalisen tiedon sisältö, ja kuinka se esitetään käyttäjälle. Asennon ja paikan mittaamiseen käytetään päähän kiinnitettyä konenäkökameraa ja inertiamittausyksikköä. Kameralla tunnistetaan työkoneen hyttiin sijoitettuja tunnistemerkkejä, joilla sekä kameran paikkaa että asentoa voidaan estimoida. Asentoestimaattia korjataan vielä inertiamittauksilla anturifuusiota hyödyntäen. Virtuaalinen tieto näytölle tulee pääasiassa laserkeilaimelta ja se lisätään tietokoneen ruudulla näkyvään videoon kolmiulotteisena pistepilvenä. Myös metsäkoneen puomi ja työkalu esitetään virtuaalisena mallina. Toteutettu järjestelmä osoittautui toimimaan oikealla laitteistolla ulkoilmassa tehdyssä kokeessa. Alustavat tulokset ovat rohkaisevia, mutta myös paikan ja asennon virheitä havaittiin ja identifioitiin. Tulevaisuuden kehityskohteita ovat tunnisteiden paikan tarkempi mittaaminen ja kokonaisvaltaisemman anturifuusion kehittäminen

A monocular pose estimation system based on infrared LEDs

We present an accurate, efficient, and robust pose estimation system based on infrared LEDs. They are mounted on a target object and are observed by a camera that is equipped with an infrared-pass filter. The correspondences between LEDs and image detections are first determined using a combinatorial approach and then tracked using a constant-velocity model. The pose of the target object is estimated with a P3P algorithm and optimized by minimizing the reprojection error. Since the system works in the infrared spectrum, it is robust to cluttered environments and illumination changes. In a variety of experiments, we show that our system outperforms state-of-the-art approaches. Furthermore, we successfully apply our system to stabilize a quadrotor both indoors and outdoors under challenging conditions. We release our implementation as open-source software

Target Tracking Using Optical Markers for Remote Handling in ITER

The thesis focuses on the development of a vision system to be used in the remote handling systems of the International Thermonuclear Experimental Rector - ITER. It presents and discusses a realistic solution to estimate the pose of key operational targets, while taking into account the specific needs and restrictions of the application. The contributions to the state of the art are in two main fronts: 1) the development of optical markers that can withstand the extreme conditions in the environment; 2) the development of a robust marker detection and identification framework that can be effectively applied to different use cases. The markers’ locations and labels are used in computing the pose. In the first part of the work, a retro reflective marker made up ITER compliant materials, particularly, fused silica and stainless steel, is designed. A methodology is proposed to optimize the markers’ performance. Highly distinguishable markers are manufactured and tested. In the second part, a hybrid pipeline is proposed that detects uncoded markers in low resolution images using classical methods and identifies them using a machine learning approach. It is demonstrated that the proposed methodology effectively generalizes to different marker constellations and can successfully detect both retro reflective markers and laser engravings. Lastly, a methodology is developed to evaluate the end-to-end accuracy of the proposed solution using the feedback provided by an industrial robotic arm. Results are evaluated in a realistic test setup for two significantly different use cases. Results show that marker based tracking is a viable solution for the problem at hand and can provide superior performance to the earlier stereo matching based approaches. The developed solutions could be applied to other use cases and applications