Semi-automatic pose estimation of a fleet of robots with embedded stereoscopic cameras.

Given a fleet of robots, automatic estimation of the relative poses between them could be inaccurate in specific environments. We propose a framework composed by the fleet of robots with embedded stereoscopic cameras providing 2D and 3D images of the scene, a human coordinator and a Human-Machine Interface. We suppose auto localising each robot through GPS or landmarks is not possible. 3D-images are used to automatically align them and deduce the relative position between robots. 2Dimages are used to reduce the alignment error in an interactive manner. A human visualises both 2D-images and the current automatic alignment, and imposes a new alignment through the Human-Machine Interface. Since the information is shared through the whole fleet, robots can deduce the position of other ones that do not visualise the same scene. Practical evaluation shows that in situations where there is a large difference between images, the interactive processes are crucial to achieve an acceptable result

A comparison of feature extractors for panorama stitching in an autonomous car architecture.

Panorama stitching consists on frames being put together to create a 360o view. This technique is proposed for its implementation in autonomous vehicles instead of the use of an external 360o camera, mostly due to its reduced cost and improved aerodynamics. This strategy requires a fast and robust set of features to be extracted from the images obtained by the cameras located around the inside of the car, in order to effectively compute the panoramic view in real time and avoid hazards on the road. In this paper, we compare and discuss three feature extraction methods (i.e. SIFT, BRISK and SURF) for image feature extraction, in order to decide which one is more suitable for a panorama stitching application in an autonomous car architecture. Experimental validation shows that SURF exhibits an improved performance under a variety of image transformations, and thus appears to be the most suitable of these three methods, given its accuracy when comparing features between both images, while maintaining a low time consumption. Furthermore, a comparison of the results obtained with respect to similar work allows to increase the reliability of our methodology and the reach of our conclusions

Pose Detection and control of multiple unmanned underwater vehicles using optical feedback

This paper proposes pose detection and control algorithms in order to control the relative pose between two Unmanned Underwater Vehicles (UUVs) using optical feedback. The leader UUV is configured to have a light source at its crest which acts as a guiding beacon for the follower UUV which has a detector array at its bow. Pose detection algorithms are developed based on a classifier, such as the Spectral Angle Mapper (SAM), and chosen image parameters. An archive look-up table is constructed for varying combinations of 5-degree-of-freedom (DOF) motion (i.e., translation along all three coordinate axes as well as pitch and yaw rotations). Leader and follower vehicles are simulated for a case in which the leader is directed to specific waypoints in horizontal plane and the follower is required to maintain a fixed distance from the leader UUV. Proportional-Derivative (PD) control (without loss of generality) is applied to maintain stability of the UUVs to show proof of concept. Preliminary results indicate that the follower UUV is able to maintain its fixed distance relative to the leader UUV to within a reasonable accuracy

Online Mapping and Perception Algorithms for Multi-robot Teams Operating in Urban Environments.

This thesis investigates some of the sensing and perception challenges faced by multi-robot teams equipped with LIDAR and camera sensors. Multi-robot teams are ideal for deployment in large, real-world environments due to their ability to parallelize exploration, reconnaissance or mapping tasks. However, such domains also impose additional requirements, including the need for a) online algorithms (to eliminate stopping and waiting for processing to finish before proceeding) and b) scalability (to handle data from many robots distributed over a large area). These general requirements give rise to specific algorithmic challenges, including 1) online maintenance of large, coherent maps covering the explored area, 2) online estimation of communication properties in the presence of buildings and other interfering structure, and 3) online fusion and segmentation of multiple sensors to aid in object detection. The contribution of this thesis is the introduction of novel approaches that leverage grid-maps and sparse multi-variate gaussian inference to augment the capability of multi-robot teams operating in urban, indoor-outdoor environments by improving the state of the art of map rasterization, signal strength prediction, colored point cloud segmentation, and reliable camera calibration. In particular, we introduce a map rasterization technique for large LIDAR-based occupancy grids that makes online updates possible when data is arriving from many robots at once. We also introduce new online techniques for robots to predict the signal strength to their teammates by combining LIDAR measurements with signal strength measurements from their radios. Processing fused LIDAR+camera point clouds is also important for many object-detection pipelines. We demonstrate a near linear-time online segmentation algorithm to this domain. However, maintaining the calibration of a fleet of 14 robots made this approach difficult to employ in practice. Therefore we introduced a robust and repeatable camera calibration process that grounds the camera model uncertainty in pixel error, allowing the system to guide novices and experts alike to reliably produce accurate calibrations.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113516/1/jhstrom_1.pd

Flexible Supervised Autonomy for Exploration in Subterranean Environments

While the capabilities of autonomous systems have been steadily improving in recent years, these systems still struggle to rapidly explore previously unknown environments without the aid of GPS-assisted navigation. The DARPA Subterranean (SubT) Challenge aimed to fast track the development of autonomous exploration systems by evaluating their performance in real-world underground search-and-rescue scenarios. Subterranean environments present a plethora of challenges for robotic systems, such as limited communications, complex topology, visually-degraded sensing, and harsh terrain. The presented solution enables long-term autonomy with minimal human supervision by combining a powerful and independent single-agent autonomy stack, with higher level mission management operating over a flexible mesh network. The autonomy suite deployed on quadruped and wheeled robots was fully independent, freeing the human supervision to loosely supervise the mission and make high-impact strategic decisions. We also discuss lessons learned from fielding our system at the SubT Final Event, relating to vehicle versatility, system adaptability, and re-configurable communications.Comment: Field Robotics special issue: DARPA Subterranean Challenge, Advancement and Lessons Learned from the Final

Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future

Personal Guides: Heterogeneous Robots Sharing Personal Tours in Multi-Floor Environments

GidaBot is an application designed to setup and run a heterogeneous team of robots to act as tour guides in multi-floor buildings. Although the tours can go through several floors, the robots can only service a single floor, and thus, a guiding task may require collaboration among several robots. The designed system makes use of a robust inter-robot communication strategy to share goals and paths during the guiding tasks. Such tours work as personal services carried out by one or more robots. In this paper, a face re-identification/verification module based on state-of-the-art techniques is developed, evaluated offline, and integrated into GidaBot’s real daily activities, to avoid new visitors interfering with those attended. It is a complex problem because, as users are casual visitors, no long-term information is stored, and consequently, faces are unknown in the training step. Initially, re-identification and verification are evaluated offline considering different face detectors and computing distances in a face embedding representation. To fulfil the goal online, several face detectors are fused in parallel to avoid face alignment bias produced by face detectors under certain circumstances, and the decision is made based on a minimum distance criterion. This fused approach outperforms any individual method and highly improves the real system’s reliability, as the tests carried out using real robots at the Faculty of Informatics in San Sebastian show.This work has been partially funded by the Basque Government, Spain, grant number IT900-16, and the Spanish Ministry of Economy and Competitiveness (MINECO), grant number RTI2018-093337-B-I00

Cooperative Vehicle Perception and Localization Using Infrastructure-based Sensor Nodes

Reliable and accurate Perception and Localization (PL) are necessary for safe intelligent transportation systems. The current vehicle-based PL techniques in autonomous vehicles are vulnerable to occlusion and cluttering, especially in busy urban driving causing safety concerns. In order to avoid such safety issues, researchers study infrastructure-based PL techniques to augment vehicle sensory systems. Infrastructure-based PL methods rely on sensor nodes that each could include camera(s), Lidar(s), radar(s), and computation and communication units for processing and transmitting the data. Vehicle to Infrastructure (V2I) communication is used to access the sensor node processed data to be fused with the onboard sensor data. In infrastructure-based PL, signal-based techniques- in which sensors like Lidar are used- can provide accurate positioning information while vision-based techniques can be used for classification. Therefore, in order to take advantage of both approaches, cameras are cooperatively used with Lidar in the infrastructure sensor node (ISN) in this thesis. ISNs have a wider field of view (FOV) and are less likely to suffer from occlusion. Besides, they can provide more accurate measurements since they are fixed at a known location. As such, the fusion of both onboard and ISN data has the potential to improve the overall PL accuracy and reliability. This thesis presents a framework for cooperative PL in autonomous vehicles (AVs) by fusing ISN data with onboard sensor data. The ISN includes cameras and Lidar sensors, and the proposed camera Lidar fusion method combines the sensor node information with vehicle motion models and kinematic constraints to improve the performance of PL. One of the main goals of this thesis is to develop a wind induced motion compensation module to address the problem of time-varying extrinsic parameters of the ISNs. The proposed module compensates for the effect of the motion of ISN posts due to wind or other external disturbances. To address this issue, an unknown input observer is developed that uses the motion model of the light post as well as the sensor data. The outputs of the ISN, the positions of all objects in the FOV, are then broadcast so that autonomous vehicles can access the information via V2I connectivity to fuse with their onboard sensory data through the proposed cooperative PL framework. In the developed framework, a KCF is implemented as a distributed fusion method to fuse ISN data with onboard data. The introduced cooperative PL incorporates the range-dependent accuracy of the ISN measurements into fusion to improve the overall PL accuracy and reliability in different scenarios. The results show that using ISN data in addition to onboard sensor data improves the performance and reliability of PL in different scenarios, specifically in occlusion cases