Supervised Autonomous Locomotion and Manipulation for Disaster Response with a Centaur-like Robot

Mobile manipulation tasks are one of the key challenges in the field of search and rescue (SAR) robotics requiring robots with flexible locomotion and manipulation abilities. Since the tasks are mostly unknown in advance, the robot has to adapt to a wide variety of terrains and workspaces during a mission. The centaur-like robot Centauro has a hybrid legged-wheeled base and an anthropomorphic upper body to carry out complex tasks in environments too dangerous for humans. Due to its high number of degrees of freedom, controlling the robot with direct teleoperation approaches is challenging and exhausting. Supervised autonomy approaches are promising to increase quality and speed of control while keeping the flexibility to solve unknown tasks. We developed a set of operator assistance functionalities with different levels of autonomy to control the robot for challenging locomotion and manipulation tasks. The integrated system was evaluated in disaster response scenarios and showed promising performance.Comment: In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 201

Alignbodynet: deep learning-based alignment of non-overlapping partial body point clouds from a single depth camera

This paper proposes a novel deep learning framework to generate omnidirectional 3D point clouds of human bodies by registering the front- and back-facing partial scans captured by a single depth camera. Our approach does not require calibration-assisting devices, canonical postures, nor does it make assumptions concerning an initial alignment or correspondences between the partial scans. This is achieved by factoring this challenging problem into ( i ) building virtual correspondences for partial scans, and ( ii ) implicitly predicting the rigid transformation between the two partial scans via the predicted virtual correspondences. In this study, we regress the SMPL vertices from the two partial scans for building the virtual correspondences. The main challenges are ( i ) estimating the body shape and pose under clothing from single partial dressed body point clouds, and ( ii ) the predicted bodies from front- and back-facing inputs required to be the same. We, thus, propose a novel deep neural network dubbed AlignBodyNet that introduces shape-interrelated features and a shape-constraint loss for resolving this problem.We also provide a simple yet efficient method for generating real-world partial scans from complete models, which fills the gap in the lack of quantitative comparisons based on the real-world data for various studies including partial registration, shape completion, and view synthesis. Experiments based on synthetic and real-world data show that our method achieves state-of-the-art performance in both objective and subjective terms

Design and Analysis of a Single-Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs)

We describe the design and 3D sensing performance of an omnidirectional stereo (omnistereo) vision system applied to Micro Aerial Vehicles (MAVs). The proposed omnistereo sensor employs a monocular camera that is co-axially aligned with a pair of hyperboloidal mirrors (a vertically-folded catadioptric configuration). We show that this arrangement provides a compact solution for omnidirectional 3D perception while mounted on top of propeller-based MAVs (not capable of large payloads). The theoretical single viewpoint (SVP) constraint helps us derive analytical solutions for the sensor’s projective geometry and generate SVP-compliant panoramic images to compute 3D information from stereo correspondences (in a truly synchronous fashion). We perform an extensive analysis on various system characteristics such as its size, catadioptric spatial resolution, field-of-view. In addition, we pose a probabilistic model for the uncertainty estimation of 3D information from triangulation of back-projected rays. We validate the projection error of the design using both synthetic and real-life images against ground-truth data. Qualitatively, we show 3D point clouds (dense and sparse) resulting out of a single image captured from a real-life experiment. We expect the reproducibility of our sensor as its model parameters can be optimized to satisfy other catadioptric-based omnistereo vision under different circumstances

Panoramic Stereovision and Scene Reconstruction

With advancement of research in robotics and computer vision, an increasingly high number of applications require the understanding of a scene in three dimensions. A variety of systems are deployed to do the same. This thesis explores a novel 3D imaging technique. This involves the use of catadioptric cameras in a stereoscopic arrangement. A secondary system aims to stabilize the system in the event that the cameras are misaligned during operation. The system provides a stark advantage due to it being a cost effective alternative to present day standard state-of-the-art systems that achieve the same goal of 3D imaging. The compromise lies in the quality of depth estimation, which can be overcome with a different imager and calibration. The result was a panoramic disparity map generated by the system

Challenges and solutions for autonomous ground robot scene understanding and navigation in unstructured outdoor environments: A review

The capabilities of autonomous mobile robotic systems have been steadily improving due to recent advancements in computer science, engineering, and related disciplines such as cognitive science. In controlled environments, robots have achieved relatively high levels of autonomy. In more unstructured environments, however, the development of fully autonomous mobile robots remains challenging due to the complexity of understanding these environments. Many autonomous mobile robots use classical, learning-based or hybrid approaches for navigation. More recent learning-based methods may replace the complete navigation pipeline or selected stages of the classical approach. For effective deployment, autonomous robots must understand their external environments at a sophisticated level according to their intended applications. Therefore, in addition to robot perception, scene analysis and higher-level scene understanding (e.g., traversable/non-traversable, rough or smooth terrain, etc.) are required for autonomous robot navigation in unstructured outdoor environments. This paper provides a comprehensive review and critical analysis of these methods in the context of their applications to the problems of robot perception and scene understanding in unstructured environments and the related problems of localisation, environment mapping and path planning. State-of-the-art sensor fusion methods and multimodal scene understanding approaches are also discussed and evaluated within this context. The paper concludes with an in-depth discussion regarding the current state of the autonomous ground robot navigation challenge in unstructured outdoor environments and the most promising future research directions to overcome these challenges

Full-space Cloud of Random Points with a Scrambling Metasurface

With the rapid progress in computer science, including artificial intelligence, big data and cloud computing, full-space spot generation can be pivotal to many practical applications, such as facial recognition, motion detection, augmented reality, etc. These opportunities may be achieved by using diffractive optical elements (DOEs) or light detection and ranging (LIDAR). However, DOEs suffer from intrinsic limitations, such as demanding depth-controlled fabrication techniques, large thicknesses (more than the wavelength), Lambertian operation only in half space, etc. LIDAR nevertheless relies on complex and bulky scanning systems, which hinders the miniaturization of the spot generator. Here, inspired by a Lambertian scatterer, we report a Hermitian-conjugate metasurface scrambling the incident light to a cloud of random points in full space with compressed information density, functioning in both transmission and reflection spaces. Over 4044 random spots are experimentally observed in the entire space, covering angles at nearly 90 degrees. Our scrambling metasurface is made of amorphous silicon with a uniform subwavelength height, a nearly continuous phase coverage, a lightweight, flexible design, and low-heat dissipation. Thus, it may be mass produced by and integrated into existing semiconductor foundry designs. Our work opens important directions for emerging 3D recognition sensors, such as motion sensing, facial recognition, and other applications.113Nsciescopu

A Real Application of an Autonomous Industrial Mobile Manipulator within Industrial Context

In modern industry there are still a large number of low added-value processes that can be automated or semi-automated with safe cooperation between robot and human operators. The European SHERLOCK project aims to integrate an autonomous industrial mobile manipulator (AIMM) to perform cooperative tasks between a robot and a human. To be able to do this, AIMMs need to have a variety of advanced cognitive skills like autonomous navigation, smart perception and task management. In this paper, we report the project’s tackle in a paradigmatic industrial application combining accurate autonomous navigation with deep learning-based 3D perception for pose estimation to locate and manipulate different industrial objects in an unstructured environment. The proposed method presents a combination of different technologies fused in an AIMM that achieve the proposed objective with a success rate of 83.33% in tests carried out in a real environment.This research was funded by EC research project “SHERLOCK—Seamless and safe human-centered robotic applications for novel collaborative workplace”. Grant number: 820683 (https://www.sherlock-project.eu accessed on 12 March 2021)