PAMPC: Perception-Aware Model Predictive Control for Quadrotors

We present the first perception-aware model predictive control framework for quadrotors that unifies control and planning with respect to action and perception objectives. Our framework leverages numerical optimization to compute trajectories that satisfy the system dynamics and require control inputs within the limits of the platform. Simultaneously, it optimizes perception objectives for robust and reliable sens- ing by maximizing the visibility of a point of interest and minimizing its velocity in the image plane. Considering both perception and action objectives for motion planning and control is challenging due to the possible conflicts arising from their respective requirements. For example, for a quadrotor to track a reference trajectory, it needs to rotate to align its thrust with the direction of the desired acceleration. However, the perception objective might require to minimize such rotation to maximize the visibility of a point of interest. A model-based optimization framework, able to consider both perception and action objectives and couple them through the system dynamics, is therefore necessary. Our perception-aware model predictive control framework works in a receding-horizon fashion by iteratively solving a non-linear optimization problem. It is capable of running in real-time, fully onboard our lightweight, small-scale quadrotor using a low-power ARM computer, to- gether with a visual-inertial odometry pipeline. We validate our approach in experiments demonstrating (I) the contradiction between perception and action objectives, and (II) improved behavior in extremely challenging lighting conditions

Calibration and Sensitivity Analysis of a Stereo Vision-Based Driver Assistance System

Az http://intechweb.org/ alatti "Books" fül alatt kell rákeresni a "Stereo Vision" címre és az 1. fejezetre

Fast, Autonomous Flight in GPS-Denied and Cluttered Environments

One of the most challenging tasks for a flying robot is to autonomously navigate between target locations quickly and reliably while avoiding obstacles in its path, and with little to no a-priori knowledge of the operating environment. This challenge is addressed in the present paper. We describe the system design and software architecture of our proposed solution, and showcase how all the distinct components can be integrated to enable smooth robot operation. We provide critical insight on hardware and software component selection and development, and present results from extensive experimental testing in real-world warehouse environments. Experimental testing reveals that our proposed solution can deliver fast and robust aerial robot autonomous navigation in cluttered, GPS-denied environments.Comment: Pre-peer reviewed version of the article accepted in Journal of Field Robotic

Visual sensor fusion for active security in robotic industrial environments

This work presents a method of information fusion involving data captured by both a standard CCD camera and a ToF camera to be used in the detection of the proximity between a manipulator robot and a human. Both cameras are assumed to be located above the work area of an industrial robot. The fusion of colour images and time of light information makes it possible to know the 3D localization of objects with respect to a world coordinate system. At the same time this allows to know their colour information. Considering that ToF information given by the range camera contains innacuracies including distance error, border error, and pixel saturation, some corrections over the ToF information are proposed and developed to improve the results. The proposed fusion method uses the calibration parameters of both cameras to reproject 3D ToF points, expressed in a common coordinate system for both cameras and a robot arm, in 2D colour images. In addition to this, using the 3D information, the motion detection in a robot industrial environment is achieved, and the fusion of information is applied to the foreground objects previously detected. This combination of information results in a matrix that links colour and 3D information, giving the possibility of characterising the object by its colour in addition to its 3D localization. Further development of these methods will make it possible to identify objects and their position in the real world, and to use this information to prevent possible collisions between the robot and such objects

Introspective Perception for Mobile Robots

Perception algorithms that provide estimates of their uncertainty are crucial to the development of autonomous robots that can operate in challenging and uncontrolled environments. Such perception algorithms provide the means for having risk-aware robots that reason about the probability of successfully completing a task when planning. There exist perception algorithms that come with models of their uncertainty; however, these models are often developed with assumptions, such as perfect data associations, that do not hold in the real world. Hence the resultant estimated uncertainty is a weak lower bound. To tackle this problem we present introspective perception - a novel approach for predicting accurate estimates of the uncertainty of perception algorithms deployed on mobile robots. By exploiting sensing redundancy and consistency constraints naturally present in the data collected by a mobile robot, introspective perception learns an empirical model of the error distribution of perception algorithms in the deployment environment and in an autonomously supervised manner. In this paper, we present the general theory of introspective perception and demonstrate successful implementations for two different perception tasks. We provide empirical results on challenging real-robot data for introspective stereo depth estimation and introspective visual simultaneous localization and mapping and show that they learn to predict their uncertainty with high accuracy and leverage this information to significantly reduce state estimation errors for an autonomous mobile robot

Visual odometry parametres optimization for autonomous underwater vehicles

Peer Reviewe

Flexible Stereo: Constrained, Non-rigid, Wide-baseline Stereo Vision for Fixed-wing Aerial Platforms

This paper proposes a computationally efficient method to estimate the time-varying relative pose between two visual-inertial sensor rigs mounted on the flexible wings of a fixed-wing unmanned aerial vehicle (UAV). The estimated relative poses are used to generate highly accurate depth maps in real-time and can be employed for obstacle avoidance in low-altitude flights or landing maneuvers. The approach is structured as follows: Initially, a wing model is identified by fitting a probability density function to measured deviations from the nominal relative baseline transformation. At run-time, the prior knowledge about the wing model is fused in an Extended Kalman filter~(EKF) together with relative pose measurements obtained from solving a relative perspective N-point problem (PNP), and the linear accelerations and angular velocities measured by the two inertial measurement units (IMU) which are rigidly attached to the cameras. Results obtained from extensive synthetic experiments demonstrate that our proposed framework is able to estimate highly accurate baseline transformations and depth maps.Comment: Accepted for publication in IEEE International Conference on Robotics and Automation (ICRA), 2018, Brisban

Visual Environment Assessment for Safe Autonomous Quadrotor Landing

openAutonomous identification and evaluation of safe landing zones are of paramount importance for ensuring the safety and effectiveness of aerial robots in the event of system failures, low battery, or the successful completion of specific tasks. In this thesis it is presented a novel approach, for detecting and assess potential landing sites for safe quadrotor landing. The proposed solution efficiently integrates both 2D and 3D environmental information and eliminates the need for external aids such as GPS and computationally intensive elevation maps. Semantic data derived from a Neural Network (NN), is combined with geometric data obtained from a disparity map, to extract environmental features and critical geometric attributes such as slope, flatness, and roughness. In particular, this method efficiently combines both metric and semantic information, making it also more robust, compared to other solutions that solely relies on one type of information only. Based on those attributes, several cost metrics are defined to evaluate safety, stability, and suitability of regions in the environments and identify the most suitable landing area. In this way we have a comprehensive evaluation of all the relevant aspects related to the safe site detection. This approach runs in real-time on quadrotors equipped with limited computational capabilities. Experimental results conducted in diverse environments demonstrate that the proposed method can effectively assess and identify suitable landing areas, enabling the safe and autonomous landing of a quadrotor in unknown environments.Autonomous identification and evaluation of safe landing zones are of paramount importance for ensuring the safety and effectiveness of aerial robots in the event of system failures, low battery, or the successful completion of specific tasks. In this thesis it is presented a novel approach, for detecting and assess potential landing sites for safe quadrotor landing. The proposed solution efficiently integrates both 2D and 3D environmental information and eliminates the need for external aids such as GPS and computationally intensive elevation maps. Semantic data derived from a Neural Network (NN), is combined with geometric data obtained from a disparity map, to extract environmental features and critical geometric attributes such as slope, flatness, and roughness. In particular, this method efficiently combines both metric and semantic information, making it also more robust, compared to other solutions that solely relies on one type of information only. Based on those attributes, several cost metrics are defined to evaluate safety, stability, and suitability of regions in the environments and identify the most suitable landing area. In this way we have a comprehensive evaluation of all the relevant aspects related to the safe site detection. This approach runs in real-time on quadrotors equipped with limited computational capabilities. Experimental results conducted in diverse environments demonstrate that the proposed method can effectively assess and identify suitable landing areas, enabling the safe and autonomous landing of a quadrotor in unknown environments

Studies on obstacle detection and path planning for a quadrotor system

Autonomous systems are one interesting topic recently investigated; for land and aerial vehicles; however, the main limitation of aerial vehicles is the weight to carry on-board, since the power consumed depends on this and hardware like sensors and processor is limited. The present thesis develops an application of digital image processing to detect obstacles using only a monocamera, there are some approaches but the present report wants to focus on the distance estimation approach that, in future works, can be combined with other methods since this approach is more general. The distance estimation approach uses feature detection algorithms in two consecutive images, matching them and thus estimate the obstacle position. The estimation is computed through a mathematical model of the camera and projections between those two images. There are many parameters to improve final results and the best parameters are found and tested with consecutive images, which were captured every 0.5m along a straight path of 5m. Fraunhofer position modules are tested with the entire algorithm. Finally, in order to establish the new path without obstacles, an optimal binary integer programming problem is proposed, adapting the approach using results obtained from the distance estimation and obstacle detection. Resulting data is suitable for combining them with information obtained from conventional sensors, such as ultrasonic sensors. The obtained mean error is between 1% and 12% in short distances (less than 2.5 m) and greater with longer distances. The complexity of this study lies in the use of a single camera for the capture of frontal images and obtaining 3D information of the environment, the computation of the obstacle detection algorithm is tested off-line and the path-planning algorithm is proposed with detected keypoints in the background.Autonome Systeme sind ein interessantes Thema vor kurzem untersucht; für Land- und Luftfahrzeuge; Allerdings ist die Hauptbegrenzung von Luftfahrzeugen das Gewicht, um an Bord zu tragen, da die verbrauchte Energie davon abhängt und Hardware wie Sensoren und Prozessor begrenzt ist. Die vorliegende Arbeit entwickelt eine Anwendung der digitalen Bildverarbeitung zur Erkennung von Hindernissen, die nur eine Monokamera verwenden, es gibt einige Ansätze, aber der vorliegende Bericht will sich auf den Abstandsschätzungsansatz konzentrieren, der in Zukunft mit anderen Methoden kombiniert werden kann, da dieser Ansatz ist allgemeiner. Der Abstandsschätzungsansatz verwendet Merkmalserkennungsalgorithmen in zwei aufeinanderfolgenden Bildern, passt sie an und schätzt somit die Hindernisposition ab. Die Schätzung wird durch ein mathematisches Modell der Kamera und Projektionen zwischen diesen beiden Bildern berechnet. Es gibt viele Parameter, um die endgültigen Ergebnisse zu verbessern, und die besten Parameter werden mit aufeinanderfolgenden Bildern gefunden und getestet, die alle 0,5 m auf einem geraden Weg von 5 m erfasst wurden. Fraunhofer-Positionsmodule werden mit dem gesamten Algorithmus getestet. Schließlich wird, um den neuen Weg ohne Hindernisse zu etablieren, ein optimales Binär-Integer-Programmierproblem vorgeschlagen, das den Ansatz unter Verwendung von Ergebnissen, die aus der Abstandsschätzung und der Hinderniserkennung erhalten wurden, anpasst. Die daraus resultierenden Daten eignen sich zur Kombination mit Informationen aus konventionellen Sensoren wie Ultraschallsensoren. Der erhaltene mittlere Fehler liegt zwischen 1 % und 12 % in kurzen Abständen (weniger als 2,5 m) und größer mit längeren Abständen. Die Komplexität dieser Studie liegt in der Verwendung einer einzigen Kamera für die Erfassung von Frontalbildern und dem Erhalten von 3D-Informationen der Umgebung, wird die Berechnung des Hinderniserfassungsalgorithmus off-line getestet und derWegplanungsalgorithmus wird mit erkannten Keypoints vorgeschlagen im Hintergrund.Tesi