Vision based navigation in a dynamic environment

Cette thèse s'intéresse au problème de la navigation autonome au long cours de robots mobiles à roues dans des environnements dynamiques. Elle s'inscrit dans le cadre du projet FUI Air-Cobot. Ce projet, porté par Akka Technologies, a vu collaborer plusieurs entreprises (Akka, Airbus, 2MORROW, Sterela) ainsi que deux laboratoires de recherche, le LAAS et Mines Albi. L'objectif est de développer un robot collaboratif (ou cobot) capable de réaliser l'inspection d'un avion avant le décollage ou en hangar. Différents aspects ont donc été abordés : le contrôle non destructif, la stratégie de navigation, le développement du système robotisé et de son instrumentation, etc. Cette thèse répond au second problème évoqué, celui de la navigation. L'environnement considéré étant aéroportuaire, il est hautement structuré et répond à des normes de déplacement très strictes (zones interdites, etc.). Il peut être encombré d'obstacles statiques (attendus ou non) et dynamiques (véhicules divers, piétons, ...) qu'il conviendra d'éviter pour garantir la sécurité des biens et des personnes. Cette thèse présente deux contributions. La première porte sur la synthèse d'un asservissement visuel permettant au robot de se déplacer sur de longues distances (autour de l'avion ou en hangar) grâce à une carte topologique et au choix de cibles dédiées. De plus, cet asservissement visuel exploite les informations fournies par toutes les caméras embarquées. La seconde contribution porte sur la sécurité et l'évitement d'obstacles. Une loi de commande basée sur les spirales équiangulaires exploite seulement les données sensorielles fournies par les lasers embarqués. Elle est donc purement référencée capteur et permet de contourner tout obstacle, qu'il soit fixe ou mobile. Il s'agit donc d'une solution générale permettant de garantir la non collision. Enfin, des résultats expérimentaux, réalisés au LAAS et sur le site d'Airbus à Blagnac, montrent l'efficacité de la stratégie développée.This thesis is directed towards the autonomous long range navigation of wheeled robots in dynamic environments. It takes place within the Air-Cobot project. This project aims at designing a collaborative robot (cobot) able to perform the preflight inspection of an aircraft. The considered environment is then highly structured (airport runway and hangars) and may be cluttered with both static and dynamic unknown obstacles (luggage or refueling trucks, pedestrians, etc.). Our navigation framework relies on previous works and is based on the switching between different control laws (go to goal controller, visual servoing, obstacle avoidance) depending on the context. Our contribution is twofold. First of all, we have designed a visual servoing controller able to make the robot move over a long distance thanks to a topological map and to the choice of suitable targets. In addition, multi-camera visual servoing control laws have been built to benefit from the image data provided by the different cameras which are embedded on the Air-Cobot system. The second contribution is related to obstacle avoidance. A control law based on equiangular spirals has been designed to guarantee non collision. This control law, based on equiangular spirals, is fully sensor-based, and allows to avoid static and dynamic obstacles alike. It then provides a general solution to deal efficiently with the collision problem. Experimental results, performed both in LAAS and in Airbus hangars and runways, show the efficiency of the developed techniques

Détection d'amers visuels pour la navigation d'un robot autonome autour d'un avion et son inspection

National audienceThis article discusses the detection of visual features for the navigation of the platform Air-Cobot around an aircraft. This autonomous mobile and collaborative robot is dedicated to the inspection of airplane. A new visual detection and inspection approach is proposed.Cet article traite de la détection d'amers visuels pour la navigation de la plateforme robotique Air-Cobot autour d'un avion. Ce robot mobile autonome et collaboratif est dédié à l'inspection des aéronefs. Une nouvelle méthode de détection et d'inspection visuelle est proposée

Localisation à partir de données laser d'un robot naviguant autour d'un avion

National audienceThis article discusses the pose estimation of the mobile platform Air-Cobot relative to the aircraft around which it operates. Autonomous and collaborative, this robot inspects aircrafts. It is equipped with distance sensors laser scans. The presented localization methods have been successfully tested in a real environment.Cet article traite du calcul de la pose de la plateforme mobile Air-Cobot par rapport à l'avion autour duquel elle évolue. Autonome et collaboratif, ce robot inspecte des aéronefs. Il est équipé de capteurs de distance à balayage laser. Les méthodes de localisation présentées ont été testées avec succès en environnement réel

INTELLIGENT VISION-BASED NAVIGATION SYSTEM

This thesis presents a complete vision-based navigation system that can plan and follow an obstacle-avoiding path to a desired destination on the basis of an internal map updated with information gathered from its visual sensor. For vision-based self-localization, the system uses new floor-edges-specific filters for detecting floor edges and their pose, a new algorithm for determining the orientation of the robot, and a new procedure for selecting the initial positions in the self-localization procedure. Self-localization is based on matching visually detected features with those stored in a prior map. For planning, the system demonstrates for the first time a real-world application of the neural-resistive grid method to robot navigation. The neural-resistive grid is modified with a new connectivity scheme that allows the representation of the collision-free space of a robot with finite dimensions via divergent connections between the spatial memory layer and the neuro-resistive grid layer. A new control system is proposed. It uses a Smith Predictor architecture that has been modified for navigation applications and for intermittent delayed feedback typical of artificial vision. A receding horizon control strategy is implemented using Normalised Radial Basis Function nets as path encoders, to ensure continuous motion during the delay between measurements. The system is tested in a simplified environment where an obstacle placed anywhere is detected visually and is integrated in the path planning process. The results show the validity of the control concept and the crucial importance of a robust vision-based self-localization process

A NOVEL APPROACH FOR DETECTION FAULT IN THE AIRCRAFT EXTERIOR BODY USING IMAGE PROCESSING

The primary objective of this thesis is to develop innovative techniques for the inspection and maintenance of aircraft structures. We aim to streamline the entire process by utilizing images to detect potential defects in the aircraft body and comparing them to properly functioning images of the aircraft. This enables us to determine whether a specific section of the aircraft is faulty or not. We achieve this by employing image processing to train a model capable of identifying faulty images. The image processing methodology we use involves the use of images of both defective and operational parts of the aircraft\u27s exterior. These images undergo a preprocessing phase that preserves valuable details. During the training period, a new image of the same section of the aircraft is used to validate the model. After processing, the algorithm grades the image as faulty or normal. To facilitate our study, we rely on the Convolutional Neural Network (CNN) approach. This technique collects distinguishing features from a single patch created by the frame segmentation of a CNN kernel. Furthermore, we use various filters to process the images using the image processing toolbox available in Python. In our initial trials, we observed that the CNN model struggled with the overfitting of the faulty class. To address this, we applied image augmentation by converting a small dataset of 87 images to an augmented dataset of 4000 images. After passing the data through multiple convolutional layers and executing multiple epochs, our proposed model achieved an impressive training accuracy of 98.28%. In addition, we designed a GUI-based interface that allows users to input an image and view the results in terms of faulty or normal. Finally, we propose that the application of this research in the field of robotics would be an ideal area for future work

Aerial Vehicles

This book contains 35 chapters written by experts in developing techniques for making aerial vehicles more intelligent, more reliable, more flexible in use, and safer in operation.It will also serve as an inspiration for further improvement of the design and application of aeral vehicles. The advanced techniques and research described here may also be applicable to other high-tech areas such as robotics, avionics, vetronics, and space

Underwater Vehicles

For the latest twenty to thirty years, a significant number of AUVs has been created for the solving of wide spectrum of scientific and applied tasks of ocean development and research. For the short time period the AUVs have shown the efficiency at performance of complex search and inspection works and opened a number of new important applications. Initially the information about AUVs had mainly review-advertising character but now more attention is paid to practical achievements, problems and systems technologies. AUVs are losing their prototype status and have become a fully operational, reliable and effective tool and modern multi-purpose AUVs represent the new class of underwater robotic objects with inherent tasks and practical applications, particular features of technology, systems structure and functional properties

Basic set of behaviours for programming assembly robots

We know from the well established Church-Turing thesis that any computer program­ming language needs just a limited set of commands in order to perform any computable process. However, programming in these terms is so very inconvenient that a larger set of machine codes need to be introduced and on top of these higher programming languages are erected.In Assembly Robotics we could theoretically formulate any assembly task, in terms of moves. Nevertheless, it is as tedious and error prone to program assemblies at this low level as it would be to program a computer by using just Turing Machine commands.An interesting survey carried out in the beginning of the nineties showed that the most common assembly operations in manufacturing industry cluster in just seven classes. Since the research conducted in this thesis is developed within the behaviour-based assembly paradigm which views every assembly task as the external manifestation of the execution of a behavioural module, we wonder whether there exists a limited and ergonomical set of elementary modules with which to program at least 80% of the most common operations.IIn order to investigate such a problem, we set a project in which, taking into account the statistics of the aforementioned survey, we analyze the experimental behavioural decomposition of three significant assembly tasks (two similar benchmarks, the STRASS assembly, and a family of torches). From these three we establish a basic set of such modules.The three test assemblies with which we ran the experiments can not possibly exhaust ah the manufacturing assembly tasks occurring in industry, nor can the results gathered or the speculations made represent a theoretical proof of the existence of the basic set. They simply show that it is possible to formulate different assembly tasks in terms of a small set of about 10 modules, which may be regarded as an embryo of a basic set of elementary modules.Comparing this set with Kondoleon’s tasks and with Balch’s general-purpose robot routines, we observed that ours was general enough to represent 80% of the most com­mon manufacturing assembly tasks and ergonomical enough to be easily used by human operators or automatic planners. A final discussion shows that it would be possible to base an assembly programming language on this kind of set of basic behavioural modules

Wide-Area Surveillance System using a UAV Helicopter Interceptor and Sensor Placement Planning Techniques

This project proposes and describes the implementation of a wide-area surveillance system comprised of a sensor/interceptor placement planning and an interceptor unmanned aerial vehicle (UAV) helicopter. Given the 2-D layout of an area, the planning system optimally places perimeter cameras based on maximum coverage and minimal cost. Part of this planning system includes the MATLAB implementation of Erdem and Sclaroff’s Radial Sweep algorithm for visibility polygon generation. Additionally, 2-D camera modeling is proposed for both fixed and PTZ cases. Finally, the interceptor is also placed to minimize shortest-path flight time to any point on the perimeter during a detection event. Secondly, a basic flight control system for the UAV helicopter is designed and implemented. The flight control system’s primary goal is to hover the helicopter in place when a human operator holds an automatic-flight switch. This system represents the first step in a complete waypoint-navigation flight control system. The flight control system is based on an inertial measurement unit (IMU) and a proportional-integral-derivative (PID) controller. This system is implemented using a general-purpose personal computer (GPPC) running Windows XP and other commercial off-the-shelf (COTS) hardware. This setup differs from other helicopter control systems which typically use custom embedded solutions or micro-controllers. Experiments demonstrate the sensor placement planning achieving \u3e90% coverage at optimized-cost for several typical areas given multiple camera types and parameters. Furthermore, the helicopter flight control system experiments achieve hovering success over short flight periods. However, the final conclusion is that the COTS IMU is insufficient for high-speed, high-frequency applications such as a helicopter control system