Vision-based SLAM system for MAVs in GPS-denied environments

Using a camera, a micro aerial vehicle (MAV) can perform visual-based navigation in periods or circumstances when GPS is not available, or when it is partially available. In this context, the monocular simultaneous localization and mapping (SLAM) methods represent an excellent alternative, due to several limitations regarding to the design of the platform, mobility and payload capacity that impose considerable restrictions on the available computational and sensing resources of the MAV. However, the use of monocular vision introduces some technical difficulties as the impossibility of directly recovering the metric scale of the world. In this work, a novel monocular SLAM system with application to MAVs is proposed. The sensory input is taken from a monocular downward facing camera, an ultrasonic range finder and a barometer. The proposed method is based on the theoretical findings obtained from an observability analysis. Experimental results with real data confirm those theoretical findings and show that the proposed method is capable of providing good results with low-cost hardware.Peer ReviewedPostprint (published version

Monocular Visual Odometry for Fixed-Wing Small Unmanned Aircraft Systems

The popularity of small unmanned aircraft systems (SUAS) has exploded in recent years and seen increasing use in both commercial and military sectors. A key interest area for the military is to develop autonomous capabilities for these systems, of which navigation is a fundamental problem. Current navigation solutions suffer from a heavy reliance on a Global Positioning System (GPS). This dependency presents a significant limitation for military applications since many operations are conducted in environments where GPS signals are degraded or actively denied. Therefore, alternative navigation solutions without GPS must be developed and visual methods are one of the most promising approaches. A current visual navigation limitation is that much of the research has focused on developing and applying these algorithms on ground-based vehicles, small hand-held devices or multi-rotor SUAS. However, the Air Force has a need for fixed-wing SUAS to conduct extended operations. This research evaluates current state-of-the-art, open-source monocular visual odometry (VO) algorithms applied on fixed-wing SUAS flying at high altitudes under fast translation and rotation speeds. The algorithms tested are Semi-Direct VO (SVO), Direct Sparse Odometry (DSO), and ORB-SLAM2 (with loop closures disabled). Each algorithm is evaluated on a fixed-wing SUAS in simulation and real-world flight tests over Camp Atterbury, Indiana. Through these tests, ORB-SLAM2 is found to be the most robust and flexible algorithm under a variety of test conditions. However, all algorithms experience great difficulty maintaining localization in the collected real-world datasets, showing the limitations of using visual methods as the sole solution. Further study and development is required to fuse VO products with additional measurements to form a complete autonomous navigation solution

Design and Development of Aerial Robotic Systems for Sampling Operations in Industrial Environment

This chapter describes the development of an autonomous fluid sampling system for outdoor facilities, and the localization solution to be used. The automated sampling system will be based on collaborative robotics, with a team of a UAV and a UGV platform travelling through a plant to collect water samples. The architecture of the system is described, as well as the hardware present in the UAV and the different software frameworks used. A visual simultaneous localization and mapping (SLAM) technique is proposed to deal with the localization problem, based on authors’ previous works, including several innovations: a new method to initialize the scale using unreliable global positioning system (GPS) measurements, integration of attitude and heading reference system (AHRS) measurements into the recursive state estimation, and a new technique to track features during the delayed feature initialization process. These procedures greatly enhance the robustness and usability of the SLAM technique as they remove the requirement of assisted scale initialization, and they reduce the computational effort to initialize features. To conclude, results from experiments performed with simulated data and real data captured with a prototype UAV are presented and discussed

Robust, Resilient Networked Communication in Challenged Environments

In challenged environments, digital communication infrastructure may be difficult or even impossible to access. This is especially true in rural and developing regions, as well as in any region during a time of political or environmental crisis. We advance the state of the art in wireless networking and security to design networks and applications that rapidly assess changing networking conditions to restore communication and provide local situational awareness. This dissertation examines new systems for responding to current and emerging needs for wireless networks. This work looks across the wireless ecosystem of widely deployed standards. We develop new tools to improve network assessment and to provide robust and reliable network communication. By incorporating new technological breakthroughs, such as the wide commercial success of Unmanned Aircraft Systems (UAS), we introduce novel methods and systems for existing wireless standards for these challenged networks. We assess how existing technologies and standards function in difficult environments: lacking end-end Internet connectivity, experiencing overload or other resource constraints, and operating in three dimensional space. Through this lens, we demonstrate how to optimize networks to serve marginalized communities outside of first world urban cities and make our networks resilient to natural and political crisis that threaten communication

UAV or Drones for Remote Sensing Applications in GPS/GNSS Enabled and GPS/GNSS Denied Environments

The design of novel UAV systems and the use of UAV platforms integrated with robotic sensing and imaging techniques, as well as the development of processing workflows and the capacity of ultra-high temporal and spatial resolution data, have enabled a rapid uptake of UAVs and drones across several industries and application domains.This book provides a forum for high-quality peer-reviewed papers that broaden awareness and understanding of single- and multiple-UAV developments for remote sensing applications, and associated developments in sensor technology, data processing and communications, and UAV system design and sensing capabilities in GPS-enabled and, more broadly, Global Navigation Satellite System (GNSS)-enabled and GPS/GNSS-denied environments.Contributions include:UAV-based photogrammetry, laser scanning, multispectral imaging, hyperspectral imaging, and thermal imaging;UAV sensor applications; spatial ecology; pest detection; reef; forestry; volcanology; precision agriculture wildlife species tracking; search and rescue; target tracking; atmosphere monitoring; chemical, biological, and natural disaster phenomena; fire prevention, flood prevention; volcanic monitoring; pollution monitoring; microclimates; and land use;Wildlife and target detection and recognition from UAV imagery using deep learning and machine learning techniques;UAV-based change detection

Monocular SLAM system for MAVs aided with altitude and range measurements: a GPS-free approach

A typical navigation system for a Micro Aerial Vehicle (MAV) relies basically on GPS for position estimation. However,for several kinds of applications, the precision of the GPS is inappropriate or even its signal can be unavailable. In this context, and due to its flexibility, Monocular Simultaneous Localization and Mapping (SLAM) methods have become a good alternative for implementing visual-based navigation systems for MAVs that must operate in GPS-denied environments. On the other hand, one of the most important challenges that arises with the use of the monocular vision is the difficulty to recover the metric scale of the world. In this work, a monocular SLAM system for MAVs is presented. In order to overcome the problem of the metric scale, a novel technique for inferring the approximate depth of visual features from an ultrasonic range-finder is developed. Additionally, the altitude of the vehicle is updated using the pressure measurements of a barometer. The proposed approach is supported by the theoretical results obtained from a nonlinear observability test. Experiments performed with both computer simulations and real data are presented in order to validate the performance of the proposal. The results confirm the theoretical findings and show that the method is able to work with low-cost sensors.Peer ReviewedPostprint (author's final draft

Automatic Pipeline Surveillance Air-Vehicle

This thesis presents the developments of a vision-based system for aerial pipeline Right-of-Way surveillance using optical/Infrared sensors mounted on Unmanned Aerial Vehicles (UAV). The aim of research is to develop a highly automated, on-board system for detecting and following the pipelines; while simultaneously detecting any third-party interference. The proposed approach of using a UAV platform could potentially reduce the cost of monitoring and surveying pipelines when compared to manned aircraft. The main contributions of this thesis are the development of the image-analysis algorithms, the overall system architecture and validation of in hardware based on scaled down Test environment. To evaluate the performance of the system, the algorithms were coded using Python programming language. A small-scale test-rig of the pipeline structure, as well as expected third-party interference, was setup to simulate the operational environment and capture/record data for the algorithm testing and validation. The pipeline endpoints are identified by transforming the 16-bits depth data of the explored environment into 3D point clouds world coordinates. Then, using the Random Sample Consensus (RANSAC) approach, the foreground and background are separated based on the transformed 3D point cloud to extract the plane that corresponds to the ground. Simultaneously, the boundaries of the explored environment are detected based on the 16-bit depth data using a canny detector. Following that, these boundaries were filtered out, after being transformed into a 3D point cloud, based on the real height of the pipeline for fast and accurate measurements using a Euclidean distance of each boundary point, relative to the plane of the ground extracted previously. The filtered boundaries were used to detect the straight lines of the object boundary (Hough lines), once transformed into 16-bit depth data, using a Hough transform method. The pipeline is verified by estimating a centre line segment, using a 3D point cloud of each pair of the Hough line segments, (transformed into 3D). Then, the corresponding linearity of the pipeline points cloud is filtered within the width of the pipeline using Euclidean distance in the foreground point cloud. Then, the segment length of the detected centre line is enhanced to match the exact pipeline segment by extending it along the filtered point cloud of the pipeline. The third-party interference is detected based on four parameters, namely: foreground depth data; pipeline depth data; pipeline endpoints location in the 3D point cloud; and Right-of-Way distance. The techniques include detection, classification, and localization algorithms. Finally, a waypoints-based navigation system was implemented for the air- vehicle to fly over the course waypoints that were generated online by a heading angle demand to follow the pipeline structure in real-time based on the online identification of the pipeline endpoints relative to a camera frame

Autonomous Navigation in Complex Indoor and Outdoor Environments with Micro Aerial Vehicles

Micro aerial vehicles (MAVs) are ideal platforms for surveillance and search and rescue in confined indoor and outdoor environments due to their small size, superior mobility, and hover capability. In such missions, it is essential that the MAV is capable of autonomous flight to minimize operator workload. Despite recent successes in commercialization of GPS-based autonomous MAVs, autonomous navigation in complex and possibly GPS-denied environments gives rise to challenging engineering problems that require an integrated approach to perception, estimation, planning, control, and high level situational awareness. Among these, state estimation is the first and most critical component for autonomous flight, especially because of the inherently fast dynamics of MAVs and the possibly unknown environmental conditions. In this thesis, we present methodologies and system designs, with a focus on state estimation, that enable a light-weight off-the-shelf quadrotor MAV to autonomously navigate complex unknown indoor and outdoor environments using only onboard sensing and computation. We start by developing laser and vision-based state estimation methodologies for indoor autonomous flight. We then investigate fusion from heterogeneous sensors to improve robustness and enable operations in complex indoor and outdoor environments. We further propose estimation algorithms for on-the-fly initialization and online failure recovery. Finally, we present planning, control, and environment coverage strategies for integrated high-level autonomy behaviors. Extensive online experimental results are presented throughout the thesis. We conclude by proposing future research opportunities

Merging Unmanned Aerial System and Laser Scanning techniques for high resolution 3D modelling of Koutouki Cave, Attica

Η επιστήμη της τηλεανίχνευσης και η τεχνολογία σάρωσης αναγλύφου με λέιζερ μας έδωσαν την ευκαιρία να μελετήσουμε κλειστούς χώρους και περιβάλλοντα όπως τα σπήλαια με τη σύνθετη και μοναδική μορφολογία τους. Στόχος της παρούσας μεταπτυχιακής διατριβής είναι η δημιουργία ενός ολοκληρωμένου τρισδιάστατου μοντέλου του σπηλαίου Κουτούκι στην Παιανία, η ποσοτική ανάλυση των γεωμορφών που συνθέτουν το σπήλαιο και το πάχος των υπερκειμένων στρωμάτων πάνω από το σπήλαιο. Χρησιμοποιήσαμε ένα laser scanner χειρός για την απόκτηση 80.000.000 σημείων με πραγματικές συντεταγμένες (X, Y, Z) για την αποτύπωση ολόκληρου του σπηλαίου συμπεριλαμβανομένων των μικρότερων διαδρομών και των σκοτεινών τμημάτων. Το νέφος σημείων αποτελείται από το δάπεδο, τα τοιχώματα και την οροφή του σπηλαίου, καθώς και τους σταλακτίτες, τους σταλαγμίτες και τις κολόνες που αποτελούν τη διακόσμηση του σπηλαίου. Η απόλυτη και ακριβής τοποθέτηση του νέφους σημείων μέσα σε ένα σύστημα αναφοράς μας δίνει την ευκαιρία για τρισδιάστατες μετρήσεις και την λεπτομερή απεικόνιση των γεωμορφών. Δημιουργώντας το ψηφιακό μοντέλο αναγλύφου του δαπέδου του σπηλαίου, εντοπίσαμε 55 κολόνες όπου με στατιστική ανάλυση μπορέσαμε να τις συσχετίσουμε με το πλαίσιο της ανάπτυξης του σπηλαίου. Οι παράμετροι που προκύπτουν είναι οι ισοϋψείς του σπηλαίου, το ύψος, η γεωμετρία και ο όγκος της κάθε κολόνας, καθώς και ο όγκος της κοιλότητας του ανοιχτού χώρου. Επιπλέον, με τη χρήση μη επανδρωμένου αεροσκάφους και εφαρμόζοντας μια μεθοδολογία βασισμένη στη φωτογραμμετρική επεξεργασία των δεδομένων εικόνας, πραγματοποιήθηκε η σάρωση του αναγλύφου πάνω από το σπήλαιο που μας οδήγησε στην παραγωγή ενός ψηφιακού μοντέλου αναγλύφου του πρανούς. Το τελικό προϊόν είναι ένα επίπεδο πληροφορίας υψηλής ανάλυσης με τις μετρήσεις του πάχους των υπερκειμένων στρωμάτων του σπηλαίου καθώς και τις τοπογραφίας με υψηλή ακρίβεια. Υποστηρίζουμε ότι, με την αποδεδειγμένη μεθοδολογία, είναι δυνατόν να εντοπίσουμε με μεγάλη λεπτομέρεια και ακρίβεια τα γεωμορφολογικά χαρακτηριστικά ενός σπηλαίου, να κάνουμε εκτιμήσεις για τη σπηλαιογένεση ενός σπηλαίου και να παρακολουθήσουμε την εξέλιξη ενός καρστικού συστήματος.Remote sensing techniques and laser scanning technology have given us the opportunity to study indoor environments such as caves with their complex and unique morphology. The objective of this Msc thesis is the generation of a complete 3D model of the Koutouki Cave in Peania, Greece, the quantification analysis of the subsurface structures that consists the cave, and the thickness of the bedding between the cave and the surface. We used a handheld laser scanner for acquiring 80,000,000 points with projected coordinate information (X, Y, Z) covering the entire show cave of Koutouki, including its hidden passages and dark corners. The point cloud covers the floor, the walls and the roof of the cave, as well as the stalactites, stalagmites and the connected columns that constitute the decoration of the cave. The absolute and exact placement of the point cloud within a geographic reference frame gives us the opportunity for three-dimensional measurements and detailed visualization of the subsurface structures. Using open - source software, we managed to make a quantification analysis of the terrain and generate morphological and geometric features of the speleothems. We identified 55 columns by using digital terrain analysis and processed them statistically in order to correlate them to the frame of the cave development. The parameters that derived are the contours, each column height, the speleothem geometry and volume, as well as the volume of the open space cavity. Furthermore, we applied a methodology based on photogrammetric processing of Unmanned Aerial System image data which led us to the production of a digital terrain model of the open-air surface above the cave. The final product is a high-resolution information layer with measurements of the rock thickness between the roof of the underground karstic structure and the open-surface topography with high accuracy. We argue that, by the demonstrated methodology, it is possible to identify with high accuracy and detail the geomorphological features of a cave, estimate the speleogenesis and monitor the evolution of a karstic system