Automated 3D scene reconstruction from open geospatial data sources: airborne laser scanning and a 2D topographic database

Open geospatial data sources provide opportunities for low cost 3D scene reconstruction. In this study, based on a sparse airborne laser scanning (ALS) point cloud (0.8 points/m2) obtained from open source databases, a building reconstruction pipeline for CAD building models was developed. The pipeline includes voxel-based roof patch segmentation, extraction of the key-points representing the roof patch outline, step edge identification and adjustment, and CAD building model generation. The advantages of our method lie in generating CAD building models without the step of enforcing the edges to be parallel or building regularization. Furthermore, although it has been challenging to use sparse datasets for 3D building reconstruction, our result demonstrates the great potential in such applications. In this paper, we also investigated the applicability of open geospatial datasets for 3D road detection and reconstruction. Road central lines were acquired from an open source 2D topographic database. ALS data were utilized to obtain the height and width of the road. A constrained search method (CSM) was developed for road width detection. The CSM method was conducted by splitting a given road into patches according to height and direction criteria. The road edges were detected patch by patch. The road width was determined by the average distance from the edge points to the central line. As a result, 3D roads were reconstructed from ALS and a topographic database

Fruit Detection and Tree Segmentation for Yield Mapping in Orchards

Accurate information gathering and processing is critical for precision horticulture, as growers aim to optimise their farm management practices. An accurate inventory of the crop that details its spatial distribution along with health and maturity, can help farmers efficiently target processes such as chemical and fertiliser spraying, crop thinning, harvest management, labour planning and marketing. Growers have traditionally obtained this information by using manual sampling techniques, which tend to be labour intensive, spatially sparse, expensive, inaccurate and prone to subjective biases. Recent advances in sensing and automation for field robotics allow for key measurements to be made for individual plants throughout an orchard in a timely and accurate manner. Farmer operated machines or unmanned robotic platforms can be equipped with a range of sensors to capture a detailed representation over large areas. Robust and accurate data processing techniques are therefore required to extract high level information needed by the grower to support precision farming. This thesis focuses on yield mapping in orchards using image and light detection and ranging (LiDAR) data captured using an unmanned ground vehicle (UGV). The contribution is the framework and algorithmic components for orchard mapping and yield estimation that is applicable to different fruit types and orchard configurations. The framework includes detection of fruits in individual images and tracking them over subsequent frames. The fruit counts are then associated to individual trees, which are segmented from image and LiDAR data, resulting in a structured spatial representation of yield. The first contribution of this thesis is the development of a generic and robust fruit detection algorithm. Images captured in the outdoor environment are susceptible to highly variable external factors that lead to significant appearance variations. Specifically in orchards, variability is caused by changes in illumination, target pose, tree types, etc. The proposed techniques address these issues by using state-of-the-art feature learning approaches for image classification, while investigating the utility of orchard domain knowledge for fruit detection. Detection is performed using both pixel-wise classification of images followed instance segmentation, and bounding-box regression approaches. The experimental results illustrate the versatility of complex deep learning approaches over a multitude of fruit types. The second contribution of this thesis is a tree segmentation approach to detect the individual trees that serve as a standard unit for structured orchard information systems. The work focuses on trellised trees, which present unique challenges for segmentation algorithms due to their intertwined nature. LiDAR data are used to segment the trellis face, and to generate proposals for individual trees trunks. Additional trunk proposals are provided using pixel-wise classification of the image data. The multi-modal observations are fine-tuned by modelling trunk locations using a hidden semi-Markov model (HSMM), within which prior knowledge of tree spacing is incorporated. The final component of this thesis addresses the visual occlusion of fruit within geometrically complex canopies by using a multi-view detection and tracking approach. Single image fruit detections are tracked over a sequence of images, and associated to individual trees or farm rows, with the spatial distribution of the fruit counting forming a yield map over the farm. The results show the advantage of using multi-view imagery (instead of single view analysis) for fruit counting and yield mapping. This thesis includes extensive experimentation in almond, apple and mango orchards, with data captured by a UGV spanning a total of 5 hectares of farm area, over 30 km of vehicle traversal and more than 7,000 trees. The validation of the different processes is performed using manual annotations, which includes fruit and tree locations in image and LiDAR data respectively. Additional evaluation of yield mapping is performed by comparison against fruit counts on trees at the farm and counts made by the growers post-harvest. The framework developed in this thesis is demonstrated to be accurate compared to ground truth at all scales of the pipeline, including fruit detection and tree mapping, leading to accurate yield estimation, per tree and per row, for the different crops. Through the multitude of field experiments conducted over multiple seasons and years, the thesis presents key practical insights necessary for commercial development of an information gathering system in orchards

New innovations in pavement materials and engineering: A review on pavement engineering research 2021

Sustainable and resilient pavement infrastructure is critical for current economic and environmental challenges. In the past 10 years, the pavement infrastructure strongly supports the rapid development of the global social economy. New theories, new methods, new technologies and new materials related to pavement engineering are emerging. Deterioration of pavement infrastructure is a typical multi-physics problem. Because of actual coupled behaviors of traffic and environmental conditions, predictions of pavement service life become more and more complicated and require a deep knowledge of pavement material analysis. In order to summarize the current and determine the future research of pavement engineering, Journal of Traffic and Transportation Engineering (English Edition) has launched a review paper on the topic of “New innovations in pavement materials and engineering: A review on pavement engineering research 2021”. Based on the joint-effort of 43 scholars from 24 well-known universities in highway engineering, this review paper systematically analyzes the research status and future development direction of 5 major fields of pavement engineering in the world. The content includes asphalt binder performance and modeling, mixture performance and modeling of pavement materials, multi-scale mechanics, green and sustainable pavement, and intelligent pavement. Overall, this review paper is able to provide references and insights for researchers and engineers in the field of pavement engineering

Location-Based Sensor Fusion for UAS Urban Navigation.

For unmanned aircraft systems (UAS) to effectively conduct missions in urban environments, a multi-sensor navigation scheme must be developed that can operate in areas with degraded Global Positioning System (GPS) signals. This thesis proposes a sensor fusion plug and play capability for UAS navigation in urban environments to test combinations of sensors. Measurements are fused using both the Extended Kalman Filter (EKF) and Ensemble Kalman Filter (EnKF), a type of Particle Filter. A Long Term Evolution (LTE) transceiver and computer vision sensor each augment the traditional GPS receiver, inertial sensors, and air data system. Availability and accuracy information for each sensor is extracted from the literature. LTE positioning is motivated by a perpetually expanding network that can provide persistent measurements in the urban environment. A location-based logic model is proposed to predict sensor availability and accuracy for a given type of urban environment based on a map database as well as real-time sensor inputs and filter outputs. The simulation is executed in MATLAB where the vehicle dynamics, environment, sensors, and filters are user-customizable. Results indicate that UAS horizontal position accuracy is most dependent on availability of high sampling rate position measurements along with GPS measurement availability. Since the simulation is able to accept LTE sensor specifications, it will be able to show how the UAS position accuracy can be improved in the future with this persistent measurement, even though the accuracy is not improved using current LTE state-of-the-art. In the unmatched true propagation and filter dynamics model scenario, filter tuning proves to be difficult as GPS availability varies from urban canyon to urban canyon. The main contribution of this thesis is the generation of accuracy data for different sensor suites in both a homogeneous urban environment (solid walls) using matched dynamics models and a heterogeneous urban environment layout using unmatched models that necessitate filter tuning. Future work should explore the use of downward facing VISION sensors and LiDAR, integrate real-time map information into sensor availability and measurement weighting decisions, including the use of LTE for approximate localization, and more finely represent expected measurement accuracies in the GPS and LTE networks.PhDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/110361/1/jrufa_1.pd

Probabilistic Maneuver Recognition in Traffic Scenarios

In this work an approach is presented to model and recognize traffic maneuvers in terms of interactions between different traffic participants on extra urban roads. Results of the recognition concept are presented and evaluated using different sensor setups and its benefit is outlined by an integration into a software framework in the field of Car-to-Car (C2C) communications. Furthermore, recognition results are used in this work to robustly predict vehicle’s trajectories while driving dynami

New innovations in pavement materials and engineering: A review on pavement engineering research 2021

Sustainable and resilient pavement infrastructure is critical for current economic and environmental challenges. In the past 10 years, the pavement infrastructure strongly supports the rapid development of the global social economy. New theories, new methods, new technologies and new materials related to pavement engineering are emerging. Deterioration of pavement infrastructure is a typical multi-physics problem. Because of actual coupled behaviors of traffic and environmental conditions, predictions of pavement service life become more and more complicated and require a deep knowledge of pavement material analysis. In order to summarize the current and determine the future research of pavement engineering, Journal of Traffic and Transportation Engineering (English Edition) has launched a review paper on the topic of “New innovations in pavement materials and engineering: A review on pavement engineering research 2021”. Based on the joint-effort of 43 scholars from 24 well-known universities in highway engineering, this review paper systematically analyzes the research status and future development direction of 5 major fields of pavement engineering in the world. The content includes asphalt binder performance and modeling, mixture performance and modeling of pavement materials, multi-scale mechanics, green and sustainable pavement, and intelligent pavement. Overall, this review paper is able to provide references and insights for researchers and engineers in the field of pavement engineering

Planning and Navigation in Dynamic Environments for Mobile Robots and Micro Aerial Vehicles

Reliable and robust navigation planning and obstacle avoidance is key for the autonomous operation of mobile robots. In contrast to stationary industrial robots that often operate in controlled spaces, planning for mobile robots has to take changing environments and uncertainties into account during plan execution. In this thesis, planning and obstacle avoidance techniques are proposed for a variety of ground and aerial robots. Common to most of the presented approaches is the exploitation of the nature of the underlying problem to achieve short planning times by using multiresolution or hierarchical approaches. Short planning times allow for continuous and fast replanning to take the uncertainty in the environment and robot motion execution into account. The proposed approaches are evaluated in simulation and real-world experiments. The first part of this thesis addresses planning for mobile ground robots. One contribution is an approach to grasp and object removal planning to pick objects from a transport box with a mobile manipulation robot. In a multistage process, infeasible grasps are pruned in offline and online processing steps. Collision-free endeffector trajectories are planned to the remaining grasps until a valid removal trajectory can be found. An object-centric local multiresolution representation accelerates trajectory planning. The mobile manipulation components are evaluated in an integrated mobile bin-picking system. Local multiresolution planning is employed for path planning for humanoid soccer robots as well. The used Nao robot is equipped with only relatively low computing power. A resource-efficient path planner including the anticipated movements of opponents on the field is developed as part of this thesis. In soccer games an important subproblem is to reach a position behind the ball to dribble or kick it towards the goal. By the assumption that the opponents have the same intention, an explicit representation of their movements is possible. This leads to paths that facilitate the robot to reach its target position with a higher probability without being disturbed by the other robot. The evaluation for the planner is performed in a physics-based soccer simulation. The second part of this thesis covers planning and obstacle avoidance for micro aerial vehicles (MAVs), in particular multirotors. To reduce the planning complexity, the planning problem is split into a hierarchy of planners running on different levels of abstraction, i.e., from abstract to detailed environment descriptions and from coarse to fine plans. A complete planning hierarchy for MAVs is presented, from mission planners for multiple application domains to low-level obstacle avoidance. Missions planned on the top layer are executed by means of coupled allocentric and egocentric path planning. Planning is accelerated by global and local multiresolution representations. The planners can take multiple objectives into account in addition to obstacle costs and path length, e.g., sensor constraints. The path planners are supplemented by trajectory optimization to achieve dynamically feasible trajectories that can be executed by the underlying controller at higher velocities. With the initialization techniques presented in this thesis, the convergence of the optimization problem is expedited. Furthermore, frequent reoptimization of the initial trajectory allows for the reaction to changes in the environment without planning and optimizing a complete new trajectory. Fast, reactive obstacle avoidance based on artificial potential fields acts as a safety layer in the presented hierarchy. The obstacle avoidance layer employs egocentric sensor data and can operate at the data acquisition frequency of up to 40 Hz. It can slow-down and stop the MAVs in front of obstacles as well as avoid approaching dynamic obstacles. We evaluate our planning and navigation hierarchy in simulation and with a variety of MAVs in real-world applications, especially outdoor mapping missions, chimney and building inspection, and automated stocktaking.Planung und Navigation in dynamischen Umgebungen für mobile Roboter und Multikopter Zuverlässige und sichere Navigationsplanung und Hindernisvermeidung ist ein wichtiger Baustein für den autonomen Einsatz mobiler Roboter. Im Gegensatz zu klassischen Industrierobotern, die in der Regel in abgetrennten, kontrollierten Bereichen betrieben werden, ist es in der mobilen Robotik unerlässlich, Änderungen in der Umgebung und die Unsicherheit bei der Aktionsausführung zu berücksichtigen. Im Rahmen dieser Dissertation werden Verfahren zur Planung und Hindernisvermeidung für eine Reihe unterschiedlicher Boden- und Flugroboter entwickelt und vorgestellt. Den meisten beschriebenen Ansätzen ist gemein, dass die Struktur der zu lösenden Probleme ausgenutzt wird, um Planungsprozesse zu beschleunigen. Häufig ist es möglich, mit abnehmender Genauigkeit zu planen desto weiter eine Aktion in der Zeit oder im Ort entfernt ist. Dieser Ansatz wird lokale Multiresolution genannt. In anderen Fällen ist eine Zerlegung des Problems in Schichten unterschiedlicher Genauigkeit möglich. Die damit zu erreichende Beschleunigung der Planung ermöglicht ein häufiges Neuplanen und somit die Reaktion auf Änderungen in der Umgebung und Abweichungen bei den ausgeführten Aktionen. Zur Evaluation der vorgestellten Ansätze werden Experimente sowohl in der Simulation als auch mit Robotern durchgeführt. Der erste Teil dieser Dissertation behandelt Planungsmethoden für mobile Bodenroboter. Um Objekte mit einem mobilen Roboter aus einer Transportkiste zu greifen und zur Weiterverarbeitung zu einem Arbeitsplatz zu liefern, wurde ein System zur Planung möglicher Greifposen und hindernisfreier Endeffektorbahnen entwickelt. In einem mehrstufigen Prozess werden mögliche Griffe an bekannten Objekten erst in mehreren Vorverarbeitungsschritten (offline) und anschließend, passend zu den erfassten Objekten, online identifiziert. Zu den verbleibenden möglichen Griffen werden Endeffektorbahnen geplant und, bei Erfolg, ausgeführt. Die Greif- und Bahnplanung wird durch eine objektzentrische lokale Multiresolutionskarte beschleunigt. Die Einzelkomponenten werden in einem prototypischen Gesamtsystem evaluiert. Eine weitere Anwendung für die lokale Multiresolutionsplanung ist die Pfadplanung für humanoide Fußballroboter. Zum Einsatz kommen Nao-Roboter, die nur über eine sehr eingeschränkte Rechenleistung verfügen. Durch die Reduktion der Planungskomplexität mit Hilfe der lokalen Multiresolution, wurde die Entwicklung eines Planers ermöglicht, der zusätzlich zur aktuellen Hindernisfreiheit die Bewegung der Gegenspieler auf dem Feld berücksichtigt. Hierbei liegt der Fokus auf einem wichtigen Teilproblem, dem Erreichen einer guten Schussposition hinter dem Ball. Die Tatsache, dass die Gegenspieler vergleichbare Ziele verfolgen, ermöglicht es, Annahmen über mögliche Laufwege zu treffen. Dadurch ist die Planung von Pfaden möglich, die das Risiko, durch einen Gegenspieler passiv geblockt zu werden, reduzieren, so dass die Schussposition schneller erreicht wird. Dieser Teil der Arbeit wird in einer physikalischen Fußballsimulation evaluiert. Im zweiten Teil dieser Dissertation werden Methoden zur Planung und Hindernisvermeidung von Multikoptern behandelt. Um die Planungskomplexität zu reduzieren, wird das zu lösenden Planungsproblem hierarchisch zerlegt und durch verschiedene Planungsebenen verarbeitet. Dabei haben höhere Planungsebenen eine abstraktere Weltsicht und werden mit niedriger Frequenz ausgeführt, zum Beispiel die Missionsplanung. Niedrigere Ebenen haben eine Weltsicht, die mehr den Sensordaten entspricht und werden mit höherer Frequenz ausgeführt. Die Granularität der resultierenden Pläne verfeinert sich hierbei auf niedrigeren Ebenen. Im Rahmen dieser Dissertation wurde eine komplette Planungshierarchie für Multikopter entwickelt, von Missionsplanern für verschiedene Anwendungsgebiete bis zu schneller Hindernisvermeidung. Pfade zur Ausführung geplanter Missionen werden durch zwei gekoppelte Planungsebenen erstellt, erst allozentrisch, und dann egozentrisch verfeinert. Hierbei werden ebenfalls globale und lokale Multiresolutionsrepräsentationen zur Beschleunigung der Planung eingesetzt. Zusätzlich zur Hindernisfreiheit und Länge der Pfade können auf diesen Planungsebenen weitere Zielfunktionen berücksichtigt werden, wie zum Beispiel die Berücksichtigung von Sensorcharakteristika. Ergänzt werden die Planungsebenen durch die Optimierung von Flugbahnen. Diese Flugbahnen berücksichtigen eine angenäherte Flugdynamik und erlauben damit ein schnelleres Verfolgen der optimierten Pfade. Um eine schnelle Konvergenz des Optimierungsproblems zu erreichen, wurde in dieser Arbeit ein Verfahren zur Initialisierung entwickelt. Des Weiteren kommen Methoden zur schnellen Verfeinerung des Optimierungsergebnisses bei Änderungen im Weltzustand zum Einsatz, diese ermöglichen die Reaktion auf neue Hindernisse oder Abweichungen von der Flugbahn, ohne eine komplette Flugbahn neu zu planen und zu optimieren. Die Sicherheit des durch die Planungs- und Optimierungsebenen erstellten Pfades wird durch eine schnelle, reaktive Hindernisvermeidung gewährleistet. Das Hindernisvermeidungsmodul basiert auf der Methode der künstlichen Potentialfelder. Durch die Verwendung dieser schnellen Methode kombiniert mit der Verwendung von nicht oder nur über kurze Zeiträume aggregierte Sensordaten, ermöglicht die Reaktion auf unbekannte Hindernisse, kurz nachdem diese von den Sensoren wahrgenommen wurden. Dabei kann der Multikopter abgebremst oder gestoppt werden, und sich von nähernden Hindernissen entfernen. Die Komponenten der Planungs- und Hindernisvermeidungshierarchie werden sowohl in der Simulation evaluiert, als auch in integrierten Gesamtsystemen mit verschiedenen Multikoptern in realen Anwendungen. Dies sind insbesondere die Kartierung von Innen- und Außenbereichen, die Inspektion von Gebäuden und Schornsteinen sowie die automatisierte Inventur von Lägern

Open Mapping towards Sustainable Development Goals

This collection amplifies the experiences of some of the world’s young people who are working to address SDGs using geospatial technologies and multi-national collaboration. Authors from every region of the world who have emerged as leaders in the YouthMappers movement share their perspectives and knowledge in an accessible and peer-friendly format. YouthMappers are university students who create and use open mapping for development and humanitarian purposes. Their work leverages digital innovations - both geospatial platforms and communications technologies - to answer the call for leadership to address sustainability challenges. The book conveys a sense of robust knowledge emerging from formal studies or informal academic experiences - in the first-person voices of students and recent graduates who are at the forefront of creating a new map of the world. YouthMappers use OpenStreetMap as the foundational sharing mechanism for creating data together. Authors impart the way they are learning about themselves, about each other, about the world. They are developing technology skills, and simultaneously teaching the rest of the world about the potential contributions of a highly connected generation of emerging world leaders for the SDGs. The book is timely, in that it captures a pivotal moment in the trajectory of the YouthMappers movement’s ability to share emerging expertise, and one that coincides with a pivotal moment in the geopolitical history of planet earth whose inhabitants need to hear from them. Most volumes that cover the topic of sustainability in terms of youth development are written by non-youth authors. Moreover, most are written by non-majoritarian, entrenched academic scholars. This book instead puts forward the diverse voices of students and recent graduates in countries where YouthMappers works, all over the world. Authors cover topics that range from water, agriculture, food, to waste, education, gender, climate action and disasters from their own eyes in working with data, mapping, and humanitarian action, often working across national boundaries and across continents. To inspire readers with their insights, the chapters are mapped to the United Nations 17 Sustainable Development Goals (SDGs) in ways that connect a youth agenda to a global agenda. With a preface written by Carrie Stokes, Chief Geographer and GeoCenter Director, United States Agency for International Development (USAID). This is an open access book

Open Mapping towards Sustainable Development Goals

This collection amplifies the experiences of some of the world’s young people who are working to address SDGs using geospatial technologies and multi-national collaboration. Authors from every region of the world who have emerged as leaders in the YouthMappers movement share their perspectives and knowledge in an accessible and peer-friendly format. YouthMappers are university students who create and use open mapping for development and humanitarian purposes. Their work leverages digital innovations - both geospatial platforms and communications technologies - to answer the call for leadership to address sustainability challenges. The book conveys a sense of robust knowledge emerging from formal studies or informal academic experiences - in the first-person voices of students and recent graduates who are at the forefront of creating a new map of the world. YouthMappers use OpenStreetMap as the foundational sharing mechanism for creating data together. Authors impart the way they are learning about themselves, about each other, about the world. They are developing technology skills, and simultaneously teaching the rest of the world about the potential contributions of a highly connected generation of emerging world leaders for the SDGs. The book is timely, in that it captures a pivotal moment in the trajectory of the YouthMappers movement’s ability to share emerging expertise, and one that coincides with a pivotal moment in the geopolitical history of planet earth whose inhabitants need to hear from them. Most volumes that cover the topic of sustainability in terms of youth development are written by non-youth authors. Moreover, most are written by non-majoritarian, entrenched academic scholars. This book instead puts forward the diverse voices of students and recent graduates in countries where YouthMappers works, all over the world. Authors cover topics that range from water, agriculture, food, to waste, education, gender, climate action and disasters from their own eyes in working with data, mapping, and humanitarian action, often working across national boundaries and across continents. To inspire readers with their insights, the chapters are mapped to the United Nations 17 Sustainable Development Goals (SDGs) in ways that connect a youth agenda to a global agenda. With a preface written by Carrie Stokes, Chief Geographer and GeoCenter Director, United States Agency for International Development (USAID). This is an open access book