From Monocular SLAM to Autonomous Drone Exploration

Micro aerial vehicles (MAVs) are strongly limited in their payload and power capacity. In order to implement autonomous navigation, algorithms are therefore desirable that use sensory equipment that is as small, low-weight, and low-power consuming as possible. In this paper, we propose a method for autonomous MAV navigation and exploration using a low-cost consumer-grade quadrocopter equipped with a monocular camera. Our vision-based navigation system builds on LSD-SLAM which estimates the MAV trajectory and a semi-dense reconstruction of the environment in real-time. Since LSD-SLAM only determines depth at high gradient pixels, texture-less areas are not directly observed so that previous exploration methods that assume dense map information cannot directly be applied. We propose an obstacle mapping and exploration approach that takes the properties of our semi-dense monocular SLAM system into account. In experiments, we demonstrate our vision-based autonomous navigation and exploration system with a Parrot Bebop MAV

Mapping and Real-Time Navigation With Application to Small UAS Urgent Landing

Small Unmanned Aircraft Systems (sUAS) operating in low-altitude airspace require flight near buildings and over people. Robust urgent landing capabilities including landing site selection are needed. However, conventional fixed-wing emergency landing sites such as open fields and empty roadways are rare in cities. This motivates our work to uniquely consider unoccupied flat rooftops as possible nearby landing sites. We propose novel methods to identify flat rooftop buildings, isolate their flat surfaces, and find touchdown points that maximize distance to obstacles. We model flat rooftop surfaces as polygons that capture their boundaries and possible obstructions on them. This thesis offers five specific contributions to support urgent rooftop landing. First, the Polylidar algorithm is developed which enables efficient non-convex polygon extraction with interior holes from 2D point sets. A key insight of this work is a novel boundary following method that contrasts computationally expensive geometric unions of triangles. Results from real-world and synthetic benchmarks show comparable accuracy and more than four times speedup compared to other state-of-the-art methods. Second, we extend polygon extraction from 2D to 3D data where polygons represent flat surfaces and interior holes representing obstacles. Our Polylidar3D algorithm transforms point clouds into a triangular mesh where dominant plane normals are identified and used to parallelize and regularize planar segmentation and polygon extraction. The result is a versatile and extremely fast algorithm for non-convex polygon extraction of 3D data. Third, we propose a framework for classifying roof shape (e.g., flat) within a city. We process satellite images, airborne LiDAR point clouds, and building outlines to generate both a satellite and depth image of each building. Convolutional neural networks are trained for each modality to extract high level features and sent to a random forest classifier for roof shape prediction. This research contributes the largest multi-city annotated dataset with over 4,500 rooftops used to train and test models. Our results show flat-like rooftops are identified with > 90% precision and recall. Fourth, we integrate Polylidar3D and our roof shape prediction model to extract flat rooftop surfaces from archived data sources. We uniquely identify optimal touchdown points for all landing sites. We model risk as an innovative combination of landing site and path risk metrics and conduct a multi-objective Pareto front analysis for sUAS urgent landing in cities. Our proposed emergency planning framework guarantees a risk-optimal landing site and flight plan is selected. Fifth, we verify a chosen rooftop landing site on real-time vertical approach with on-board LiDAR and camera sensors. Our method contributes an innovative fusion of semantic segmentation using neural networks with computational geometry that is robust to individual sensor and method failure. We construct a high-fidelity simulated city in the Unreal game engine with a statistically-accurate representation of rooftop obstacles. We show our method leads to greater than 4% improvement in accuracy for landing site identification compared to using LiDAR only. This work has broad impact for the safety of sUAS in cities as well as Urban Air Mobility (UAM). Our methods identify thousands of additional rooftop landing sites in cities which can provide safe landing zones in the event of emergencies. However, the maps we create are limited by the availability, accuracy, and resolution of archived data. Methods for quantifying data uncertainty or performing real-time map updates from a fleet of sUAS are left for future work.PHDRoboticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/170026/1/jdcasta_1.pd

Metalevel Motion Planning for Unmanned Aircraft Systems: Metrics Definition and Algorithm Selection

A diverse suite of manned and unmanned aircraft will occupy future urban airspace. Flight plans must accommodate specific aircraft characteristics, including physical volume with safety zone clearance, landing/takeoff procedures, kinodynamics, and a wide range of flight environments. No single motion planner is applicable across all possible aircraft configurations and operating conditions. This dissertation proposes the first motion planning algorithm selection capability with application to small Unmanned Aircraft System (UAS) multicopters operating in and over a complex urban landscape. Alternative data-driven fail-safe protocols are presented to improve on contemporary ``fly-home'' or automatic landing protocols, focusing on rooftops as safe urban landing sites. In a fail-safe direct strategy, the multicopter identifies, generates, and follows a flight plan to the closest available rooftop suitable for landing. In a fail-safe supervisory strategy, the multicopter examines rooftops en route to a planned landing site, diverting to a closer, clear landing site when possible. In a fail-safe coverage strategy, the multicopter cannot preplan a safe landing site due to missing data. The multicopter executes a coverage path to explore the area and evaluate overflown rooftops to find a safe landing site. These three fail-safe algorithms integrate map generation, flight planning, and area coverage capabilities. The motion planning algorithm selection problem (ASP) requires qualitative and quantitative metrics to inform the ASP of user/agent, algorithm, and configuration space preferences and constraints. Urban flight map-based, path-based, and software-based cost metrics are defined to provide insights into the urban canyon properties needed to construct safe and efficient flight plans. Map-based metrics describe the operating environment by constructing a collection of GPS/Lidar navigation performance, population density, and obstacle risk exposure metric maps. Path-based metrics account for a vehicle's energy consumption and distance traveled. Software-based metrics measure memory consumption and execution time of an algorithm. The proposed metrics provide pre-flight insights typically ignored by obstacle-only planning environment definitions. An algorithm portfolio consisting of geometric (Point-to-Point: PTP), graph-based (A* variants), and sampling-based (BIT* variants) motion planners were considered in this work. Path cost, execution time, and success rate benchmarks were investigated using Monte Carlo problem instances with A* "plus" producing the lowest cost paths, PTP having the fastest executions, and A* "dist" having the best overall success rates. The BIT* variant paths typically had higher cost but their success rate increased relative to altitude. The problem instances and metric maps informed two new machine learning solutions for urban small UAS motion planning ASP. Rule-based decision trees were simple to construct but unable to capture both complex cost metrics and algorithm properties. The investigated neural network-based ASP formulations produced promising results, with a hybrid two-stage selection scheme having the best algorithm selection accuracy, laying the seeds for future work. The most significant innovation of this dissertation is motion planning ASP for UAS. Non-traditional open-source databases also advance the field of data-driven flight planning, contributing to fail-safe UAS operations as well as ASP. Path planning algorithms integrated a new suite of diverse cost metrics accompanied by a novel multi-objective admissible heuristic function. Neural network and decision tree ASP options were presented and evaluated as a first-case practical approach to solving the motion planning ASP for small UAS urban flight.PHDRoboticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/168060/1/cosme_1.pd

Technology Assessment of eVTOL Personal Air Transportation System

This thesis intended to provide a holistic vision on the potential consequences of the introduction of emerging electrical Vertical Takeoff and Landing (e VTOL) Personal Air Transportation System (PATS) to contribute to the forming of public and policy opinion, and to assess the impacts and the feasibility of that. Instead of looking from a detailed vehicle design viewpoint, we tried to understand the need, the impacts, and the perceptions and the concerns of stakeholders. Thus, it was set a framework and methodology starting with a technology assessment point of view in the light of transportation system analysis. Limitations of the current ground and airline transportation systems, increasing congestion, poor block speed, combined with expanding population and demand for affordable on- demand mobility are driving the development of future transportation technology and policy. The third wave of aeronautics might be the answer and could bring about great new capabilities for society that would bring aviation into a new age of being relevant in daily lives since eVTOL PATS is envisioned as the next logical step in the natural progression in the history of disruptive transportation system innovations. However, there are a lot of questions. Although there was difficulty since the system was an emerging air transportation mode, an interdisciplinary study has been conducted to assess the impacts of developing such a capability. The research questions were determined to address the research objectives. What is the current state of mobility and eVTOL air transportation mode? What are the potential benefits of eVTOL air transportation mode for user and society? What are the perceptions of service providers, regulator, and user? What are the main challenges including technology, regulation, operation, social and environment aspects to enable the system? What are the enabling technologies? Nevertheless, with the results obtained lately from the research activities, revolutionary technologies and regulations are bringing us closer to eVTOL PATS reality every day. It can be argued that a new socio-technical transition will come about like the transition from horse drawn carriers to cars. Even if it is still a long way to go, it seems rather likely that the time has been arriving in the next decade. Their existence and operation would therefore need to be taken into consideration for today’s planning considerations and construction projects to be able to have this emerging air transportation mode available in the future. As the technology underlying eVTOL PATS evolves, wider eVTOL adoption across various markets is likely to be supported further if a set of key challenges such as safety and security, ease of use and autonomy, noise, infrastructure, and air traffic management are overcome. Achieving drastic improvements in ease of use, safety and community acceptable noise are the most critical steps towards the future feasibility of this market. Multi-use demos and demonstrating successful operation with early vehicles, namely eVTOL PATS prototype field operations, will create public acceptance and understanding of potentials in emerging air transportation mode for public good, use and learn in multiple applications. The overall perception of the user, service provider and regulator are positive, and the support is high. Shortly, a successful implementation and sustainable transition will depend on overcoming technological hurdles, regulatory frameworks, operational safety, cost competitiveness, and sensibilities of the affected communities. There is a need to enable people and goods to have the convenience of on-demand, point-to-point safe travel, further, anywhere in less travel time, through a network of pocket airports/vertiports, and there is a significant potential benefit so that policy makers, regulators and metropoles’ transportation planning departments should consider an inclusion of eVTOL air transportation mode into the scenarios and policies of the future.Esta tese pretende fornecer uma visão holística sobre as potenciais consequências da introdução do Sistema de Transporte Aéreo Pessoal (PATS) de Decolagem e Pouso Vertical elétrico emergente (e VTOL) para contribuir para a formação de opinião pública e política, e para avaliar os impactos e a viabilidade disso. Em vez de olhar de um ponto de vista detalhado o projeto do veículo, tentamos entender a necessidade, os impactos, as percepções e as preocupações das partes interessadas. Assim, foi definido um quadro e uma metodologia partindo de um ponto de vista de avaliação de tecnologia à luz da análise do sistema de transporte. As limitações dos atuais sistemas de transporte terrestre e aéreo, o aumento do congestionamento, a baixa velocidade do tráfego, combinados com a expansão da população e a mobilidade com procura acessível estão impulsionando o desenvolvimento de futuras tecnologias e políticas de transporte. A terceira onda da aeronáutica pode ser a resposta e pode trazer grandes novas capacidades para a sociedade que trariam a aviação para uma nova era de ser relevante na vida cotidiana, uma vez que o VTOL PATS é visto como o próximo passo lógico na progressão natural na história das inovações disruptivas do sistema de transporte. No entanto, há muitas perguntas. Embora tenha havido dificuldade por se tratar de um modo de transporte aéreo emergente, um estudo interdisciplinar foi realizado para avaliar os impactos do desenvolvimento de tal capacidade. As questões de investigação foram determinadas para atender aos objetivos do projeto. Qual é o estado atual da mobilidade e do modo de transporte aéreo eVTOL? Quais são os benefícios potenciais do modo de transporte aéreo eVTOL para o utilizador e a sociedade? Quais são as percepções dos provedores de serviços, regulador e utilizador? Quais são os principais desafios, incluindo tecnologia, regulamentação, operação, aspectos sociais e ambientais para habilitar o sistema? Quais são as tecnologias facilitadoras? No entanto, com os resultados obtidos ultimamente nas atividades de pesquisa, tecnologias e regulamentações revolucionárias estão nos aproximando cada dia mais da realidade do VTOL PATS. Pode-se argumentar que uma nova transição sócio-técnica ocorrerá como a transição de carruagens puxadas por cavalos para automóveis. Mesmo que ainda seja um longo caminho a percorrer, parece bastante provável que a hora esteja chegando na próxima década. A sua existência e operação, portanto, precisam ser levadas em consideração para as questões de planeamento e projetos de construção de hoje para poder ter esse modo de transporte aéreo emergente disponível no futuro. À medida que a tecnologia subjacente ao eVTOL PATS evolui, é provável que a adoção mais ampla do eVTOL em vários mercados seja ainda mais apoiada se um conjunto de desafios importantes, como segurança e proteção, facilidade de uso e autonomia, ruído, infraestrutura e gestão de tráfego aéreo forem superados. Alcançar melhorias drásticas na facilidade de uso, segurança e ruído aceitável pela comunidade são os passos mais críticos para a viabilidade futura deste mercado. Demonstrações multi-uso e demonstração de operação bem- sucedida com veículos iniciais, ou seja, operações de campo do protótipo eVTOL PATS, criarão aceitação pública e compreensão dos potenciais no modo de transporte aéreo emergente para o bem público, uso e aprendizado em várias aplicações. A percepção geral do utilizador, prestador de serviço e regulador é positiva, e o suporte é alto. Uma implementação bem-sucedida e uma transição sustentável dependerá da superação de obstáculos tecnológicos, estruturas regulatórias, segurança operacional, competitividade de custos e sensibilidade das comunidades afetadas. Há uma necessidade de permitir que pessoas e mercadorias tenham a conveniência de viagens seguras de que necessitam, ponto a ponto, e além disso, em qualquer lugar em menos tempo de viagem. Isso pode ser feito por meio de uma rede de aeroportos/vertiports, e há um benefício potencial significativo para que os formuladores de políticas, reguladores e departamentos de planeamento de transporte das grandes metrópoles considerem a inclusão do modo de transporte aéreo eVTOL nos cenários e políticas do futuro

Aeronautical engineering: A continuing bibliography with indexes (supplement 305)

This bibliography lists 239 reports, articles, and other documents recently introduced into the NASA scientific and technical information system. Subject coverage includes the following: the design, construction, and testing of aircraft and aircraft engines; aircraft components, equipment, and systems; ground support systems; and theoretical and applied aspects of aerodynamics and general fluid dynamics

Context-Enabled Visualization Strategies for Automation Enabled Human-in-the-loop Inspection Systems to Enhance the Situation Awareness of Windstorm Risk Engineers

Insurance loss prevention survey, specifically windstorm risk inspection survey is the process of investigating potential damages associated with a building or structure in the event of an extreme weather condition such as a hurricane or tornado. Traditionally, the risk inspection process is highly subjective and depends on the skills of the engineer performing it. This dissertation investigates the sensemaking process of risk engineers while performing risk inspection with special focus on various factors influencing it. This research then investigates how context-based visualizations strategies enhance the situation awareness and performance of windstorm risk engineers. An initial study investigated the sensemaking process and situation awareness requirements of the windstorm risk engineers. The data frame theory of sensemaking was used as the framework to carry out this study. Ten windstorm risk engineers were interviewed, and the data collected were analyzed following an inductive thematic approach. The themes emerged from the data explained the sensemaking process of risk engineers, the process of making sense of contradicting information, importance of their experience level, internal and external biases influencing the inspection process, difficulty developing mental models, and potential technology interventions. More recently human in the loop systems such as drones have been used to improve the efficiency of windstorm risk inspection. This study provides recommendations to guide the design of such systems to support the sensemaking process and situation awareness of windstorm visual risk inspection. The second study investigated the effect of context-based visualization strategies to enhance the situation awareness of the windstorm risk engineers. More specifically, the study investigated how different types of information contribute towards the three levels of situation awareness. Following a between subjects study design 65 civil/construction engineering students completed this study. A checklist based and predictive display based decision aids were tested and found to be effective in supporting the situation awareness requirements as well as performance of windstorm risk engineers. However, the predictive display only helped with certain tasks like understanding the interaction among different components on the rooftop. For remaining tasks, checklist alone was sufficient. Moreover, the decision aids did not place any additional cognitive demand on the participants. This study helped us understand the advantages and disadvantages of the decision aids tested. The final study evaluated the transfer of training effect of the checklist and predictive display based decision aids. After one week of the previous study, participants completed a follow-up study without any decision aids. The performance and situation awareness of participants in the checklist and predictive display group did not change significantly from first trial to second trial. However, the performance and situation awareness of participants in the control condition improved significantly in the second trial. They attributed this to their exposure to SAGAT questionnaire in the first study. They knew what issues to look for and what tasks need to be completed in the simulation. The confounding effect of SAGAT questionnaires needs to be studied in future research efforts

Marshall Space Flight Center Research and Technology Report 2019

Today, our calling to explore is greater than ever before, and here at Marshall Space Flight Centerwe make human deep space exploration possible. A key goal for Artemis is demonstrating and perfecting capabilities on the Moon for technologies needed for humans to get to Mars. This years report features 10 of the Agencys 16 Technology Areas, and I am proud of Marshalls role in creating solutions for so many of these daunting technical challenges. Many of these projects will lead to sustainable in-space architecture for human space exploration that will allow us to travel to the Moon, on to Mars, and beyond. Others are developing new scientific instruments capable of providing an unprecedented glimpse into our universe. NASA has led the charge in space exploration for more than six decades, and through the Artemis program we will help build on our work in low Earth orbit and pave the way to the Moon and Mars. At Marshall, we leverage the skills and interest of the international community to conduct scientific research, develop and demonstrate technology, and train international crews to operate further from Earth for longer periods of time than ever before first at the lunar surface, then on to our next giant leap, human exploration of Mars. While each project in this report seeks to advance new technology and challenge conventions, it is important to recognize the diversity of activities and people supporting our mission. This report not only showcases the Centers capabilities and our partnerships, it also highlights the progress our people have achieved in the past year. These scientists, researchers and innovators are why Marshall and NASA will continue to be a leader in innovation, exploration, and discovery for years to come

Combining Sensors and Multibody Models for Applications in Vehicles, Machines, Robots and Humans

The combination of physical sensors and computational models to provide additional information about system states, inputs and/or parameters, in what is known as virtual sensing, is becoming increasingly popular in many sectors, such as the automotive, aeronautics, aerospatial, railway, machinery, robotics and human biomechanics sectors. While, in many cases, control-oriented models, which are generally simple, are the best choice, multibody models, which can be much more detailed, may be better suited to some applications, such as during the design stage of a new product