Real-Time, Multiple Pan/Tilt/Zoom Computer Vision Tracking and 3D Positioning System for Unmanned Aerial System Metrology

The study of structural characteristics of Unmanned Aerial Systems (UASs) continues to be an important field of research for developing state of the art nano/micro systems. Development of a metrology system using computer vision (CV) tracking and 3D point extraction would provide an avenue for making these theoretical developments. This work provides a portable, scalable system capable of real-time tracking, zooming, and 3D position estimation of a UAS using multiple cameras. Current state-of-the-art photogrammetry systems use retro-reflective markers or single point lasers to obtain object poses and/or positions over time. Using a CV pan/tilt/zoom (PTZ) system has the potential to circumvent their limitations. The system developed in this paper exploits parallel-processing and the GPU for CV-tracking, using optical flow and known camera motion, in order to capture a moving object using two PTU cameras. The parallel-processing technique developed in this work is versatile, allowing the ability to test other CV methods with a PTZ system using known camera motion. Utilizing known camera poses, the object\u27s 3D position is estimated and focal lengths are estimated for filling the image to a desired amount. This system is tested against truth data obtained using an industrial system

Flying Qualities Built-In-Test For Unmanned Aerial Systems

This paper presents a flying qualities built-in-test for UAS application with the scope limited to the longitudinal axis. A doublet input waveform excites the AV and both α and q are used by EUDKF to estimate the A and B matrices which are short period approximations of the system. ζ, ω, GM, PM, observability, and controllability are calculated to determine flying qualities with the results displayed to the AVO in a color-coded, easy to interpret display. While SID algorithms have been flying in vehicles with adaptive control schemes, vehicles with other schemes (such as classical feedback) lack this built-in self assessment tool. In addition, adaptive control SID results are not analyzed and displayed but instead used internally. This work intends to extend this self-assessment option to all UASs regardless of control scheme as a “plug-and-play” add-on by building a reliable and robust tool that requires little tuning

Innovative Piloting Technique for a Semi-Autonomous UAV Lighter-Than-Air Platform Simulator

UAS design has in these years reached a point in which trends and objectives are well beyond the actual test capabilities. The tendency of the past to build and test has clearly been overridden by new design concepts for many reasons, one of these being the scarce or null possibility of testing safety-critical systems such as UAV systems. This is the context in which the Elettra-Twin-Flyer (ETF) Simulator is constantly upgraded and rearranged to incorporate new features and more advanced capabilities. In this paper it is shown how the piloting modes have been differentiated, to improve the airship autonomy and allow path following operations. Innovative piloting tools have been introduced and a new Human-Machine-Interface has been proposed along

Large-Scale Unmanned Aerial Systems Traffic Density Prediction and Management

In recent years, the applications of Unmanned Aerial Systems (UAS) has become more and more popular. We envision that in the near future, the complicated and high density UAS traffic will impose significant burden to air traffic management. Lot of works focus on the application development of individual Small Unmanned Aerial Systems (sUAS) or sUAS management Policy, however, the study of the UAS cluster behaviors such as forecasting and managing of the UAS traffic has generally not been addressed. In order to address the above issue, there is an urgent need to investigate three research directions. The first direction is to develop a high fidelity simulator for the UAS cluster behavior evaluation. The second direction to study real time trajectory planning algorithms to mitigate the high dense UAS traffic. The last direction is to investigate techniques that rapidly and accurately forecast the UAS traffic pattern in the future. In this thesis, we elaborate these three research topics and present a universal paradigm to predict and manage the traffic for the large-scale unmanned aerial systems. To enable the research in UAS traffic management and prediction, a Java based Multi-Agent Air Traffic and Resource Usage Simulation (MATRUS) framework is first developed. We use two types of UAS trajectories, Point-to-Point (P2P) and Man- hattan, as the case study to describe the capability of presented framework. Various communication and propagation models (i.e. log-distance-path loss) can be integrated with the framework to model the communication between UASs and base stations. The results show that MATRUS has the ability to evaluate different sUAS traffic management policies, and can provide insights on the relationships between air traf- fic and communication resource usage for further studies. Moreover, the framework can be extended to study the effect of sUAS Detect-and-Avoid (DAA) mechanisms, implement additional traffic management policies, and handle more complex traffic demands and geographical distributions. Based on the MATRUS framework, we propose a Sparse Represented Temporal- Spatial (SRTS) UAS trajectory planning algorithm. The SRTS algorithm allows the sUAS to avoid static no-fly areas (i.e. static obstacles) or other areas that have congested air traffic or communication traffic. The core functionality of the routing algorithm supports the instant refresh of the in-flight environment making it appropri- ate for highly dynamic air traffic scenarios. The characterization of the routing time and memory usage demonstrate that the SRTS algorithm outperforms a traditional Temporal-Spatial routing algorithm. The deep learning based approach has shown an outstanding success in many areas, we first investigated the possibility of applying the deep neural network in predicting the trajectory of a single vehicle in a given traffic scene. A new trajectory prediction model is developed, which allows information sharing among vehicles using a graph neural network. The prediction is based on the embedding feature, which is derived from multi-dimensional input sequences including the historical trajectory of target and neighboring vehicles, and their relative positions. Compared to other existing trajectory prediction methods, the proposed approach can reduce the pre- diction error by up to 50.00%. Then, we present a deep neural network model that extracts the features from both spatial and temporal domains to predict the UAS traffic density. In addition, a novel input representation of the future sUAS mission information is proposed. The pre-scheduled missions are categorized into 3 types according to their launching times. The results show that our presented model out- performs all of the baseline models. Meanwhile, the qualitative results demonstrate that our model can accurately predict the hot spot in the future traffic map

A Hierarchical Architectural Framework for Securing Unmanned Aerial Systems

Unmanned Aerial Systems (UAS) are becoming more widely used in the new era of evolving technology; increasing performance while decreasing size, weight, and cost. A UAS equipped with a Flight Control System (FCS) that can be used to fly semi- or fully-autonomous is a prime example of a Cyber Physical and Safety Critical system. Current Cyber-Physical defenses against malicious attacks are structured around security standards for best practices involving the development of protocols and the digital software implementation. Thus far, few attempts have been made to embed security into the architecture of the system considering security as a holistic problem. Therefore, a Hierarchical, Embedded, Cyber Attack Detection (HECAD) framework is developed to provide security in a holistic manor, providing resiliency against cyber-attacks as well as introducing strategies for mitigating and dealing with component failures. Traversing the hardware/software barrier, HECAD provides detection of malicious faults at the hardware and software level; verified through the development of an FPGA implementation and tested using a UAS FCS

Influencing Factors for Use of Unmanned Aerial Systems in Support of Aviation Accident and Emergency Response

The purpose of this research paper was to examine the influencing factors associated with the use of unmanned aerial system (UAS) technology to support aviation accident and emergency response. The ability of first responders to react to an emergency is dependent on the quality, accuracy, timeliness, and usability of information. With aviation accidents such as the Asiana Airlines Flight 214 crash at San Francisco International Airport, the ability to sense and communicate the location of victims may reduce the potential for accidental passenger death. Furthermore, the ability to obtain information en-route to an accident may also to assist to reduce overall response and coordination time of first responders (e.g., Aviation Rescue and Firefighting [ARFF]). By identifying and examining current and potential practices, capabilities, and technology (e.g., human-machine-interface [HMI], human factors, tools, and capability modifiers) a more comprehensive model of the influencing factors is established to further support the growing body of knowledge (i.e., safety, human computer interaction, human-robot systems, socio-economical systems, service and public sector systems, and technological forecasting). A series of recommendations regarding the technology and application are provided to support future development or adaptation of regulations, policies, or future research. --from the article

Unmanned Vehicle Systems & Operations on Air, Sea, Land

Unmanned Vehicle Systems & Operations On Air, Sea, Land is our fourth textbook in a series covering the world of Unmanned Aircraft Systems (UAS) and Counter Unmanned Aircraft Systems (CUAS). (Nichols R. K., 2018) (Nichols R. K., et al., 2019) (Nichols R. , et al., 2020)The authors have expanded their purview beyond UAS / CUAS systems. Our title shows our concern for growth and unique cyber security unmanned vehicle technology and operations for unmanned vehicles in all theaters: Air, Sea and Land – especially maritime cybersecurity and China proliferation issues. Topics include: Information Advances, Remote ID, and Extreme Persistence ISR; Unmanned Aerial Vehicles & How They Can Augment Mesonet Weather Tower Data Collection; Tour de Drones for the Discerning Palate; Underwater Autonomous Navigation & other UUV Advances; Autonomous Maritime Asymmetric Systems; UUV Integrated Autonomous Missions & Drone Management; Principles of Naval Architecture Applied to UUV’s; Unmanned Logistics Operating Safely and Efficiently Across Multiple Domains; Chinese Advances in Stealth UAV Penetration Path Planning in Combat Environment; UAS, the Fourth Amendment and Privacy; UV & Disinformation / Misinformation Channels; Chinese UAS Proliferation along New Silk Road Sea / Land Routes; Automaton, AI, Law, Ethics, Crossing the Machine – Human Barrier and Maritime Cybersecurity.Unmanned Vehicle Systems are an integral part of the US national critical infrastructure The authors have endeavored to bring a breadth and quality of information to the reader that is unparalleled in the unclassified sphere. Unmanned Vehicle (UV) Systems & Operations On Air, Sea, Land discusses state-of-the-art technology / issues facing U.S. UV system researchers / designers / manufacturers / testers. We trust our newest look at Unmanned Vehicles in Air, Sea, and Land will enrich our students and readers understanding of the purview of this wonderful technology we call UV.https://newprairiepress.org/ebooks/1035/thumbnail.jp

An Empirical Methodology for Engineering Human Systems Integration

The systems engineering technical processes are not sufficiently supported by methods and tools that quantitatively integrate human considerations into early system design. Because of this, engineers must often rely on qualitative judgments or delay critical decisions until late in the system lifecycle. Studies reveal that this is likely to result in cost, schedule, and performance consequences. This dissertation presents a methodology to improve the application of systems engineering technical processes for design. This methodology is mathematically rigorous, is grounded in relevant theory, and applies extant human subjects data to critical systems development challenges. The methodology is expressed in four methods that support early systems engineering activities: a requirements elicitation method, a function allocation method, an input device design method, and a display layout design method. These form a coherent approach to early system development. Each method is separately discussed and demonstrated using a prototypical system development program. In total, this original and significant work has a broad range of systems engineer applicability to improve the engineering of human systems integration

An Unmanned Lighter-Than-Air Platform for Large Scale Land Monitoring

The concept and preliminary design of an unmanned lighter-than-air (LTA) platform instrumented with different remote sensing technologies is presented. The aim is to assess the feasibility of using a remotely controlled airship for the land monitoring of medium sized (up to 107 m2) urban or rural areas at relatively low altitudes (below 1000 m) and its potential convenience with respect to other standard remote and in-situ sensing systems. The proposal includes equipment for high-definition visual, thermal, and hyperspectral imaging as well as LiDAR scanning. The data collected from these different sources can be then combined to obtain geo-referenced products such as land use land cover (LULC), soil water content (SWC), land surface temperature (LSC), and leaf area index (LAI) maps, among others. The potential uses for diffuse structural health monitoring over built-up areas are discussed as well. Several mission typologies are considere