Semi-Structured Decision Processes: A Conceptual Framework for Understanding Human-Automation Decision Systems

The purpose of this work is to improve understanding of existing and proposed decision systems, ideally to improve the design of future systems. A "decision system" is defined as a collection of information-processing components -- often involving humans and automation (e.g., computers) -- that interact towards a common set of objectives. Since a key issue in the design of decision systems is the division of work between humans and machines (a task known as "function allocation"), this report is primarily intended to help designers incorporate automation more appropriately within these systems. This report does not provide a design methodology, but introduces a way to qualitatively analyze potential designs early in the system design process. A novel analytical framework is presented, based on the concept of "semi-Structured" decision processes. It is believed that many decisions involve both well-defined "Structured" parts (e.g., formal procedures, traditional algorithms) and ill-defined "Unstructured" parts (e.g., intuition, judgement, neural networks) that interact in a known manner. While Structured processes are often desired because they fully prescribe how a future decision (during "operation") will be made, they are limited by what is explicitly understood prior to operation. A system designer who incorporates Unstructured processes into a decision system understands which parts are not understood sufficiently, and relinquishes control by deferring decision-making from design to operation. Among other things, this design choice tends to add flexibility and robustness. The value of the semi-Structured framework is that it forces people to consider system design concepts as operational decision processes in which both well-defined and ill-defined components are made explicit. This may provide more insight into decision systems, and improve understanding of the implications of design choices. The first part of this report defines the semi-Structured process and introduces a diagrammatic notation for decision process models. In the second part, the semi-Structured framework is used to understand and explain highly evolved decision system designs (these are assumed to be representative of "good" designs) whose components include feedback controllers, alerts, decision aids, and displays. Lastly, the semi-Structured framework is applied to a decision system design for a mobile robot.Charles Stark Draper Laboratory, Inc., under IR&D effort 101

Large scale dynamic systems

Classes of large scale dynamic systems were discussed in the context of modern control theory. Specific examples discussed were in the technical fields of aeronautics, water resources and electric power

Theory of Effectiveness Measurement

Effectiveness measures provide decision makers feedback on the impact of deliberate actions and affect critical issues such as allocation of scarce resources, as well as whether to maintain or change existing strategy. Currently, however, there is no formal foundation for formulating effectiveness measures. This research presents a new framework for effectiveness measurement from both a theoretical and practical view. First, accepted effects-based principles, as well as fundamental measurement concepts are combined into a general, domain independent, effectiveness measurement methodology. This is accomplished by defining effectiveness measurement as the difference, or conceptual distance from a given system state to some reference system state (e.g. desired end-state). Then, by developing system attribute measures such that they yield a system state-space that can be characterized as a metric space, differences in system states relative to the reference state can be gauged over time, yielding a generalized, axiomatic definition of effectiveness measurement. The effectiveness measurement framework is then extended to mitigate the influence of measurement error and uncertainty by employing Kalman filtering techniques. Finally, the pragmatic nature of the approach is illustrated by measuring the effectiveness of a notional, security force response strategy in a scenario involving a terrorist attack on a United States Air Force base

A set-based approach to passenger aircraft family design

In today's highly competitive civil aviation market, aircraft manufacturers develop aircraft families in order to satisfy a wide range of requirements from multiple airlines, with reduced costs of ownership and shorter lead time. Traditional methods for designing passenger aircraft families employ a sequential, optimisation-based approach, where a single configuration and systems architecture is selected fairly early which is then iteratively analysed and modified until all the requirements are met. The problem with such an approach is the tendency of the optimisers to exploit assumptions already 'hard-wired' in the computational models. Subsequently the design is driven towards a solution which, while promising to the optimiser, may be infeasible due to the factors not considered by the models, e.g. integration and installation of promising novel technological solutions, which result in costly design rework later in the design process. Within this context, the aim is to develop a methodology for designing passenger aircraft families, which provides an environment for designers to interactively explore wider design space and foster innovation. To achieve this aim, a novel methodology for passenger aircraft family design is proposed where multiple aircraft family solutions are synthesised from the outset by integrating major components sets and systems architectures set. This is facilitated by integrating set theory principles and model-based design exploration methods. As more design knowledge is gained through analysis, the set of aircraft family solutions is gradually narrowed-down by discarding infeasible and inferior solutions. This is achieved through constraint analysis using iso-contours. The evaluation has been carried out through an application case-study (of a three-member passenger aircraft family design) which was executed with both the proposed methodology and the traditional approach for comparison. The proposed methodology and the case-study (along with the comparison results) were presented to a panel of industrial experts who were asked to comment on the merits and potential challenges of the proposed methodology. The conclusion is that the proposed methodology is expected to reduce the number of costly design changes, enabling designers to consider novel systems technologies and gain knowledge through interactive design space exploration. It was pointed out, however, that while the computational enablers behind the proposed approach are reaching a stage of maturity, allowing a multitude of concepts to be analysed rapidly and simultaneously, this still is expected to present a challenge from organisational process and resource point of view. It was agreed that by considering a set of aircraft family solutions, the proposed approach would enable the designers to delay critical decisions until more knowledge is available, which helps to mitigate risks associated with innovative systems architectures and technologies

Complexity challenges in ATM

After more than 4 years of activity, the ComplexWorld Network, together with the projects and PhDs covered under the SESAR long-term research umbrella, have developed sound research material contributing to progress beyond the state of the art in fields such as resilience, uncertainty, multi-agent systems, metrics and data science. The achievements made by the ComplexWorld stakeholders have also led to the identification of new challenges that need to be addressed in the future. In order to pave the way for complexity science research in Air Traffic Management (ATM) in the coming years, ComplexWorld requested external assessments on how the challenges have been covered and where there are existing gaps. For that purpose, ComplexWorld, with the support of EUROCONTROL, established an expert panel to review selected documentation developed by the network and provide their assessment on their topic of expertise

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Metamodeling Techniques to Aid in the Aggregation Process of Large Hierarchical Simulation Models

This research investigates how aggregation is currently conducted for simulation of large systems. The purpose is to examine how to achieve suitable aggregation in the simulation of large systems. More specifically, investigating how to accurately aggregate hierarchical lower-level (higher resolution) models into the next higher-level in order to reduce the complexity of the overall simulation model. The focus is on the exploration of the different aggregation techniques for hierarchical lower-level (higher resolution) models into the next higher-level. We develop aggregation procedures between two simulation levels (e.g., aggregation of engagement level models into a mission level model) to address how much and what information needs to pass from the high resolution to the low-resolution model in order to preserve statistical fidelity. We present a mathematical representation of the simulation model based on network theory and procedures for simulation aggregation that are logical and executable. This research examines the effectiveness of several statistical techniques, to include regression and three types of artificial neural networks, as an aggregation technique in predicting outputs of the lower-level model and evaluating its effects as an input into the next higher-level model. The proposed process is a collection of various conventional statistical and aggregation techniques, to include one novel concept and extensions to the regression and neural network methods, which are compared to the truth simulation model, where the truth model is when actual lower-level model outputs are used as a direct input into the next higher-level model. The aggregation methodology developed in this research provides an analytic foundation that formally defines the necessary steps essential in appropriately and effectively simulating large hierarchical systems

3D-in-2D Displays for ATC.

This paper reports on the efforts and accomplishments of the 3D-in-2D Displays for ATC project at the end of Year 1. We describe the invention of 10 novel 3D/2D visualisations that were mostly implemented in the Augmented Reality ARToolkit. These prototype implementations of visualisation and interaction elements can be viewed on the accompanying video. We have identified six candidate design concepts which we will further research and develop. These designs correspond with the early feasibility studies stage of maturity as defined by the NASA Technology Readiness Level framework. We developed the Combination Display Framework from a review of the literature, and used it for analysing display designs in terms of display technique used and how they are combined. The insights we gained from this framework then guided our inventions and the human-centered innovation process we use to iteratively invent. Our designs are based on an understanding of user work practices. We also developed a simple ATC simulator that we used for rapid experimentation and evaluation of design ideas. We expect that if this project continues, the effort in Year 2 and 3 will be focus on maturing the concepts and employment in a operational laboratory settings

Learning and Evolving Flight Controller for Fixed-Wing Unmanned Aerial Systems

Artificial intelligence has been called the fourth wave of industrialization following steam power, electricity, and computation. The field of aerospace engineering has been significantly impacted by this revolution, presenting the potential to build neural network-based high-performance autonomous flight systems. This work presents a novel application of machine learning technology to develop evolving neural network controllers for fixed-wing unmanned aerial systems. The hypothesis for an artificial neural network being capable of replacing a physics-based autopilot system consisting of guidance, navigation, and control, or a combination of these, is evaluated and proven through empirical experiments. Building upon widely use supervised learning methods and its variants, labeled data is generated leveraging non-zero set point linear quadratic regulator based autopilot systems to train neural network models, thereby developing a novel imitation learning algorithm. The ultimate goal of this research is to build a robust learning flight controller using low-cost and engineering level aircraft dynamic model and have the ability to evolve in time. Discovering the limitations of supervised learning methods, reinforcement learning techniques are employed to learn directly from data, breaking feedback correlations and dynamic model dependence for a control system. This manifests into a policy-based neural network controller that is robust towards un-modeled dynamics and uncertainty in aircraft dynamic model. To fundamentally change flight controller tuning practices, a unique evolution methodology is developed that directly uses flight data from a real aircraft: factual dynamic states and the rewards associated with them, in order to re-train a neural network controller. This work has the following unique contributions: 1. Novel imitation learning algorithms that mimic "expert" policy decisions using data aggregation are developed, which allow for unification of guidance and control algorithms into a single loop using artificial neural networks. 2. A time-based and dynamic model dependent moving window data aggregation algorithm is uniquely developed to accurately capture aircraft transient behavior and to mitigate neural network over-fitting, which caused low amplitude and low frequency oscillations in control predictions. 3. Due to substantial dependence of imitation learning algorithms on "expert" policies and physics-based flight controllers, reinforcement learning is used, which can train neural network controllers directly from data. Although, the developed neural network controller was trained using engineering level dynamic model of the aircraft with low-fidelity in low Reynold's numbers, it demonstrates unique capabilities to generalize a control policy in a series of flight tests and exhibits robustness to achieve the desired performance in presence of external disturbances (cross wind, gust, etc.). 4. In addition to extensive hardware in the loop simulations, this work was uniquely validated by actual flight tests on a foam-based, pusher, twin-boom Skyhunter aircraft. 5. Reliability and consistency of the longitudinal neural network controller is validated in 15 distinct flight tests, spread over a period of 5 months (November 2019 to March 2020), consisting of 21 different flight scenarios. Automatic flight missions are deployed to conduct a fair comparison of linear quadratic regulator and neural network controllers. 6. An evolution technique is developed to re-train artificial neural network flight controllers directly from flight data and mitigate dependence on aircraft dynamic models, using a modified Deep Deterministic Policy Gradients algorithm and is implemented via TensorFlow software to attain the goals of evolution