A framework based on Gaussian mixture models and Kalman filters for the segmentation and tracking of anomalous events in shipboard video

Anomalous indications in monitoring equipment on board U.S. Navy vessels must be handled in a timely manner to prevent catastrophic system failure. The development of sensor data analysis techniques to assist a ship\u27s crew in monitoring machinery and summon required ship-to-shore assistance is of considerable benefit to the Navy. In addition, the Navy has a large interest in the development of distance support technology in its ongoing efforts to reduce manning on ships. In this thesis, algorithms have been developed for the detection of anomalous events that can be identified from the analysis of monochromatic stationary ship surveillance video streams. The specific anomalies that we have focused on are the presence and growth of smoke and fire events inside the frames of the video stream. The algorithm consists of the following steps. First, a foreground segmentation algorithm based on adaptive Gaussian mixture models is employed to detect the presence of motion in a scene. The algorithm is adapted to emphasize gray-level characteristics related to smoke and fire events in the frame. Next, shape discriminant features in the foreground are enhanced using morphological operations. Following this step, the anomalous indication is tracked between frames using Kalman filtering. Finally, gray level shape and motion features corresponding to the anomaly are subjected to principal component analysis and classified using a multilayer perceptron neural network. The algorithm is exercised on 68 video streams that include the presence of anomalous events (such as fire and smoke) and benign/nuisance events (such as humans walking the field of view). Initial results show that the algorithm is successful in detecting anomalies in video streams, and is suitable for application in shipboard environments

Data-driven Ship Performance Models - - Emphasis on Energy Efficiency and Fatigue Safety

Due to digitalization in the maritime industry, a huge amount of ship operation-related data has been collected. The main objective of this thesis is to exploit machine learning/big data analytics to build data-driven ship performance models, focusing on speed-power relationship modeling, and fatigue accumulation assessment during a ship’s operation at sea.The speed-power performance models are established in three different ways: 1) semi-empirical white-box models, 2) machine learning black-box methods, and 3) physics-informed grey-box models. The white-box models include improved semi-empirical formulas for ship added resistance due to head waves, and further developed formulas in arbitrary wave headings. Validation studies using three case study ships show good agreement between the speed predictions by the white-box models and the long-term averages of full-scale measurements. Different supervised machine learning methods’ capabilities have been compared for black-box modeling. The XGBoost algorithm is found to have the most reliable predictive ability, with the highest efficiency suitable for onboard devices. The novel grey-box models are proposed by considering the physical principles in model tests and big data information from real sailing. It has been demonstrated that the proposed grey-box models can improve prediction accuracy by approximately 30% for ship speed estimation and provides 50% less cumulative error of sailing time than the black-box methods.The impact of voyage optimization-aided operations on the encountered wave conditions and ship fatigue damage is investigated in this thesis. By recommending appropriate routes, voyage optimization can greatly extend the fatigue life of a ship by at least 50%. The machine learning techniques are also applied to a ship’s fatigue assessment. The results indicate that the proposed data-driven fatigue assessment model could increase accuracy by approximately 70% for the case study vessel compared to other prominent spectral methods

Surrogate-Assisted Unified Optimization Framework for Investigating Marine Structural Design Under Information Uncertainty.

Structural decisions made in the early stages of marine systems design can have a large impact on future acquisition, maintenance and life-cycle costs. However, owing to the unique nature of early stage marine system design, these critical structure decisions are often made on the basis of incomplete information or knowledge about the design. When coupled with design optimization analysis, the complex, uncertain early stage design environment makes it very difficult to deliver a quantified trade-off analysis for decision making. This work presents a novel decision support method that integrates design optimization, high-fidelity analysis, and modeling of information uncertainty for early stage design and analysis. To support this method this dissertation improves the design optimization methods for marine structures by proposing several novel surrogate modeling techniques and strategies. The proposed work treats the uncertainties that are sourced from limited information in a non-statistical interval uncertainty form. This interval uncertainty is treated as an objective function in an optimization framework in order to explore the impact of information uncertainty on structural design performance. In this examination, the potential structural weight penalty regarding information uncertainty can be quickly identified in early stage, avoiding costly redesign later in the design. This dissertation then continues to explore a balanced computational structure between fidelity and efficiency. A proposed novel variable fidelity approach can be applied to wisely allocate expensive high-fidelity computational simulations. In achieving the proposed capabilities for design optimization, several surrogate modeling methods are developed concerning worst-case estimation, clustered multiple meta-modeling, and mixed variable modeling techniques. These surrogate methods have been demonstrated to significantly improve the efficiency of optimizer in dealing with the challenges of early stage marine structure design.PhDNaval Architecture and Marine EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/133365/1/yanliuch_1.pd

Human-AI complex task planning

The process of complex task planning is ubiquitous and arises in a variety of compelling applications. A few leading examples include designing a personalized course plan or trip plan, designing music playlists/work sessions in web applications, or even planning routes of naval assets to collaboratively discover an unknown destination. For all of these aforementioned applications, creating a plan requires satisfying a basic construct, i.e., composing a sequence of sub-tasks (or items) that optimizes several criteria and satisfies constraints. For instance, in course planning, sub-tasks or items are core and elective courses, and degree requirements capture their complex dependencies as constraints. In trip planning, sub-tasks are points of interest (POIs) and constraints represent time and monetary budget, or user-specified requirements. Needless to say, task plans are to be individualized and designed considering uncertainty. When done manually, the process is human-intensive and tedious, and unlikely to scale. The goal of this dissertation is to present computational frameworks that synthesize the capabilities of human and AI algorithms to enable task planning at scale while satisfying multiple objectives and complex constraints. This dissertation makes significant contributions in four main areas, (i) proposing novel models, (ii) designing principled scalable algorithms, (iii) conducting rigorous experimental analysis, and (iv) deploying designed solutions in the real-world. A suite of constrained and multi-objective optimization problems has been formalized, with a focus on their applicability across diverse domains. From an algorithmic perspective, the dissertation proposes principled algorithms with theoretical guarantees adapted from discrete optimization techniques, as well as Reinforcement Learning based solutions. The memory and computational efficiency of these algorithms have been studied, and optimization opportunities have been proposed. The designed solutions are extensively evaluated on various large-scale real-world and synthetic datasets and compared against multiple baseline solutions after appropriate adaptation. This dissertation also presents user study results involving human subjects to validate the effectiveness of the proposed models. Lastly, a notable outcome of this dissertation is the deployment of one of the developed solutions at the Naval Postgraduate School. This deployment enables simultaneous route planning for multiple assets that are robust to uncertainty under multiple contexts

Techniques for the allocation of resources under uncertainty

L’allocation de ressources est un problème omniprésent qui survient dès que des ressources limitées doivent être distribuées parmi de multiples agents autonomes (e.g., personnes, compagnies, robots, etc). Les approches standard pour déterminer l’allocation optimale souffrent généralement d’une très grande complexité de calcul. Le but de cette thèse est de proposer des algorithmes rapides et efficaces pour allouer des ressources consommables et non consommables à des agents autonomes dont les préférences sur ces ressources sont induites par un processus stochastique. Afin d’y parvenir, nous avons développé de nouveaux modèles pour des problèmes de planifications, basés sur le cadre des Processus Décisionnels de Markov (MDPs), où l’espace d’actions possibles est explicitement paramétrisés par les ressources disponibles. Muni de ce cadre, nous avons développé des algorithmes basés sur la programmation dynamique et la recherche heuristique en temps-réel afin de générer des allocations de ressources pour des agents qui agissent dans un environnement stochastique. En particulier, nous avons utilisé la propriété acyclique des créations de tâches pour décomposer le problème d’allocation de ressources. Nous avons aussi proposé une stratégie de décomposition approximative, où les agents considèrent des interactions positives et négatives ainsi que les actions simultanées entre les agents gérants les ressources. Cependant, la majeure contribution de cette thèse est l’adoption de la recherche heuristique en temps-réel pour l’allocation de ressources. À cet effet, nous avons développé une approche basée sur la Q-décomposition munie de bornes strictes afin de diminuer drastiquement le temps de planification pour formuler une politique optimale. Ces bornes strictes nous ont permis d’élaguer l’espace d’actions pour les agents. Nous montrons analytiquement et empiriquement que les approches proposées mènent à des diminutions de la complexité de calcul par rapport à des approches de planification standard. Finalement, nous avons testé la recherche heuristique en temps-réel dans le simulateur SADM, un simulateur d’allocation de ressource pour une frégate.Resource allocation is an ubiquitous problem that arises whenever limited resources have to be distributed among multiple autonomous entities (e.g., people, companies, robots, etc). The standard approaches to determine the optimal resource allocation are computationally prohibitive. The goal of this thesis is to propose computationally efficient algorithms for allocating consumable and non-consumable resources among autonomous agents whose preferences for these resources are induced by a stochastic process. Towards this end, we have developed new models of planning problems, based on the framework of Markov Decision Processes (MDPs), where the action sets are explicitly parameterized by the available resources. Given these models, we have designed algorithms based on dynamic programming and real-time heuristic search to formulating thus allocations of resources for agents evolving in stochastic environments. In particular, we have used the acyclic property of task creation to decompose the problem of resource allocation. We have also proposed an approximative decomposition strategy, where the agents consider positive and negative interactions as well as simultaneous actions among the agents managing the resources. However, the main contribution of this thesis is the adoption of stochastic real-time heuristic search for a resource allocation. To this end, we have developed an approach based on distributed Q-values with tight bounds to diminish drastically the planning time to formulate the optimal policy. These tight bounds enable to prune the action space for the agents. We show analytically and empirically that our proposed approaches lead to drastic (in many cases, exponential) improvements in computational efficiency over standard planning methods. Finally, we have tested real-time heuristic search in the SADM simulator, a simulator for the resource allocation of a platform

Managing Epistemic Uncertainty in Design Models through Type-2 Fuzzy Logic Multidisciplinary Optimization

Humans have a natural ability to operate in dynamic environments and perform complex tasks with little perceived effort. An experienced ship designer can intuitively understand the general consequences of design choices and the general attributes of a good vessel. A person's knowledge is often ill-structured, subjective, and imprecise, but still incredibly effective at capturing general patterns of the real-world or of a design space. Computers on the other hand, can rapidly perform a large number of precise computations using well-structured, objective mathematical models, providing detailed analyses and formal evaluations of a specfic set of design candidates. In ship design, which involves generating knowledge for decision-making through time, engineers interactively use their own mental models and information gathered from computer-based optimization tools to make decisions which steer a vessel's design. In recent decades, the belief that large synthesis codes can help achieve cutting-edge ship performance has led to an increased popularity of optimization methods, potentially leading to rewarding results. And while optimization has proven fruitful to structural engineering and the aerospace industry, its applicability to early-stage design is more limited for three main reasons. First, mathematical models are by definition a reduction which cannot properly describe all aspects of the ship design problem. Second, in multidisciplinary optimization, a low-fidelity model may incorrectly drive a design, biasing the system level solution. Finally, early-stage design is plagued with limited information, limiting the designer's ability to develop models to inform decisions. This research extends previously done work by incorporating type-2 fuzzy logic into a human-centric multidisciplinary optimization framework. The original framework used type-1 fuzzy logic to incorporate human expertise into optimization models through linguistic variables. However, a type-1 system does not properly account for the uncertainty associated with linguistic terms, and thus does not properly represent the uncertainty associated with a human mental model. This limitation is corrected with the type-2 fuzzy logic multidisciplinary optimization presented in this work, which more accurately models a designer's ability to "communicate, reason and make rational decisions in an environment of imprecision, uncertainty, incompleteness of information and partiality of truth" (Mendel et al., 2010). It uses fuzzy definitions of linguistic variables and rule banks to incorporate "human intelligence" into design models, and better handles the linguistic uncertainty inherent to human knowledge and communication. A general mathematical optimization proof of concept and a planing craft case study are presented in this dissertation to show how mathematical models can be enhanced by incorporating expert opinion into them. Additionally, the planing craft case study shows how human mental models can be leveraged to quickly estimate plausible values of ship parameters when no model exists, increasing the designer's ability to run optimization methods when information is limited.PHDNaval Architecture & Marine EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/145891/1/doriancb_1.pd