Machine learning and its applications in reliability analysis systems

In this thesis, we are interested in exploring some aspects of Machine Learning (ML) and its application in the Reliability Analysis systems (RAs). We begin by investigating some ML paradigms and their- techniques, go on to discuss the possible applications of ML in improving RAs performance, and lastly give guidelines of the architecture of learning RAs. Our survey of ML covers both levels of Neural Network learning and Symbolic learning. In symbolic process learning, five types of learning and their applications are discussed: rote learning, learning from instruction, learning from analogy, learning from examples, and learning from observation and discovery. The Reliability Analysis systems (RAs) presented in this thesis are mainly designed for maintaining plant safety supported by two functions: risk analysis function, i.e., failure mode effect analysis (FMEA) ; and diagnosis function, i.e., real-time fault location (RTFL). Three approaches have been discussed in creating the RAs. According to the result of our survey, we suggest currently the best design of RAs is to embed model-based RAs, i.e., MORA (as software) in a neural network based computer system (as hardware). However, there are still some improvement which can be made through the applications of Machine Learning. By implanting the 'learning element', the MORA will become learning MORA (La MORA) system, a learning Reliability Analysis system with the power of automatic knowledge acquisition and inconsistency checking, and more. To conclude our thesis, we propose an architecture of La MORA

Fault diagnosis in a five-level multilevel inverter using an artificial neural network approach

Introduction. Cascaded H-bridge multilevel inverters (CHB-MLI) are becoming increasingly used in applications such as distribution systems, electrical traction systems, high voltage direct conversion systems, and many others. Despite the fact that multilevel inverters contain a large number of control switches, detecting a malfunction takes a significant amount of time. In the fault switch configurations diode included for freewheeling operation during open-fault condition. During short circuit fault conditions are carried out by the fuse, which can reveal the freewheeling current direction. The fault category can be identified independently and also failure of power switches harmed by the functioning and reliability of CHB-MLI. This paper investigates the effects and performance of open and short switching faults of multilevel inverters. Output voltage characteristics of 5 level MLI are frequently determined from distinctive switch faults with modulation index value of 0.85 is used during simulation analysis. In the simulation experiment for the modulation index value of 0.85, one second open and short circuit faults are created for the place of faulty switch. Fault is identified automatically by means of artificial neural network (ANN) technique using sinusoidal pulse width modulation based on distorted total harmonic distortion (THD) and managed by its own. The novelty of the proposed work consists of a fast Fourier transform (FFT) and ANN to identify faulty switch. Purpose. The proposed architecture is to identify faulty switch during open and short failures, which has to be reduced THD and make the system in reliable operation. Methods. The proposed topology is to be design and evaluate using MATLAB/Simulink platform. Results. Using the FFT and ANN approaches, the normal and faulty conditions of the MLI are explored, and the faulty switch is detected based on voltage changing patterns in the output. Practical value. The proposed topology has been very supportive for implementing non-conventional energy sources based multilevel inverter, which is connected to large demand in grid.Вступ. Каскадні багаторівневі інвертори H-bridge все частіше використовуються в таких пристроях, як розподільні системи, електричні тягові системи, системи прямого перетворення високої напруги та багато інших. Незважаючи на те, що багаторівневі інвертори містять велику кількість перемикачів, що управляють, виявлення несправності займає значний час. У конфігурації аварійного вимикача увімкнено діод для роботи в режимі вільного ходу в умовах обриву несправності. При короткому замиканні аварійні стани виконуються запобіжником, який може визначити напрямок струму вільного ходу. Категорія несправності може бути визначена самостійно, а також відмова силових вимикачів, що порушує функціонування та надійність каскадних багаторівневих інверторів H-bridge. У цій статті досліджуються наслідки та характеристики обривів та коротких замикань багаторівневих інверторів. Характеристики вихідної напруги 5-рівневого інвертору часто визначаються характерними несправностями перемикача, при цьому при аналізі моделювання використовується значення індексу модуляції 0,85. В імітаційному експерименті значення індексу модуляції 0,85 в місці несправного перемикача створюються односекундні обриви і коротке замикання. Несправність ідентифікується автоматично за допомогою методу штучної нейронної мережі з використанням синусоїдальної широтно-імпульсної модуляції на основі спотвореного повного гармонійного спотворення та керується самостійно. Новизна запропонованої роботи полягає у застосуванні швидкого перетворення Фур’є та штучної нейронної мережі для ідентифікації несправного перемикача. Мета. Пропонована архітектура призначена для виявлення несправного комутатора при розмиканні та короткочасних відмовах, що має знизити повне гармонійне спотворення та забезпечити надійну роботу системи. методи. Запропонована топологія має бути спроектована та оцінена з використанням платформи MATLAB/Simulink. Результати. Використовуючи підходи швидкого перетворення Фур’є та штучної нейронної мережі, досліджуються нормальні та несправні стани багаторівневих інверторів, і несправний перемикач виявляється на основі моделей зміни напруги на виході. Практична цінність. Запропонована топологія дуже сприятлива для реалізації нетрадиційних джерел енергії на основі багаторівневого інвертора, пов'язаного з великим попитом у мережі

Autonomously Reconfigurable Artificial Neural Network on a Chip

Artificial neural network (ANN), an established bio-inspired computing paradigm, has proved very effective in a variety of real-world problems and particularly useful for various emerging biomedical applications using specialized ANN hardware. Unfortunately, these ANN-based systems are increasingly vulnerable to both transient and permanent faults due to unrelenting advances in CMOS technology scaling, which sometimes can be catastrophic. The considerable resource and energy consumption and the lack of dynamic adaptability make conventional fault-tolerant techniques unsuitable for future portable medical solutions. Inspired by the self-healing and self-recovery mechanisms of human nervous system, this research seeks to address reliability issues of ANN-based hardware by proposing an Autonomously Reconfigurable Artificial Neural Network (ARANN) architectural framework. Leveraging the homogeneous structural characteristics of neural networks, ARANN is capable of adapting its structures and operations, both algorithmically and microarchitecturally, to react to unexpected neuron failures. Specifically, we propose three key techniques --- Distributed ANN, Decoupled Virtual-to-Physical Neuron Mapping, and Dual-Layer Synchronization --- to achieve cost-effective structural adaptation and ensure accurate system recovery. Moreover, an ARANN-enabled self-optimizing workflow is presented to adaptively explore a "Pareto-optimal" neural network structure for a given application, on the fly. Implemented and demonstrated on a Virtex-5 FPGA, ARANN can cover and adapt 93% chip area (neurons) with less than 1% chip overhead and O(n) reconfiguration latency. A detailed performance analysis has been completed based on various recovery scenarios

Development of a Reference Design for Intrusion Detection Using Neural Networks for a Smart Inverter

The purpose of this thesis is to develop a reference design for a base level implementation of an intrusion detection module using artificial neural networks that is deployed onto an inverter and runs on live data for cybersecurity purposes, leveraging the latest deep learning algorithms and tools. Cybersecurity in the smart grid industry focuses on maintaining optimal standards of security in the system and a key component of this is being able to detect cyberattacks. Although researchers and engineers aim to design such devices with embedded security, attacks can and do still occur. The foundation for eventually mitigating these attacks and achieving more robust security is to identify them reliably. Thus, a high-fidelity intrusion detection system (IDS) capable of identifying a variety of attacks must be implemented. This thesis provides an implementation of a behavior-based intrusion detection system that uses a recurrent artificial neural network deployed on hardware to detect cyberattacks in real time. Leveraging the growing power of artificial intelligence, the strength of this approach is that given enough data, it is capable of learning to identify highly complex patterns in the data that may even go undetected by humans. By intelligently identifying malicious activity at the fundamental behavior level, the IDS remains robust against new methods of attack. This work details the process of collecting and simulating data, selecting the particular algorithm, training the neural network, deploying the neural network onto hardware, and then being able to easily update the deployed model with a newly trained one. The full system is designed with a focus on modularity, such that it can be easily adapted to perform well on different use cases, different hardware, and fulfill changing requirements. The neural network behavior-based IDS is found to be a very powerful method capable of learning highly complex patterns and identifying intrusion from different types of attacks using a single unified algorithm, achieving up to 98% detection accuracy in distinguishing between normal and anomalous behavior. Due to the ubiquitous nature of this approach, the pipeline developed here can be applied in the future to build in more and more sophisticated detection abilities depending on the desired use case. The intrusion detection module is implemented in an ARM processor that exists at the communication layer of the inverter. There are four main components described in this thesis that explain the process of deploying an artificial neural network intrusion detection algorithm onto the inverter: 1) monitoring and collecting data through a front-end web based graphical user interface that interacts with a Digital Signal Processor that is connected to power-electronics, 2) simulating various malicious datasets based on attack vectors that violate the Confidentiality-Integrity-Availability security model, 3) training and testing the neural network to ensure that it successfully identifies normal behavior and malicious behavior with a high degree of accuracy, and lastly 4) deploying the machine learning algorithm onto the hardware and having it successfully classify the behavior as normal or malicious with the data feeding into the model running in real time. The results from the experimental setup will be analyzed, a conclusion will be made based upon the work, and lastly discussions of future work and optimizations will be discussed

Biologically inspired computational structures and processes for autonomous agents and robots

Recent years have seen a proliferation of intelligent agent applications: from robots for space exploration to software agents for information filtering and electronic commerce on the Internet. Although the scope of these agent applications have blossomed tremendously since the advent of compact, affordable computing (and the recent emergence of the World Wide Web), the design of such agents for specific applications remains a daunting engineering problem;Rather than approach the design of artificial agents from a purely engineering standpoint, this dissertation views animals as biological agents, and considers artificial analogs of biological structures and processes in the design of effective agent behaviors. In particular, it explores behaviors generated by artificial neural structures appropriately shaped by the processes of evolution and spatial learning;The first part of this dissertation deals with the evolution of artificial neural controllers for a box-pushing robot task. We show that evolution discovers high fitness structures using little domain-specific knowledge, even in feedback-impoverished environments. Through a careful analysis of the evolved designs we also show how evolution exploits the environmental constraints and properties to produce designs of superior adaptive value. By modifying the task constraints in controlled ways, we also show the ability of evolution to quickly adapt to these changes and exploit them to obtain significant performance gains. We also use evolution to design the sensory systems of the box-pushing robots, particularly the number, placement, and ranges of their sensors. We find that evolution automatically discards unnecessary sensors retaining only the ones that appear to significantly affect the performance of the robot. This optimization of design across multiple dimensions (performance, number of sensors, size of neural controller, etc.) is implicitly achieved by the evolutionary algorithm without any external pressure (e.g., penalty on the use of more sensors or neurocontroller units). When used in the design of robots with limited battery capacities , evolution produces energy-efficient robot designs that use minimal numbers of components and yet perform reasonably well. The performance as well as the complexity of robot designs increase when the robots have access to a spatial learning mechanism that allows them to learn, remember, and navigate to power sources in the environment;The second part of this dissertation develops a computational characterization of the hippocampal formation which is known to play a significant role in animal spatial learning. The model is based on neuroscientific and behavioral data, and learns place maps based on interactions of sensory and dead-reckoning information streams. Using an estimation mechanism known as Kalman filtering, the model explicitly deals with uncertainties in the two information streams, allowing the robot to effectively learn and localize even in the presence sensing and motion errors. Additionally, the model has mechanisms to handle perceptual aliasing problems (where multiple places in the environment appear sensorily identical), incrementally learn and integrate local place maps, and learn and remember multiple goal locations in the environment. We show a number of properties of this spatial learning model including computational replication of several behavioral experiments performed with rodents. Not only does this model make significant contributions to robot localization, but also offers a number of predictions and suggestions that can be validated (or refuted) through systematic neurobiological and behavioral experiments with animals

Nature-Inspired Topology Optimization of Recurrent Neural Networks

Hand-crafting effective and efficient structures for recurrent neural networks (RNNs) is a difficult, expensive, and time-consuming process. To address this challenge, this work presents three nature-inspired (NI) algorithms for neural architecture search (NAS), introducing the subfield of nature-inspired neural architecture search (NI-NAS). These algorithms, based on ant colony optimization (ACO), progress from memory cell structure optimization, to bounded discrete-space architecture optimization, and finally to unbounded continuous-space architecture optimization. These methods were applied to real-world data sets representing challenging engineering problems, such as data from a coal-fired power plant, wind-turbine power generators, and aircraft flight data recorder (FDR) data. Initial work utilized ACO to select optimal connections inside recurrent long short-term memory (LSTM) cell structures. Viewing each LSTM cell as a graph, ants would choose potential input and output connections based on the pheromones previously laid down over those connections as done in a standard ACO search. However, this approach did not optimize the overall network of the RNN, particularly its synaptic parameters. I addressed this issue by introducing the Ant-based Neural Topology Search (ANTS) algorithm to directly optimize the entire RNN topology. ANTS utilizes a discrete-space superstructure representing a completely connected RNN where each node is connected to every other node, forming an extremely dense mesh of edges and recurrent edges. ANTS can select from a library of modern RNN memory cells. ACO agents (ants), in this thesis, build RNNs from the superstructure determined by pheromones laid out on the superstructure\u27s connections. Backpropagation is then used to train the generated RNNs in an asynchronous parallel computing design to accelerate the optimization process. The pheromone update depends on the evaluation of the tested RNN against a population of best performing RNNs. Several variations of the core algorithm was investigated to test several designed heuristics for ANTS and evaluate their efficacy in the formation of sparser synaptic connectivity patterns. This was done primarily by formulating different functions that drive the underlying pheromone simulation process as well as by introducing ant agents with 3 specialized roles (inspired by real-world ants) to construct the RNN structure. This characterization of the agents enables ants to focus on specific structure building roles. ``Communal intelligence\u27\u27 was also incorporated, where the best set of weights was across locally-trained RNN candidates for weight initialization, reducing the number of backpropagation epochs required to train each candidate RNN and speeding up the overall search process. However, the growth of the superstructure increased by an order of magnitude, as more input and deeper structures are utilized, proving to be one limitation of the proposed procedure. The limitation of ANTS motivated the development of the continuous ANTS algorithm (CANTS), which works with a continuous search space for any fixed network topology. In this process, ants moving within a (temporally-arranged) set of continuous/real-valued planes based on proximity and density of pheromone placements. The motion of the ants over these continuous planes, in a sense, more closely mimicks how actual ants move in the real world. Ants traverse a 3-dimensional space from the inputs to the outputs and across time lags. This continuous search space frees the ant agents from the limitations imposed by ANTS\u27 discrete massively connected superstructure, making the structural options unbounded when mapping the movements of ants through the 3D continuous space to a neural architecture graph. In addition, CANTS has fewer hyperparameters to tune than ANTS, which had five potential heuristic components that each had their own unique set of hyperparameters, as well as requiring the user to define the maximum recurrent depth, number of layers and nodes within each layer. CANTS only requires specifying the number ants and their pheromone sensing radius. The three applied strategies yielded three important successes. Applying ACO on optimizing LSTMs yielded a 1.34\% performance enhancement and more than 55% sparser structures (which is useful for speeding up inference). ANTS outperformed the NAS benchmark, NEAT, and the NAS state-of-the-art algorithm, EXAMM. CANTS showed competitive results to EXAMM and competed with ANTS while offering sparser structures, offering a promising path forward for optimizing (temporal) neural models with nature-inspired metaheuristics based the metaphor of ants

Virtual metrology for plasma etch processes.

Plasma processes can present dicult control challenges due to time-varying dynamics and a lack of relevant and/or regular measurements. Virtual metrology (VM) is the use of mathematical models with accessible measurements from an operating process to estimate variables of interest. This thesis addresses the challenge of virtual metrology for plasma processes, with a particular focus on semiconductor plasma etch. Introductory material covering the essentials of plasma physics, plasma etching, plasma measurement techniques, and black-box modelling techniques is rst presented for readers not familiar with these subjects. A comprehensive literature review is then completed to detail the state of the art in modelling and VM research for plasma etch processes. To demonstrate the versatility of VM, a temperature monitoring system utilising a state-space model and Luenberger observer is designed for the variable specic impulse magnetoplasma rocket (VASIMR) engine, a plasma-based space propulsion system. The temperature monitoring system uses optical emission spectroscopy (OES) measurements from the VASIMR engine plasma to correct temperature estimates in the presence of modelling error and inaccurate initial conditions. Temperature estimates within 2% of the real values are achieved using this scheme. An extensive examination of the implementation of a wafer-to-wafer VM scheme to estimate plasma etch rate for an industrial plasma etch process is presented. The VM models estimate etch rate using measurements from the processing tool and a plasma impedance monitor (PIM). A selection of modelling techniques are considered for VM modelling, and Gaussian process regression (GPR) is applied for the rst time for VM of plasma etch rate. Models with global and local scope are compared, and modelling schemes that attempt to cater for the etch process dynamics are proposed. GPR-based windowed models produce the most accurate estimates, achieving mean absolute percentage errors (MAPEs) of approximately 1:15%. The consistency of the results presented suggests that this level of accuracy represents the best accuracy achievable for the plasma etch system at the current frequency of metrology. Finally, a real-time VM and model predictive control (MPC) scheme for control of plasma electron density in an industrial etch chamber is designed and tested. The VM scheme uses PIM measurements to estimate electron density in real time. A predictive functional control (PFC) scheme is implemented to cater for a time delay in the VM system. The controller achieves time constants of less than one second, no overshoot, and excellent disturbance rejection properties. The PFC scheme is further expanded by adapting the internal model in the controller in real time in response to changes in the process operating point

Virtual metrology for plasma etch processes.

Plasma processes can present dicult control challenges due to time-varying dynamics and a lack of relevant and/or regular measurements. Virtual metrology (VM) is the use of mathematical models with accessible measurements from an operating process to estimate variables of interest. This thesis addresses the challenge of virtual metrology for plasma processes, with a particular focus on semiconductor plasma etch. Introductory material covering the essentials of plasma physics, plasma etching, plasma measurement techniques, and black-box modelling techniques is rst presented for readers not familiar with these subjects. A comprehensive literature review is then completed to detail the state of the art in modelling and VM research for plasma etch processes. To demonstrate the versatility of VM, a temperature monitoring system utilising a state-space model and Luenberger observer is designed for the variable specic impulse magnetoplasma rocket (VASIMR) engine, a plasma-based space propulsion system. The temperature monitoring system uses optical emission spectroscopy (OES) measurements from the VASIMR engine plasma to correct temperature estimates in the presence of modelling error and inaccurate initial conditions. Temperature estimates within 2% of the real values are achieved using this scheme. An extensive examination of the implementation of a wafer-to-wafer VM scheme to estimate plasma etch rate for an industrial plasma etch process is presented. The VM models estimate etch rate using measurements from the processing tool and a plasma impedance monitor (PIM). A selection of modelling techniques are considered for VM modelling, and Gaussian process regression (GPR) is applied for the rst time for VM of plasma etch rate. Models with global and local scope are compared, and modelling schemes that attempt to cater for the etch process dynamics are proposed. GPR-based windowed models produce the most accurate estimates, achieving mean absolute percentage errors (MAPEs) of approximately 1:15%. The consistency of the results presented suggests that this level of accuracy represents the best accuracy achievable for the plasma etch system at the current frequency of metrology. Finally, a real-time VM and model predictive control (MPC) scheme for control of plasma electron density in an industrial etch chamber is designed and tested. The VM scheme uses PIM measurements to estimate electron density in real time. A predictive functional control (PFC) scheme is implemented to cater for a time delay in the VM system. The controller achieves time constants of less than one second, no overshoot, and excellent disturbance rejection properties. The PFC scheme is further expanded by adapting the internal model in the controller in real time in response to changes in the process operating point