Power Flow and Optimal Power Flow via Physics-Informed Typed Graph Neural Networks

Esta tesis explora las redes neuronales de grafos tipados informadas por la física aplicadas al modelado de redes de transporte de energía eléctrica, concretamente a los problemas de flujo de potencia y flujo de potencia óptimo. Las redes de transporte de energía eléctrica son complejos sistemas interconectados, cruciales para garantizar un suministro estable de electricidad. Para lograr la transición hacia sistemas energéticos asequibles, fiables y sostenibles, la electrificación de diversos sectores económicos y la integración de fuentes de energía renovable en la red de transmisión han aumentado significativamente. La mayoría de las tecnologías para la generación de energía renovable añaden fluctuación e incertidumbre a la generación de electricidad, por lo que, para garantizar un suministro eléctrico eficiente y estable en todo momento, los operadores de la red de transporte deben realizar frecuentes simulaciones de flujo de potencia y de flujo de potencia óptimo para evaluar el estado de la red. Por estas razones es necesario investigar técnicas nuevas, flexibles y más eficientes para resolver estos análisis. Los recientes avances en el aprendizaje automático, y en particular en las redes neuronales artificiales, indican que estos métodos tienen potencial para resolver problemas de análisis de redes eléctricas de forma rápida y fiable. Hasta la fecha, pocos trabajos han intentado aprovechar las capacidades de aprendizaje de las redes neuronales artificiales para abordar estos temas. Sin embargo, la mayoría de los trabajos publicados no resuelven dos grandes retos: en primer lugar, la necesidad de grandes cantidades de datos de entrenamiento y, en segundo lugar, la falta de capacidad de generalización para analizar redes de transporte realistas con topología variable. En esta tesis, se superan estos inconvenientes introduciendo redes neuronales de grafos tipados, que están especializados para procesar datos estructurados en forma de grafos con distintos tipos de elementos. La red de transmisión puede representarse directamente como un grafo, y al asignar distintos tipos de nodos para representar los diferentes elementos de la red de transmisión, se incrementa la precisión y la interpretabilidad del modelo propuesto. El modelo resultante es un modelo de red de transporte adaptable que puede aplicarse a diversos problemas, como las aplicaciones de flujo de potencia y flujo de potencia óptimo que se presentan en esta tesis. El esquema de aprendizaje presentado está informado por la física, de forma que el entrenamiento no está supervisado, sino que incorpora información de las leyes físicas del sistema subyacente en la función de costo. Además, el modelo resultante puede probarse en redes eléctricas con diferentes configuraciones y, en el caso del flujo de potencia, con redes de diferentes tamaños. Se demuestra que el método propuesto, con las aplicaciones consideradas, consigue resultados similares a los obtenidos con un método convencional pero hasta cuatro órdenes de magnitud más rápido, sin necesidad de datos de entrenamiento y con capacidad de generalización a diferentes redes de transporte. Se puede concluir, por tanto, que el trabajo presentado en esta tesis ofrece un método basado en redes neuronales para agilizar la resolución del complejo sistema de ecuaciones no lineales presente en el problema de flujo de potencia, así como el problema de optimización con restricciones presente en el problema de flujo de potencia óptimo. Estos resultados proporcionan un valioso paso hacia el desarrollo de un sistema general para ayudar a los operadores de sistemas de transmisión a optimizar la integración de nuevas tecnologías en la red convencional, y mejorar la fiabilidad y sostenibilidad de los sistemas eléctricos.<br /

Intelligent data mining using artificial neural networks and genetic algorithms : techniques and applications

Data Mining (DM) refers to the analysis of observational datasets to find relationships and to summarize the data in ways that are both understandable and useful. Many DM techniques exist. Compared with other DM techniques, Intelligent Systems (ISs) based approaches, which include Artificial Neural Networks (ANNs), fuzzy set theory, approximate reasoning, and derivative-free optimization methods such as Genetic Algorithms (GAs), are tolerant of imprecision, uncertainty, partial truth, and approximation. They provide flexible information processing capability for handling real-life situations. This thesis is concerned with the ideas behind design, implementation, testing and application of a novel ISs based DM technique. The unique contribution of this thesis is in the implementation of a hybrid IS DM technique (Genetic Neural Mathematical Method, GNMM) for solving novel practical problems, the detailed description of this technique, and the illustrations of several applications solved by this novel technique. GNMM consists of three steps: (1) GA-based input variable selection, (2) Multi- Layer Perceptron (MLP) modelling, and (3) mathematical programming based rule extraction. In the first step, GAs are used to evolve an optimal set of MLP inputs. An adaptive method based on the average fitness of successive generations is used to adjust the mutation rate, and hence the exploration/exploitation balance. In addition, GNMM uses the elite group and appearance percentage to minimize the randomness associated with GAs. In the second step, MLP modelling serves as the core DM engine in performing classification/prediction tasks. An Independent Component Analysis (ICA) based weight initialization algorithm is used to determine optimal weights before the commencement of training algorithms. The Levenberg-Marquardt (LM) algorithm is used to achieve a second-order speedup compared to conventional Back-Propagation (BP) training. In the third step, mathematical programming based rule extraction is not only used to identify the premises of multivariate polynomial rules, but also to explore features from the extracted rules based on data samples associated with each rule. Therefore, the methodology can provide regression rules and features not only in the polyhedrons with data instances, but also in the polyhedrons without data instances. A total of six datasets from environmental and medical disciplines were used as case study applications. These datasets involve the prediction of longitudinal dispersion coefficient, classification of electrocorticography (ECoG)/Electroencephalogram (EEG) data, eye bacteria Multisensor Data Fusion (MDF), and diabetes classification (denoted by Data I through to Data VI). GNMM was applied to all these six datasets to explore its effectiveness, but the emphasis is different for different datasets. For example, the emphasis of Data I and II was to give a detailed illustration of how GNMM works; Data III and IV aimed to show how to deal with difficult classification problems; the aim of Data V was to illustrate the averaging effect of GNMM; and finally Data VI was concerned with the GA parameter selection and benchmarking GNMM with other IS DM techniques such as Adaptive Neuro-Fuzzy Inference System (ANFIS), Evolving Fuzzy Neural Network (EFuNN), Fuzzy ARTMAP, and Cartesian Genetic Programming (CGP). In addition, datasets obtained from published works (i.e. Data II & III) or public domains (i.e. Data VI) where previous results were present in the literature were also used to benchmark GNMM’s effectiveness. As a closely integrated system GNMM has the merit that it needs little human interaction. With some predefined parameters, such as GA’s crossover probability and the shape of ANNs’ activation functions, GNMM is able to process raw data until some human-interpretable rules being extracted. This is an important feature in terms of practice as quite often users of a DM system have little or no need to fully understand the internal components of such a system. Through case study applications, it has been shown that the GA-based variable selection stage is capable of: filtering out irrelevant and noisy variables, improving the accuracy of the model; making the ANN structure less complex and easier to understand; and reducing the computational complexity and memory requirements. Furthermore, rule extraction ensures that the MLP training results are easily understandable and transferrable

Sparse Distributed Memory is a Continual Learner

Continual learning is a problem for artificial neural networks that their biological counterparts are adept at solving. Building on work using Sparse Distributed Memory (SDM) to connect a core neural circuit with the powerful Transformer model, we create a modified Multi-Layered Perceptron (MLP) that is a strong continual learner. We find that every component of our MLP variant translated from biology is necessary for continual learning. Our solution is also free from any memory replay or task information, and introduces novel methods to train sparse networks that may be broadly applicable.Comment: 9 Pages. ICLR Acceptanc

Étude de la propagation acoustique en milieu complexe par des réseaux de neurones profonds

Abstract : Predicting the propagation of aerocoustic noise is a challenging task in the presence of complex mean flows and geometry installation effects. The design of future quiet propul- sion systems requires tools that are able to perform many accurate evaluations with a low computational cost. Analytical models or hybrid numerical approaches have tradition- ally been employed for that purpose. However, such methods are typically constrained by simplifying hypotheses that are not easily relaxed. Thus, the main objective of this thesis is to develop and validate novel methods for the fast and accurate prediction of aeroacoustic propagation in complex mean flows and geometries. For that, data-driven deep convolutional neural networks acting as auto-regressive spatio-temporal predictors are considered. These surrogates are trained on high-fidelity data, generated by direct aeroacoustic numerical solvers. Such datasets are able to model complex flow phenomena, along with complex geometrical parameters. The neural network is designed to substitute the high-fidelity solver at a much lower computational cost once the training is finished, while predicting the time-domain acoustic propagation with sufficient accuracy. Three test cases of growing complexity are employed to test the approach, where the learned surrogate is compared to analytical and numerical solutions. The first one corresponds to the two-dimensional propagation of Gaussian pulses in closed domains, which allows understanding the fundamental behavior of the employed convolution neural networks. Second, the approach is extended in order to consider a variety of boundary conditions, from non-reflecting to curved reflecting obstacles, including the reflection and scattering of waves at obstacles. This allows the prediction of acoustic propagation in configurations closer to industrial problems. Finally, the effects of complex mean flows is investigated through a dataset of acoustic waves propagating inside sheared flows. These applications highlight the flexibility of the employed data-driven methods using convolutional neural networks. They allow a significant acceleration of the acoustic predictions, while keeping an adequate accuracy and being also able to correctly predict the acoustic propagation outside the range of the training data. For that, prior knowledge about the wave propa- gation physics is included during and after the neural network training phase, allowing an increased control over the error performed by the surrogate. Among this prior knowledge, the conservation of physics quantities and the correct treatment of boundary conditions are identified as key parameters that improve the surrogate predictions.Prédire la propagation du bruit aéroacoustique est une tâche difficile en présence d’écoulements moyens complexes et d’effets géométriques d’installation. La conception des futurs systèmes de propulsion silencieux appelle au développement d’outils capables d’effectuer de nombreuses évaluations avec une faible erreur et un faible coût de calcul. Traditionnellement, des modèles analytiques ou des approches numériques hybrides ont été utilisés à cette fin. Cependant, ces méthodes sont généralement contraintes par des hypothèses simplificatrices qui ne sont pas facilement assouplies. Ainsi, l’objectif principal de cette thèse est de développer et de valider de nouvelles méthodes pour la prédiction rapide et précise de la propagation aéroacoustique dans des écoulements moyens et des géométries complexes. Pour cela, des réseaux de neurones profonds à convolution, entraînés sur des données, et agissant comme prédicteurs spatio-temporels sont considérés. Ces modèles par substitution sont entraînés sur des données de haute fidélité, générées par des solveurs numériques aérocoustiques directs. De telles bases de données sont capables de modéliser des phénomènes d’écoulement, ainsi que des paramètres géométriques complexes. Le réseau de neurones est conçu pour remplacer le solveur haute fidélité à un coût de calcul beaucoup plus faible une fois la phase d’entraînement terminée, tout en prédisant la propagation acoustique dans le domaine temporel avec une précision suffisante. Trois cas de test, de complexité croissante, sont utilisés pour tester l’approche, où le substitut appris est comparé à des solutions analytiques et numériques. Le premier cas correspond à la propagation acoustique bidimensionnelle dans des domaines fermés, où des sources impulsionnelles Gaussiennes sont considérées. Ceci permet de comprendre le comportement fondamental des réseaux de neurones à convolution étudiés. Deuxièmement, l’approche est étendue afin de prendre en compte une variété de conditions aux limites, notamment des conditions aux limites non réfléchissantes et des obstacles réfléchissants de géométrie arbitraire, modélisant la réflexion et la diffusion des ondes acoustiques sur ces obstacles. Cela permet de prédire la propagation acoustique dans des configurations plus proches des problématiques industrielles. Enfin, les effets des écoulements moyens complexes sont étudiés à travers une base de données d’ondes acoustiques qui se propagent à l’intérieur d’écoulements cisaillés. Ces applications mettent en évidence la flexibilité des méthodes basées sur les données, utilisant des réseaux de neurones à convolution. Ils permettent une accélération significative des prédictions acoustiques, tout en gardant une précision adéquate et en étant également capables de prédire correctement la propagation acoustique en dehors de la gamme de paramètres des données d’apprentissage. Pour cela, des connaissances préalables sur la physique de propagation des ondes sont incluses pendant et après la phase d’apprentissage du réseau de neurones, permettant un contrôle accru sur l’erreur effectuée par le substitut. Parmi ces connaissances préalables, la conservation des grandeurs physiques et le traitement correct des conditions aux limites sont identifiés comme des paramètres clés qui améliorent les prédictions du modèle proposé

Low Power Memory/Memristor Devices and Systems

This reprint focusses on achieving low-power computation using memristive devices. The topic was designed as a convenient reference point: it contains a mix of techniques starting from the fundamental manufacturing of memristive devices all the way to applications such as physically unclonable functions, and also covers perspectives on, e.g., in-memory computing, which is inextricably linked with emerging memory devices such as memristors. Finally, the reprint contains a few articles representing how other communities (from typical CMOS design to photonics) are fighting on their own fronts in the quest towards low-power computation, as a comparison with the memristor literature. We hope that readers will enjoy discovering the articles within

Measuring fault resilience in neural networks

In an extension to research into modeling a biological network of neurons this expands the basic characteristics of an Artificial Neural Network (ANN)computational model to measure functional compensation exhibited by a biological neural network during damage or loss of structure. Whilst current research has highlighted the availability of various technologies and methods relevant to this area of study, none provide a sufficient description as to how fault tolerance is measured nor how damage is evaluated. Such metrics must be consistent, reproducible, and applicable to a plethora of neural network architectures and techniques. Furthermore, measuring fault resilience of biologically inspired ANN architectures provides insight into how biological networks are able to exhibit this amazing ability. This research brings together previous works into a comprehensive damage resilient ANN framework as well as, and more importantly, provides consistent measurement of fault tolerance within this framework. The proposed set of fault resilience metrics provides the means to evaluate the efficacy of networks which are subjectable to damage. These metrics and their source algorithms rely on the modification of various statistical methods and observations currently used for network training optimization