1,418 research outputs found
Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
On top of machine learning models, uncertainty quantification (UQ) functions
as an essential layer of safety assurance that could lead to more principled
decision making by enabling sound risk assessment and management. The safety
and reliability improvement of ML models empowered by UQ has the potential to
significantly facilitate the broad adoption of ML solutions in high-stakes
decision settings, such as healthcare, manufacturing, and aviation, to name a
few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods
for ML models with a particular focus on neural networks and the applications
of these UQ methods in tackling engineering design as well as prognostics and
health management problems. Toward this goal, we start with a comprehensive
classification of uncertainty types, sources, and causes pertaining to UQ of ML
models. Next, we provide a tutorial-style description of several
state-of-the-art UQ methods: Gaussian process regression, Bayesian neural
network, neural network ensemble, and deterministic UQ methods focusing on
spectral-normalized neural Gaussian process. Established upon the mathematical
formulations, we subsequently examine the soundness of these UQ methods
quantitatively and qualitatively (by a toy regression example) to examine their
strengths and shortcomings from different dimensions. Then, we review
quantitative metrics commonly used to assess the quality of predictive
uncertainty in classification and regression problems. Afterward, we discuss
the increasingly important role of UQ of ML models in solving challenging
problems in engineering design and health prognostics. Two case studies with
source codes available on GitHub are used to demonstrate these UQ methods and
compare their performance in the life prediction of lithium-ion batteries at
the early stage and the remaining useful life prediction of turbofan engines
2017 GREAT Day Program
SUNY Geneseo’s Eleventh Annual GREAT Day.https://knightscholar.geneseo.edu/program-2007/1011/thumbnail.jp
Contributions to improve the technologies supporting unmanned aircraft operations
Mención Internacional en el título de doctorUnmanned Aerial Vehicles (UAVs), in their smaller versions known as drones, are becoming increasingly important in today's societies. The systems that make them up present a multitude of challenges, of which error can be considered the common denominator. The perception of the environment is measured by sensors that have errors, the models that interpret the information and/or define behaviors are approximations of the world and therefore also have errors. Explaining error allows extending the limits of deterministic models to address real-world problems. The performance of the technologies embedded in drones depends on our ability to understand, model, and control the error of the systems that integrate them, as well as new technologies that may emerge.
Flight controllers integrate various subsystems that are generally dependent on other systems. One example is the guidance systems. These systems provide the engine's propulsion controller with the necessary information to accomplish a desired mission. For this purpose, the flight controller is made up of a control law for the guidance system that reacts to the information perceived by the perception and navigation systems. The error of any of the subsystems propagates through the ecosystem of the controller, so the study of each of them is essential.
On the other hand, among the strategies for error control are state-space estimators, where the Kalman filter has been a great ally of engineers since its appearance in the 1960s. Kalman filters are at the heart of information fusion systems, minimizing the error covariance of the system and allowing the measured states to be filtered and estimated in the absence of observations. State Space Models (SSM) are developed based on a set of hypotheses for modeling the world. Among the assumptions are that the models of the world must be linear, Markovian, and that the error of their models must be Gaussian. In general, systems are not linear, so linearization are performed on models that are already approximations of the world. In other cases, the noise to be controlled is not Gaussian, but it is approximated to that distribution in order to be able to deal with it. On the other hand, many systems are not Markovian, i.e., their states do not depend only on the previous state, but there are other dependencies that state space models cannot handle.
This thesis deals a collection of studies in which error is formulated and reduced. First, the error in a computer vision-based precision landing system is studied, then estimation and filtering problems from the deep learning approach are addressed. Finally, classification concepts with deep learning over trajectories are studied. The first case of the collection xviiistudies
the consequences of error propagation in a machine vision-based precision landing system. This paper proposes a set of strategies to reduce the impact on the guidance system, and ultimately reduce the error. The next two studies approach the estimation and filtering problem from the deep learning approach, where error is a function to be minimized by learning. The last case of the collection deals with a trajectory classification problem with real data. This work completes the two main fields in deep learning, regression and classification, where the error is considered as a probability function of class membership.Los vehículos aéreos no tripulados (UAV) en sus versiones de pequeño tamaño conocidos como drones, van tomando protagonismo en las sociedades actuales. Los sistemas que los componen presentan multitud de retos entre los cuales el error se puede considerar como el denominador común. La percepción del entorno se mide mediante sensores que tienen error, los modelos que interpretan la información y/o definen comportamientos son aproximaciones del mundo y por consiguiente también presentan error. Explicar el error permite extender los límites de los modelos deterministas para abordar problemas del mundo real. El rendimiento de las tecnologías embarcadas en los drones, dependen de nuestra capacidad de comprender, modelar y controlar el error de los sistemas que los integran, así como de las nuevas tecnologías que puedan surgir.
Los controladores de vuelo integran diferentes subsistemas los cuales generalmente son dependientes de otros sistemas. Un caso de esta situación son los sistemas de guiado. Estos sistemas son los encargados de proporcionar al controlador de los motores información necesaria para cumplir con una misión deseada. Para ello se componen de una ley de control de guiado que reacciona a la información percibida por los sistemas de percepción y navegación. El error de cualquiera de estos sistemas se propaga por el ecosistema del controlador siendo vital su estudio.
Por otro lado, entre las estrategias para abordar el control del error se encuentran los estimadores en espacios de estados, donde el filtro de Kalman desde su aparición en los años 60, ha sido y continúa siendo un gran aliado para los ingenieros. Los filtros de Kalman son el corazón de los sistemas de fusión de información, los cuales minimizan la covarianza del error del sistema, permitiendo filtrar los estados medidos y estimarlos cuando no se tienen observaciones. Los modelos de espacios de estados se desarrollan en base a un conjunto de hipótesis para modelar el mundo. Entre las hipótesis se encuentra que los modelos del mundo han de ser lineales, markovianos y que el error de sus modelos ha de ser gaussiano. Generalmente los sistemas no son lineales por lo que se realizan linealizaciones sobre modelos que a su vez ya son aproximaciones del mundo. En otros casos el ruido que se desea controlar no es gaussiano, pero se aproxima a esta distribución para poder abordarlo. Por otro lado, multitud de sistemas no son markovianos, es decir, sus estados no solo dependen del estado anterior, sino que existen otras dependencias que los modelos de espacio de estados no son capaces de abordar. Esta tesis aborda un compendio de estudios sobre los que se formula y reduce el error. En primer lugar, se estudia el error en un sistema de aterrizaje de precisión basado en visión por computador. Después se plantean problemas de estimación y filtrado desde la aproximación del aprendizaje profundo. Por último, se estudian los conceptos de clasificación con aprendizaje profundo sobre trayectorias. El primer caso del compendio estudia las consecuencias de la propagación del error de un sistema de aterrizaje de precisión basado en visión artificial. En este trabajo se propone un conjunto de estrategias para reducir el impacto sobre el sistema de guiado, y en última instancia reducir el error. Los siguientes dos estudios abordan el problema de estimación y filtrado desde la perspectiva del aprendizaje profundo, donde el error es una función que minimizar mediante aprendizaje. El último caso del compendio aborda un problema de clasificación de trayectorias con datos reales. Con este trabajo se completan los dos campos principales en aprendizaje profundo, regresión y clasificación, donde se plantea el error como una función de probabilidad de pertenencia a una clase.I would like to thank the Ministry of Science and Innovation for granting me the funding with reference PRE2018-086793, associated to the project TEC2017-88048-C2-2-R, which provide me the opportunity to carry out all my PhD. activities, including completing an international research internship.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: Antonio Berlanga de Jesús.- Secretario: Daniel Arias Medina.- Vocal: Alejandro Martínez Cav
From model-driven to data-driven : a review of hysteresis modeling in structural and mechanical systems
Hysteresis is a natural phenomenon that widely exists in structural and mechanical systems. The characteristics of structural hysteretic behaviors are complicated. Therefore, numerous methods have been developed to describe hysteresis. In this paper, a review of the available hysteretic modeling methods is carried out. Such methods are divided into: a) model-driven and b) datadriven methods. The model-driven method uses parameter identification to determine parameters. Three types of parametric models are introduced including polynomial models, differential based models, and operator based models. Four algorithms as least mean square error algorithm, Kalman filter algorithm, metaheuristic algorithms, and Bayesian estimation are presented to realize parameter identification. The data-driven method utilizes universal mathematical models to describe hysteretic behavior. Regression model, artificial neural network, least square support vector machine, and deep learning are introduced in turn as the classical data-driven methods. Model-data driven hybrid methods are also discussed to make up for the shortcomings of the two methods. Based on a multi-dimensional evaluation, the existing problems and open challenges of different hysteresis modeling methods are discussed. Some possible research directions about hysteresis description are given in the final section
Recommended from our members
Machine Learning Framework for Causal Modeling for Process Fault Diagnosis and Mechanistic Explanation Generation
Machine learning models, typically deep learning models, often come at the cost of explainability. To generate explanations of such systems, models need to be rooted in first-principles, at least mechanistically. In this work we look at a gamete of machine learning models based on different levels of process knowledge for process fault diagnosis and generating mechanistic explanations of processes. In chapter 1, we introduce the thesis using a range of problems from causality, explainability, aiming towards the goal of generating mechanistic explanations of process systems. Chapter 2 looks at an approach for generating causal models purely through data-centric approach, with minimal process knowledge with respect to equipment connectivity and identifying causality in the domains. These causal models generated can be utilized for process fault diagnosis.
Chapter 3 and chapter 4 show how deep learning models can be used for both classification for process fault diagnosis and regression. We see that depending on the hyperparameters, i.e., purely the breadth and depth of a neural network, the learned hidden representations vary from a simple set of features, to more complex sets of features. While these hidden representations may be exploited to aid in classification and regression problems, the true explanations of these representations do not correlate with mechanisms in the system of interest. There is thus a requirement to add more mechanistic information about the features generated to aid in explainability.
Chapter 5 shows how incorporating process knowledge can aid in generating such mechanistic explanations based on automated variable transformations. In this chapter we show how process knowledge can be used to generate features, or model forms to generate explainable models. These models have the ability of extracting the true models of the system from the model knowledge provided
Discovering Causal Relations and Equations from Data
Physics is a field of science that has traditionally used the scientific
method to answer questions about why natural phenomena occur and to make
testable models that explain the phenomena. Discovering equations, laws and
principles that are invariant, robust and causal explanations of the world has
been fundamental in physical sciences throughout the centuries. Discoveries
emerge from observing the world and, when possible, performing interventional
studies in the system under study. With the advent of big data and the use of
data-driven methods, causal and equation discovery fields have grown and made
progress in computer science, physics, statistics, philosophy, and many applied
fields. All these domains are intertwined and can be used to discover causal
relations, physical laws, and equations from observational data. This paper
reviews the concepts, methods, and relevant works on causal and equation
discovery in the broad field of Physics and outlines the most important
challenges and promising future lines of research. We also provide a taxonomy
for observational causal and equation discovery, point out connections, and
showcase a complete set of case studies in Earth and climate sciences, fluid
dynamics and mechanics, and the neurosciences. This review demonstrates that
discovering fundamental laws and causal relations by observing natural
phenomena is being revolutionised with the efficient exploitation of
observational data, modern machine learning algorithms and the interaction with
domain knowledge. Exciting times are ahead with many challenges and
opportunities to improve our understanding of complex systems.Comment: 137 page
Specialized astrocytes mediate glutamatergic gliotransmission in the CNS
Multimodal astrocyte–neuron communications govern brain circuitry assembly and function1. For example, through rapid glutamate release, astrocytes can control excitability, plasticity and synchronous activity2,3 of synaptic networks, while also contributing to their dysregulation in neuropsychiatric conditions4–7. For astrocytes to communicate through fast focal glutamate release, they should possess an apparatus for Ca2+-dependent exocytosis similar to neurons8–10. However, the existence of this mechanism has been questioned11–13 owing to inconsistent data14–17 and a lack of direct supporting evidence. Here we revisited the astrocyte glutamate exocytosis hypothesis by considering the emerging molecular heterogeneity of astrocytes18–21 and using molecular, bioinformatic and imaging approaches, together with cell-specific genetic tools that interfere with glutamate exocytosis in vivo. By analysing existing single-cell RNA-sequencing databases and our patch-seq data, we identified nine molecularly distinct clusters of hippocampal astrocytes, among which we found a notable subpopulation that selectively expressed synaptic-like glutamate-release machinery and localized to discrete hippocampal sites. Using GluSnFR-based glutamate imaging22 in situ and in vivo, we identified a corresponding astrocyte subgroup that responds reliably to astrocyte-selective stimulations with subsecond glutamate release events at spatially precise hotspots, which were suppressed by astrocyte-targeted deletion of vesicular glutamate transporter 1 (VGLUT1). Furthermore, deletion of this transporter or its isoform VGLUT2 revealed specific contributions of glutamatergic astrocytes in cortico-hippocampal and nigrostriatal circuits during normal behaviour and pathological processes. By uncovering this atypical subpopulation of specialized astrocytes in the adult brain, we provide insights into the complex roles of astrocytes in central nervous system (CNS) physiology and diseases, and identify a potential therapeutic target
- …