269 research outputs found

    Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial

    Full text link
    On top of machine learning models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Toward this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. Two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage and the remaining useful life prediction of turbofan engines

    Multi-Fidelity Bayesian Optimization for Efficient Materials Design

    Get PDF
    Materials design is a process of identifying compositions and structures to achieve desirable properties. Usually, costly experiments or simulations are required to evaluate the objective function for a design solution. Therefore, one of the major challenges is how to reduce the cost associated with sampling and evaluating the objective. Bayesian optimization is a new global optimization method which can increase the sampling efficiency with the guidance of the surrogate of the objective. In this work, a new acquisition function, called consequential improvement, is proposed for simultaneous selection of the solution and fidelity level of sampling. With the new acquisition function, the subsequent iteration is considered for potential selections at low-fidelity levels, because evaluations at the highest fidelity level are usually required to provide reliable objective values. To reduce the number of samples required to train the surrogate for molecular design, a new recursive hierarchical similarity metric is proposed. The new similarity metric quantifies the differences between molecules at multiple levels of hierarchy simultaneously based on the connections between multiscale descriptions of the structures. The new methodologies are demonstrated with simulation-based design of materials and structures based on fully atomistic and coarse-grained molecular dynamics simulations, and finite-element analysis. The new similarity metric is demonstrated in the design of tactile sensors and biodegradable oligomers. The multi-fidelity Bayesian optimization method is also illustrated with the multiscale design of a piezoelectric transducer by concurrently optimizing the atomic composition of the aluminum titanium nitride ceramic and the device’s porous microstructure at the micrometer scale.Ph.D

    Ny forståelse av gasshydratfenomener og naturlige inhibitorer i råoljesystemer gjennom massespektrometri og maskinlæring

    Get PDF
    Gas hydrates represent one of the main flow assurance issues in the oil and gas industry as they can cause complete blockage of pipelines and process equipment, forcing shut downs. Previous studies have shown that some crude oils form hydrates that do not agglomerate or deposit, but remain as transportable dispersions. This is commonly believed to be due to naturally occurring components present in the crude oil, however, despite decades of research, their exact structures have not yet been determined. Some studies have suggested that these components are present in the acid fractions of the oils or are related to the asphaltene content of the oils. Crude oils are among the worlds most complex organic mixtures and can contain up to 100 000 different constituents, making them difficult to characterise using traditional mass spectrometers. The high mass accuracy of Fourier Transform Ion Cyclotron Resonance Mass Spectrometry (FT-ICR MS) yields a resolution greater than traditional techniques, making FT-ICR MS able to characterise crude oils to a greater extent, and possibly identify hydrate active components. FT-ICR MS spectra usually contain tens of thousands of peaks, and data treatment methods able to find underlying relationships in big data sets are required. Machine learning and multivariate statistics include many methods suitable for big data. A literature review identified a number of promising methods, and the current status for the use of machine learning for analysis of gas hydrates and FT-ICR MS data was analysed. The literature study revealed that although many studies have used machine learning to predict thermodynamic properties of gas hydrates, very little work have been done in analysing gas hydrate related samples measured by FT-ICR MS. In order to aid their identification, a successive accumulation procedure for increasing the concentrations of hydrate active components was developed by SINTEF. Comparison of the mass spectra from spiked and unspiked samples revealed some peaks that increased in intensity over the spiking levels. Several classification methods were used in combination with variable selection, and peaks related to hydrate formation were identified. The corresponding molecular formulas were determined, and the peaks were assumed to be related to asphaltenes, naphthenes and polyethylene glycol. To aid the characterisation of the oils, infrared spectroscopy (both Fourier Transform infrared and near infrared) was combined with FT-ICR MS in a multiblock analysis to predict the density of crude oils. Two different strategies for data fusion were attempted, and sequential fusion of the blocks achieved the highest prediction accuracy both before and after reducing the dimensions of the data sets by variable selection. As crude oils have such complex matrixes, samples are often very different, and many methods are not able to handle high degrees of variations or non-linearities between the samples. Hierarchical cluster-based partial least squares regression (HC-PLSR) clusters the data and builds local models within each cluster. HC-PLSR can thus handle non-linearities between clusters, but as PLSR is a linear model the data is still required to be locally linear. HC-PLSR was therefore expanded into deep learning (HC-CNN and HC-RNN) and SVR (HC-SVR). The deep learning-based models outperformed HC-PLSR for a data set predicting average molecular weights from hydrolysed raw materials. The analysis of the FT-ICR MS spectra revealed that the large amounts of information contained in the data (due to the high resolution) can disturb the predictive models, but the use of variable selection counteracts this effect. Several methods from machine learning and multivariate statistics were proven valuable for prediction of various parameters from FT-ICR MS using both classification and regression methods.Gasshydrater er et av hovedproblemene for Flow assurance i olje- og gassnæringen ettersom at de kan forårsake blokkeringer i oljerørledninger og prosessutstyr som krever at systemet må stenges ned. Tidligere studier har vist at noen råoljer danner hydrater som ikke agglomererer eller avsetter, men som forblir som transporterbare dispersjoner. Dette antas å være på grunn av naturlig forekommende komponenter til stede i råoljen, men til tross for årevis med forskning er deres nøyaktige strukturer enda ikke bestemt i detalj. Noen studier har indikert at disse komponentene kan stamme fra syrefraksjonene i oljen eller være relatert til asfalteninnholdet i oljene. Råoljer er blant verdens mest komplekse organiske blandinger og kan inneholde opptil 100 000 forskjellige bestanddeler, som gjør dem vanskelig å karakterisere ved bruk av tradisjonelle massespektrometre. Den høye masseoppløsningen Fourier-transform ion syklotron resonans massespektrometri (FT-ICR MS) gir en høyere oppløsning enn tradisjonelle teknikker, som gjør FT-ICR MS i stand til å karakterisere råoljer i større grad og muligens identifisere hydrataktive komponenter. FT-ICR MS spektre inneholder vanligvis titusenvis av topper, og det er nødvendig å bruke databehandlingsmetoder i stand til å håndtere store datasett, med muligheter til å finne underliggende forhold for å analysere spektrene. Maskinlæring og multivariat statistikk har mange metoder som er passende for store datasett. En litteratur studie identifiserte flere metoder og den nåværende statusen for bruken av maskinlæring for analyse av gasshydrater og FT-ICR MS data. Litteraturstudien viste at selv om mange studier har brukt maskinlæring til å predikere termodynamiske egenskaper for gasshydrater, har lite arbeid blitt gjort med å analysere gasshydrat relaterte prøver målt med FT-ICR MS. For å bistå identifikasjonen ble en suksessiv akkumuleringsprosedyre for å øke konsentrasjonene av hydrataktive komponenter utviklet av SINTEF. Sammenligninger av massespektrene fra spikede og uspikede prøver viste at noen topper økte sammen med spikingnivåene. Flere klassifikasjonsmetoder ble brukt i kombinasjon med ariabelseleksjon for å identifisere topper relatert til hydratformasjon. Molekylformler ble bestemt og toppene ble antatt å være relatert til asfaltener, naftener og polyetylenglykol. For å bistå karakteriseringen av oljene ble infrarød spektroskopi inkludert med FT-ICR MS i en multiblokk analyse for å predikere tettheten til råoljene. To forskjellige strategier for datafusjonering ble testet og sekvensiell fusjonering av blokkene oppnådde den høyeste prediksjonsnøyaktigheten både før og etter reduksjon av datasettene med bruk av variabelseleksjon. Ettersom råoljer har så kompleks sammensetning, er prøvene ofte veldig forskjellige og mange metoder er ikke egnet for å håndtere store variasjoner eller ikke-lineariteter mellom prøvene. Hierarchical cluster-based partial least squares regression (HCPLSR) grupperer dataene og lager lokale modeller for hver gruppe. HC-PLSR kan dermed håndtere ikke-lineariteter mellom gruppene, men siden PLSR er en lokal modell må dataene fortsatt være lokalt lineære. HC-PLSR ble derfor utvidet til convolutional neural networks (HC-CNN) og recurrent neural networks (HC-RNN) og support vector regression (HC-SVR). Disse dyp læring metodene utkonkurrerte HC-PLSR for et datasett som predikerte gjennomsnittlig molekylvekt fra hydrolyserte råmaterialer. Analysen av FT-ICR MS spektre viste at spektrene inneholder veldig mye informasjon. Disse store mengdene med data kan forstyrre prediksjonsmodeller, men bruken av variabelseleksjon motvirket denne effekten. Flere metoder fra maskinlæring og multivariat statistikk har blitt vist å være nyttige for prediksjon av flere parametere from FT-ICR MS data ved bruk av både klassifisering og regresjon

    Machine learning algorithms for efficient process optimisation of variable geometries at the example of fabric forming

    Get PDF
    Für einen optimalen Betrieb erfordern moderne Produktionssysteme eine sorgfältige Einstellung der eingesetzten Fertigungsprozesse. Physikbasierte Simulationen können die Prozessoptimierung wirksam unterstützen, jedoch sind deren Rechenzeiten oft eine erhebliche Hürde. Eine Möglichkeit, Rechenzeit einzusparen sind surrogate-gestützte Optimierungsverfahren (SBO1). Surrogates sind recheneffiziente, datengetriebene Ersatzmodelle, die den Optimierer im Suchraum leiten. Sie verbessern in der Regel die Konvergenz, erweisen sich aber bei veränderlichen Optimierungsaufgaben, etwa häufigen Bauteilanpassungen nach Kundenwunsch, als unhandlich. Um auch solche variablen Optimierungsaufgaben effizient zu lösen, untersucht die vorliegende Arbeit, wie jüngste Fortschritte im Maschinenlernen (ML) – im Speziellen bei neuronalen Netzen – bestehende SBO-Techniken ergänzen können. Dabei werden drei Hauptaspekte betrachtet: erstens, ihr Potential als klassisches Surrogate für SBO, zweitens, ihre Eignung zur effiziente Bewertung der Herstellbarkeit neuer Bauteilentwürfe und drittens, ihre Möglichkeiten zur effizienten Prozessoptimierung für variable Bauteilgeometrien. Diese Fragestellungen sind grundsätzlich technologieübergreifend anwendbar und werden in dieser Arbeit am Beispiel der Textilumformung untersucht. Der erste Teil dieser Arbeit (Kapitel 3) diskutiert die Eignung tiefer neuronaler Netze als Surrogates für SBO. Hierzu werden verschiedene Netzarchitekturen untersucht und mehrere Möglichkeiten verglichen, sie in ein SBO-Framework einzubinden. Die Ergebnisse weisen ihre Eignung für SBO nach: Für eine feste Beispielgeometrie minimieren alle Varianten erfolgreich und schneller als ein Referenzalgorithmus (genetischer Algorithmus) die Zielfunktion. Um die Herstellbarkeit variabler Bauteilgeometrien zu bewerten, untersucht Kapitel 4 anschließend, wie Geometrieinformationen in ein Prozess-Surrogate eingebracht werden können. Hierzu werden zwei ML-Ansätze verglichen, ein merkmals- und ein rasterbasierter Ansatz. Der merkmalsbasierte Ansatz scannt ein Bauteil nach einzelnen, prozessrelevanten Geometriemerkmalen, der rasterbasierte Ansatz hingegen interpretiert die Geometrie als Ganzes. Beide Ansätze können das Prozessverhalten grundsätzlich erlernen, allerdings erweist sich der rasterbasierte Ansatz als einfacher übertragbar auf neue Geometrievarianten. Die Ergebnisse zeigen zudem, dass hauptsächlich die Vielfalt und weniger die Menge der Trainingsdaten diese Übertragbarkeit bestimmt. Abschließend verbindet Kapitel 5 die Surrogate-Techniken für flexible Geometrien mit variablen Prozessparametern, um eine effiziente Prozessoptimierung für variable Bauteile zu erreichen. Hierzu interagiert ein ML-Algorithmus in einer Simulationsumgebung mit generischen Geometriebeispielen und lernt, welche Geometrie, welche Umformparameter erfordert. Nach dem Training ist der Algorithmus in der Lage, auch für nicht-generische Bauteilgeometrien brauchbare Empfehlungen auszugeben. Weiter zeigt sich, dass die Empfehlungen mit ähnlicher Geschwindigkeit wie die klassische SBO zum tatsächlichen Prozessoptimum konvergieren, jedoch kein bauteilspezifisches A-priori-Sampling nötig ist. Einmal trainiert, ist der entwickelte Ansatz damit effizienter. Insgesamt zeigt diese Arbeit, wie ML-Techniken gegenwärtige SBOMethoden erweitern und so die Prozess- und Produktoptimierung zu frühen Entwicklungszeitpunkten effizient unterstützen können. Die Ergebnisse der Untersuchungen münden in Folgefragen zur Weiterentwicklung der Methoden, etwa die Integration physikalischer Bilanzgleichungen, um die Modellprognosen physikalisch konsistenter zu machen

    Variations on bayesian optimization applied to numerical flow simulations

    Get PDF
    Bayesian Optimization (BO) has recently regained interest in optimization problems involving expensive black-box objective functions. Several variants have been proposed in the literature, such as including gradient and/or multi-fidelity information, and it has been extended to multi-objective optimization problems. Despite its recent applications to numerical flow simulations, the efficiency of this method and its variants remains to be characterized in typical applications involving canonical flows. In this work, the efficiency of classical BO and alternative derivative-free methods is compared on a simplified flow case, i.e. drag reduction in the two-dimensional flow around a cylinder. The application of BO to complex flows is then showcased by considering a three-dimensional case at Reynolds number Re = 3900. Next, the performance of BO with gradient and/or multi-fidelity information is investigated for global modelling and optimization on typical benchmark objective functions and on the cylinder case at Re = 200. Finally, an algorithm combining dimension reduction and Multi-objective Bayesian Optimization (MOBO) is proposed. It is found that BO was more efficient than other derivative-free alternatives and showed promising results on the three-dimensional cylinder at Re = 3900 by reducing drag by 23 %. The performance of the algorithm was further improved when multi-fidelity and/or gradient information was included, both for modelling and optimization. Including gradient information on the low-fidelity model was useful for global modelling and to decrease rapidly the objective function in a BO framework. On the contrary, adding derivative information on the high-fidelity model generally gave the most accurate approximation of the minimum but was inefficient for global modelling when the computational cost of the gradient was high. Finally, the developed algorithm combining dimension reduction and MOBO enabled us to obtain more precise and diverse minima.136 página

    Heavy Baryons in Compact Stars

    Full text link
    We review the physics of hyperons and Δ\Delta-resonances in dense matter in compact stars. The covariant density functional approach to the equation of state and composition of dense nuclear matter in the mean-field Hartree and Hartree-Fock approximation is presented, with regimes covering cold β\beta-equilibrated matter, hot and dense matter with and without neutrinos relevant for the description of supernovas and binary neutron star mergers, as well as dilute expanding nuclear matter in collision experiments. We discuss the static properties of compact stars with hyperons and Δ\Delta-resonances in light of constraints placed in recent years by the multimessenger astrophysics of compact stars on the compact stars' masses, radii, and tidal deformabilities. The effects of kaon condensation and strong magnetic fields on the composition of hypernuclear stars are also discussed. The properties of rapidly rotating compact hypernuclear stars are discussed and confronted with the observations of 2.5-2.8 solar mass compact objects in gravitational wave events. We further discuss the cooling of hypernuclear stars, the neutrino emission reactions, hyperonic pairing, and the mass hierarchy in the cooling curves that arises due to the onset of hyperons. The effects of hyperons and Δ\Delta-resonances on the equation of state of hot nuclear matter in the dense regime, relevant for the transient astrophysical event and in the dilute regime relevant to the collider physics is discussed. The review closes with a discussion of universal relations among the integral parameters of hot and cold hypernuclear stars and their implications for the analysis of binary neutron star merger events.Comment: v2: 79 pages, 26 figures, final version (minor changes and additions, typos corrected). v1: 76 pages, 26 figures. arXiv admin note: text overlap with arXiv:2105.1405

    An integrated species distribution modelling framework for heterogeneous biodiversity data

    Get PDF
    Most knowledge about species and habitats is in-homogeneously distributed, with biases existing in space, time and taxonomic and functional knowledge. Yet, controversially the total amount of biodiversity data has never been greater. A key challenge is thus how to make effective use of the various sources of biodiversity data in an integrated manner. Particularly for widely used modelling approaches, such as species distribution models (SDMs), the need for integration is urgent, if spatial and temporal predictions are to be accurate enough in addressing global challenges. Here, I present a modelling framework that brings together several ideas and methodological advances for creating integrated species distribution models (iSDM). The ibis.iSDM R-package is a set of modular convenience functions that allows the integration of different data sources, such as presence-only, community survey, expert ranges or species habitat preferences, in a single model or ensemble of models. Further it supports convenient parameter transformations and tuning, data preparation helpers and allows the creation of spatial-temporal projections and scenarios. Ecological constraints such as projection limits, dispersal, connectivity or adaptability can be added in a modular fashion thus helping to prevent unrealistic estimates of species distribution changes. The ibis.iSDM R-package makes use of a series of methodological advances and is aimed to be a vehicle for creating more realistic and constrained spatial predictions. Besides providing convenience functions for a range of different statistical models as well as an increasing number of wrappers for mechanistic modules, ibis.iSDM also introduces several innovative concepts such as sequential or weighted integration, or thresholding by prediction uncertainty. The overall framework will be continued to be improved and further functionalities be added

    Tumour growth: An approach to calibrate parameters of a multiphase porous media model based on in vitro observations of Neuroblastoma spheroid growth in a hydrogel microenvironment

    Get PDF
    To unravel processes that lead to the growth of solid tumours, it is necessary to link knowledge of cancer biology with the physical properties of the tumour and its interaction with the surrounding microenvironment. Our understanding of the underlying mechanisms is however still imprecise. We therefore developed computational physics-based models, which incorporate the interaction of the tumour with its surroundings based on the theory of porous media. However, the experimental validation of such models represents a challenge to its clinical use as a prognostic tool. This study combines a physics-based model with in vitro experiments based on microfluidic devices used to mimic a three-dimensional tumour microenvironment. By conducting a global sensitivity analysis, we identify the most influential input parameters and infer their posterior distribution based on Bayesian calibration. The resulting probability density is in agreement with the scattering of the experimental data and thus validates the proposed workflow. This study demonstrates the huge challenges associated with determining precise parameters with usually only limited data for such complex processes and models, but also demonstrates in general how to indirectly characterise the mechanical properties of neuroblastoma spheroids that cannot feasibly be measured experimentally

    Uncertainty analysis of structural output with closed-form expression based on surrogate model

    Get PDF
    Uncertainty analysis (UA) is the process that quantitatively identifies and characterizes the output uncertainty and has a crucial implication in engineering applications. The research of efficient estimation of structural output moments in probability space plays an important part in the UA and has great engineering significance. Given this point, a new UA method based on the Kriging surrogate model related to closed-form expressions for the perception of the estimation of mean and variance is proposed in this paper. The new proposed method is proven effective because of its direct reflection on the prediction uncertainty of the output moments of metamodel to quantify the accuracy level. The estimation can be completed by directly using the redefined closed-form expressions of the model's output mean and variance to avoid excess post-processing computational costs and errors. Furthermore, a novel framework of adaptive Kriging estimating mean (AKEM) is demonstrated for more efficiently reducing uncertainty in the estimation of output moment. In the adaptive strategy of AKEM, a new learning function based on the closed-form expression is proposed. Based on the closed-form expression which modifies the computational error caused by the metamodeling uncertainty, the proposed learning function enables the updating of metamodel to reduce prediction uncertainty efficiently and realize the decrease in computational costs. Several applications are introduced to prove the effectiveness and efficiency of the AKEM compared with a universal adaptive Kriging method. Through the good performance of AKEM, its potential in engineering applications can be spotted

    Koopman with inputs and control for constitutive law identification

    Get PDF
    Constitutive laws characterise the stress-strain relationship in a material. Determining a consti- tutive law experimentally typically involves subjecting the material to a prescribed deformation and measuring the force required to achieve it. There are numerous constitutive laws which have been developed to model the stress response of viscoelastic fluids, and the decision on which constitutive law should be fitted to data is largely based on the rheologist’s knowledge about the fluid in relation to the catalogue of standard models appearing in the literature. In this thesis, we present an alternative approach for determining a viscoelastic fluid’s constitutive law based on methods related to Koopman operator theory and Dynamic Mode Decomposition in the context of control. Our approach systematically extracts the material parameters that arise in stress-evolution equations of viscoelastic fluids directly from simulation or experimen- tal data. We will present results from various applications of the framework that highlight its accuracy and robustness in identifying material parameters and reconstructing the under- lying constitutive law. We will discuss how data should be supplied to the method, and also demonstrate how data from recently developed experimental protocols, as well as combined data from multiple experiments, can be used to improve resolution. Finally, we will show that our approach provides a natural way to utilise data from the nonlinear regime and extends to higher-dimensional data sets where spatial data within a sample is available.Open Acces
    corecore