Formal Verification of Input-Output Mappings of Tree Ensembles

Recent advances in machine learning and artificial intelligence are now being considered in safety-critical autonomous systems where software defects may cause severe harm to humans and the environment. Design organizations in these domains are currently unable to provide convincing arguments that their systems are safe to operate when machine learning algorithms are used to implement their software. In this paper, we present an efficient method to extract equivalence classes from decision trees and tree ensembles, and to formally verify that their input-output mappings comply with requirements. The idea is that, given that safety requirements can be traced to desirable properties on system input-output patterns, we can use positive verification outcomes in safety arguments. This paper presents the implementation of the method in the tool VoTE (Verifier of Tree Ensembles), and evaluates its scalability on two case studies presented in current literature. We demonstrate that our method is practical for tree ensembles trained on low-dimensional data with up to 25 decision trees and tree depths of up to 20. Our work also studies the limitations of the method with high-dimensional data and preliminarily investigates the trade-off between large number of trees and time taken for verification

Intelligent Data Mining using Kernel Functions and Information Criteria

Radial Basis Function (RBF) Neural Networks and Support Vector Machines (SVM) are two powerful kernel related intelligent data mining techniques. The current major problems with these methods are over-fitting and the existence of too many free parameters. The way to select the parameters can directly affect the generalization performance(test error) of theses models. Current practice in how to choose the model parameters is an art, rather than a science in this research area. Often, some parameters are predetermined, or randomly chosen. Other parameters are selected through repeated experiments that are time consuming, costly, and computationally very intensive. In this dissertation, we provide a two-stage analytical hybrid-training algorithm by building a bridge among regression tree, EM algorithm, and Radial Basis Function Neural Networks together. Information Complexity (ICOMP) criterion of Bozdogan along with other information based criteria are introduced and applied to control the model complexity, and to decide the optimal number of kernel functions. In the first stage of the hybrid, regression tree and EM algorithm are used to determine the kernel function parameters. In the second stage of the hybrid, the weights (coefficients) are calculated and information criteria are scored. Kernel Principal Component Analysis (KPCA) using EM algorithm for feature selection and data preprocessing is also introduced and studied. Adaptive Support Vector Machines (ASVM) and some efficient algorithms are given to deal with massive data sets in support vector classifications. Versatility and efficiency of the new proposed approaches are studied on real data sets and via Monte Carlo sim- ulation experiments

Reliability-based design optimization using kriging surrogates and subset simulation

The aim of the present paper is to develop a strategy for solving reliability-based design optimization (RBDO) problems that remains applicable when the performance models are expensive to evaluate. Starting with the premise that simulation-based approaches are not affordable for such problems, and that the most-probable-failure-point-based approaches do not permit to quantify the error on the estimation of the failure probability, an approach based on both metamodels and advanced simulation techniques is explored. The kriging metamodeling technique is chosen in order to surrogate the performance functions because it allows one to genuinely quantify the surrogate error. The surrogate error onto the limit-state surfaces is propagated to the failure probabilities estimates in order to provide an empirical error measure. This error is then sequentially reduced by means of a population-based adaptive refinement technique until the kriging surrogates are accurate enough for reliability analysis. This original refinement strategy makes it possible to add several observations in the design of experiments at the same time. Reliability and reliability sensitivity analyses are performed by means of the subset simulation technique for the sake of numerical efficiency. The adaptive surrogate-based strategy for reliability estimation is finally involved into a classical gradient-based optimization algorithm in order to solve the RBDO problem. The kriging surrogates are built in a so-called augmented reliability space thus making them reusable from one nested RBDO iteration to the other. The strategy is compared to other approaches available in the literature on three academic examples in the field of structural mechanics.Comment: 20 pages, 6 figures, 5 tables. Preprint submitted to Springer-Verla

Forward Invariance in Neural Network Controlled Systems

We present a framework based on interval analysis and monotone systems theory to certify and search for forward invariant sets in nonlinear systems with neural network controllers. The framework (i) constructs localized first-order inclusion functions for the closed-loop system using Jacobian bounds and existing neural network verification tools; (ii) builds a dynamical embedding system where its evaluation along a single trajectory directly corresponds with a nested family of hyper-rectangles provably converging to an attractive set of the original system; (iii) utilizes linear transformations to build families of nested paralleletopes with the same properties. The framework is automated in Python using our interval analysis toolbox $\texttt{npinterval}$, in conjunction with the symbolic arithmetic toolbox $\texttt{sympy}$, demonstrated on an $8$-dimensional leader-follower system

Fast modeling of turbulent transport in fusion plasmas using neural networks

We present an ultrafast neural network (NN) model, QLKNN, which predicts core tokamak transport heat and particle fluxes. QLKNN is a surrogate model based on a database of 300 million flux calculations of the quasilinear gyrokinetic transport model QuaLiKiz. The database covers a wide range of realistic tokamak core parameters. Physical features such as the existence of a critical gradient for the onset of turbulent transport were integrated into the neural network training methodology. We have coupled QLKNN to the tokamak modelling framework JINTRAC and rapid control-oriented tokamak transport solver RAPTOR. The coupled frameworks are demonstrated and validated through application to three JET shots covering a representative spread of H-mode operating space, predicting turbulent transport of energy and particles in the plasma core. JINTRAC-QLKNN and RAPTOR-QLKNN are able to accurately reproduce JINTRAC-QuaLiKiz T i,e and n e profiles, but 3 to 5 orders of magnitude faster. Simulations which take hours are reduced down to only a few tens of seconds. The discrepancy in the final source-driven predicted profiles between QLKNN and QuaLiKiz is on the order 1%-15%. Also the dynamic behaviour was well captured by QLKNN, with differences of only 4%-10% compared to JINTRAC-QuaLiKiz observed at mid-radius, for a study of density buildup following the L-H transition. Deployment of neural network surrogate models in multi-physics integrated tokamak modelling is a promising route towards enabling accurate and fast tokamak scenario optimization, Uncertainty Quantification, and control applications.Comment: 18 pages, 11 figures, Physics of Plasmas, ICDDPS 2019 conference pape

Reliability-based design optimization of shells with uncertain geometry using adaptive Kriging metamodels

Optimal design under uncertainty has gained much attention in the past ten years due to the ever increasing need for manufacturers to build robust systems at the lowest cost. Reliability-based design optimization (RBDO) allows the analyst to minimize some cost function while ensuring some minimal performances cast as admissible failure probabilities for a set of performance functions. In order to address real-world engineering problems in which the performance is assessed through computational models (e.g., finite element models in structural mechanics) metamodeling techniques have been developed in the past decade. This paper introduces adaptive Kriging surrogate models to solve the RBDO problem. The latter is cast in an augmented space that "sums up" the range of the design space and the aleatory uncertainty in the design parameters and the environmental conditions. The surrogate model is used (i) for evaluating robust estimates of the failure probabilities (and for enhancing the computational experimental design by adaptive sampling) in order to achieve the requested accuracy and (ii) for applying a gradient-based optimization algorithm to get optimal values of the design parameters. The approach is applied to the optimal design of ring-stiffened cylindrical shells used in submarine engineering under uncertain geometric imperfections. For this application the performance of the structure is related to buckling which is addressed here by means of a finite element solution based on the asymptotic numerical method