Efficient Localization of Discontinuities in Complex Computational Simulations

Surrogate models for computational simulations are input-output approximations that allow computationally intensive analyses, such as uncertainty propagation and inference, to be performed efficiently. When a simulation output does not depend smoothly on its inputs, the error and convergence rate of many approximation methods deteriorate substantially. This paper details a method for efficiently localizing discontinuities in the input parameter domain, so that the model output can be approximated as a piecewise smooth function. The approach comprises an initialization phase, which uses polynomial annihilation to assign function values to different regions and thus seed an automated labeling procedure, followed by a refinement phase that adaptively updates a kernel support vector machine representation of the separating surface via active learning. The overall approach avoids structured grids and exploits any available simplicity in the geometry of the separating surface, thus reducing the number of model evaluations required to localize the discontinuity. The method is illustrated on examples of up to eleven dimensions, including algebraic models and ODE/PDE systems, and demonstrates improved scaling and efficiency over other discontinuity localization approaches

Machine Learning and System Identification for Estimation in Physical Systems

In this thesis, we draw inspiration from both classical system identification and modern machine learning in order to solve estimation problems for real-world, physical systems. The main approach to estimation and learning adopted is optimization based. Concepts such as regularization will be utilized for encoding of prior knowledge and basis-function expansions will be used to add nonlinear modeling power while keeping data requirements practical.The thesis covers a wide range of applications, many inspired by applications within robotics, but also extending outside this already wide field.Usage of the proposed methods and algorithms are in many cases illustrated in the real-world applications that motivated the research.Topics covered include dynamics modeling and estimation, model-based reinforcement learning, spectral estimation, friction modeling and state estimation and calibration in robotic machining.In the work on modeling and identification of dynamics, we develop regularization strategies that allow us to incorporate prior domain knowledge into flexible, overparameterized models. We make use of classical control theory to gain insight into training and regularization while using tools from modern deep learning. A particular focus of the work is to allow use of modern methods in scenarios where gathering data is associated with a high cost.In the robotics-inspired parts of the thesis, we develop methods that are practically motivated and make sure that they are implementable also outside the research setting. We demonstrate this by performing experiments in realistic settings and providing open-source implementations of all proposed methods and algorithms

Forecasting economic variables with nonlinear models

This article is concerned with forecasting from nonlinear conditional mean models. First, a number of often applied nonlinear conditional mean models are introduced and their main properties discussed. The next section is devoted to techniques of building nonlinear models. Ways of computing multi-step ahead forecasts from nonlinear models are surveyed. Tests of forecast accuracy in the case where the models generating the forecasts are nested are discussed. There is a numerical example, showing that even when a stationary nonlinear process generates the observations, future obervations may in some situations be better forecast by a linear model with a unit root. Finally, some empirical studies that compare forecasts from linear and nonlinear models are discussed.Forecast accuracy; forecast comparison; hidden Markov model; neural network; nonlinear modelling; recursive forecast; smooth transition regression; switching regression;

Estimating Probability Distributions from Complex Models with Bifurcations: The Case of Ocean Circulation Collapse

Abstract in HTML and technical report in PDF available on the Massachusetts Institute of Technology Joint Program on the Science and Policy of Global Change website (http://mit.edu/globalchange/www/).Studying the uncertainty in computationally expensive models has required the development of specialized methods, including alternative sampling techniques and response surface approaches. However, existing techniques for response surface development break down when the model being studied exhibits discontinuities or bifurcations. One uncertain variable that exhibits this behavior is the thermohaline circulation (THC) as modeled in three-dimensional general circulation models. This is a critical uncertainty for climate change policy studies. We investigate the development of a response surface for studying uncertainty in THC using the Deterministic Equivalent Modeling Method, a stochastic technique using expansions in orthogonal polynomials. We show that this approach is unable to reasonably approximate the model response. We demonstrate an alternative representation that accurately simulates the model’s response, using a basis function with properties similar to the model’s response over the uncertain parameter space. This indicates useful directions for future methodological improvements.This research was supported in part by the Methods and Models for Integrated Assessments Program of the National Science Foundation, Grant ATM-9909139, by the Office of Science (BER), U.S. Department of Energy, Grant Nos. DE-FG02-02ER63468 and DE-FG02-93ER61677, and by the MIT Joint Program on the Science and Policy of Global Change (JPSPGC)