Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information

Recent works in learning-integrated optimization have shown promise in settings where the optimization problem is only partially observed or where general-purpose optimizers perform poorly without expert tuning. By learning an optimizer $\mathbf{g}$ to tackle these challenging problems with $f$ as the objective, the optimization process can be substantially accelerated by leveraging past experience. The optimizer can be trained with supervision from known optimal solutions or implicitly by optimizing the compound function $f\circ \mathbf{g}$. The implicit approach may not require optimal solutions as labels and is capable of handling problem uncertainty; however, it is slow to train and deploy due to frequent calls to optimizer $\mathbf{g}$ during both training and testing. The training is further challenged by sparse gradients of $\mathbf{g}$, especially for combinatorial solvers. To address these challenges, we propose using a smooth and learnable Landscape Surrogate $M$ as a replacement for $f\circ \mathbf{g}$. This surrogate, learnable by neural networks, can be computed faster than the solver $\mathbf{g}$, provides dense and smooth gradients during training, can generalize to unseen optimization problems, and is efficiently learned via alternating optimization. We test our approach on both synthetic problems, including shortest path and multidimensional knapsack, and real-world problems such as portfolio optimization, achieving comparable or superior objective values compared to state-of-the-art baselines while reducing the number of calls to $\mathbf{g}$. Notably, our approach outperforms existing methods for computationally expensive high-dimensional problems

A Computational Framework for Efficient Reliability Analysis of Complex Networks

With the growing scale and complexity of modern infrastructure networks comes the challenge of developing efficient and dependable methods for analysing their reliability. Special attention must be given to potential network interdependencies as disregarding these can lead to catastrophic failures. Furthermore, it is of paramount importance to properly treat all uncertainties. The survival signature is a recent development built to effectively analyse complex networks that far exceeds standard techniques in several important areas. Its most distinguishing feature is the complete separation of system structure from probabilistic information. Because of this, it is possible to take into account a variety of component failure phenomena such as dependencies, common causes of failure, and imprecise probabilities without reevaluating the network structure. This cumulative dissertation presents several key improvements to the survival signature ecosystem focused on the structural evaluation of the system as well as the modelling of component failures. A new method is presented in which (inter)-dependencies between components and networks are modelled using vine copulas. Furthermore, aleatory and epistemic uncertainties are included by applying probability boxes and imprecise copulas. By leveraging the large number of available copula families it is possible to account for varying dependent effects. The graph-based design of vine copulas synergizes well with the typical descriptions of network topologies. The proposed method is tested on a challenging scenario using the IEEE reliability test system, demonstrating its usefulness and emphasizing the ability to represent complicated scenarios with a range of dependent failure modes. The numerical effort required to analytically compute the survival signature is prohibitive for large complex systems. This work presents two methods for the approximation of the survival signature. In the first approach system configurations of low interest are excluded using percolation theory, while the remaining parts of the signature are estimated by Monte Carlo simulation. The method is able to accurately approximate the survival signature with very small errors while drastically reducing computational demand. Several simple test systems, as well as two real-world situations, are used to show the accuracy and performance. However, with increasing network size and complexity this technique also reaches its limits. A second method is presented where the numerical demand is further reduced. Here, instead of approximating the whole survival signature only a few strategically selected values are computed using Monte Carlo simulation and used to build a surrogate model based on normalized radial basis functions. The uncertainty resulting from the approximation of the data points is then propagated through an interval predictor model which estimates bounds for the remaining survival signature values. This imprecise model provides bounds on the survival signature and therefore the network reliability. Because a few data points are sufficient to build the interval predictor model it allows for even larger systems to be analysed. With the rising complexity of not just the system but also the individual components themselves comes the need for the components to be modelled as subsystems in a system-of-systems approach. A study is presented, where a previously developed framework for resilience decision-making is adapted to multidimensional scenarios in which the subsystems are represented as survival signatures. The survival signature of the subsystems can be computed ahead of the resilience analysis due to the inherent separation of structural information. This enables efficient analysis in which the failure rates of subsystems for various resilience-enhancing endowments are calculated directly from the survival function without reevaluating the system structure. In addition to the advancements in the field of survival signature, this work also presents a new framework for uncertainty quantification developed as a package in the Julia programming language called UncertaintyQuantification.jl. Julia is a modern high-level dynamic programming language that is ideal for applications such as data analysis and scientific computing. UncertaintyQuantification.jl was built from the ground up to be generalised and versatile while remaining simple to use. The framework is in constant development and its goal is to become a toolbox encompassing state-of-the-art algorithms from all fields of uncertainty quantification and to serve as a valuable tool for both research and industry. UncertaintyQuantification.jl currently includes simulation-based reliability analysis utilising a wide range of sampling schemes, local and global sensitivity analysis, and surrogate modelling methodologies

Integration of AI and mechanistic modeling in generative adversarial networks for stochastic inverse problems

The problem of finding distributions of input parameters for deterministic mechanistic models to match distributions of model outputs to stochastic observations, i.e., the "Stochastic Inverse Problem" (SIP), encompasses a range of common tasks across a variety of scientific disciplines. Here, we demonstrate that SIP could be reformulated as a constrained optimization problem and adapted for applications in intervention studies to simultaneously infer model input parameters for two sets of observations, under control conditions and under an intervention. In the constrained optimization problem, the solution of SIP is enforced to accommodate the prior knowledge on the model input parameters and to produce outputs consistent with given observations by minimizing the divergence between the inferred distribution of input parameters and the prior. Unlike in standard SIP, the prior incorporates not only knowledge about model input parameters for objects in each set, but also information on the joint distribution or the deterministic map between the model input parameters in two sets of observations. To solve standard and intervention SIP, we employed conditional generative adversarial networks (GANs) and designed novel GANs that incorporate multiple generators and discriminators and have structures that reflect the underlying constrained optimization problems. This reformulation allows us to build computationally scalable solutions to tackle complex model input parameter inference scenarios, which appear routinely in physics, biophysics, economics and other areas, and which currently could not be handled with existing methods

Active learning-driven uncertainty reduction for in-flight particle characteristics of atmospheric plasma spraying of silicon

In this study, the first-of-its-kind use of active learning (AL) framework in thermal spray is adapted to improve the prediction accuracy of the in-flight particle characteristics and uses Gaussian Process (GP) ML model as a surrogate that generalises a global solution without necessarily involving physical mechanisms. The AL framework via the Bayesian Optimisation was utilised to: (a) reduce the maximum uncertainty in the given database and (b) reduce local uncertainty around a contrived test point. The initial dataset consists of 26 atmospheric plasma spray (APS) parameters of silicon, aimed at ceramic matrix composites (CMCs) for the next generation of aerospace applications. The maximum uncertainty in the initial dataset was reduced by AL-driven identification of search spaces and conducting six guided spray trails in the identified search spaces. On average, a 52.9% improvement (error reduction) of RMSE and an R2 increase of 8.5% were reported on the predicted in-flight particle velocities and temperatures after the AL-driven optimisation. Furthermore, the Bayesian Optimisation around a contrived test point to predict the best possible characteristics resulted in a three-fold increase in prediction accuracy as compared to the non-optimised prediction. These AL-guided experimental validations not only increase the informativeness of the limited dataset but is adaptable for other thermal spraying methods without necessarily involving physical mechanisms and underlying mechanisms. The use of AL-driven optimisation may drive the thermal spraying towards resource-efficiency and may serve as the first step towards fully digital thermal spraying environments

Modeling and Communicating Flexibility in Smart Grids Using Artificial Neural Networks as Surrogate Models

Increasing shares of renewable energies and the transition towards electric vehicles pose major challenges to the energy system. In order to tackle these in an economically sensible way, the flexibility of distributed energy resources (DERs), such as battery energy storage systems, combined heat and power plants, and heat pumps, needs to be exploited. Modeling and communicating this flexibility is a fundamental step when trying to achieve control over DERs. The literature proposes and makes use of many different approaches, not only for the exploitation itself, but also in terms of models. In the first step, this thesis presents an extensive literature review and a general framework for classifying exploitation approaches and the communicated models. Often, the employed models only apply to specific types of DERs, or the models are so abstract that they neglect constraints and only roughly outline the true flexibility. Surrogate models, which are learned from data, can pose as generic DER models and may potentially be trained in a fully automated process. In this thesis, the idea of encoding the flexibility of DERs into ANNs is systematically investigated. Based on the presented framework, a set of ANN-based surrogate modeling approaches is derived and outlined, of which some are only applicable for specific use cases. In order to establish a baseline for the approximation quality, one of the most versatile identified approaches is evaluated in order to assess how well a set of reference models is approximated. If this versatile model is able to capture the flexibility well, a more specific model can be expected to do so even better. The results show that simple DERs are very closely approximated, and for more complex DERs and combinations of multiple DERs, a high approximation quality can be achieved by introducing buffers. Additionally, the investigated approach has been tested in scheduling tasks for multiple different DERs, showing that it is indeed possible to use ANN-based surrogates for the flexibility of DERs to derive load schedules. Finally, the computational complexity of utilizing the different approaches for controlling DERs is compared

Harnessing Heterogeneity To Improve Patient Outcomes.

We investigate methods of improving medical outcomes through exploiting heterogeneity, with focus on actual implementation. Advances in data-mining and big-data methods have allowed new and exciting opportunities to alter the precise nature of statistical medical research. Whereas traditional science experimentation has attempted to eliminate causes of variability beyond a small set of variables of interest to be investigated, machine-learning techniques to extract weak and complex signals from noisy data now allow handling of heterogeneous experiments and subjects. We propose that viewed through the lens of these modern machine-learning methods, heterogeneous and highly-variable data should be regarded as a boon not a nuisance. In particular such data allows for the investigation and construction of individualized treatment rules for patients, that is for the advance of precision medicine. Two facets of this view are especially explored. Firstly the practical design and implementation of appropriate data collection experiments allowing for a machine-learning approach, whilst simultaneously permitting a traditional experimental view in order to satisfy investigators from both paradigms. We reference a particular example, the design for a clinical trial investigating the optimal treatment of burns patients (the LIBERTI trial). This example highlights some particular challenges, statistical, philosophical and logistical, and hopefully some corresponding solutions, that arise when bridging traditional and modern paradigms. Whilst we present our design as an initial solution, from the attempted implementation of this trial we discover, and then explore, particular aspects that are apt for further improvement. Secondly we investigate methods to combine and make effective traditional clustering techniques in higher dimensional data with weak signals, where existing techniques may fail. Motivated by an example of COPD sufferers’ data (the SPIROMICS study), we attempt to develop ways combining more traditional methods with a machine-learning approach, and more fuzzy data-mining methods, with ones permitting better inference. We illustrate our methods on Fisher's Iris data, and the Wisconsin Breast Cancer data set. We explore extensions of the traditional Gaussian mixture model to more general log-concave distributions and highlight what should be interesting theory for such approximations.Doctor of Philosoph

Quantitative analysis of woodpecker habitat using high-resolution airborne LiDAR estimates of forest structure and composition

Light detection and ranging (LiDAR) technology has the potential to radically alter theway researchers and managers collect data onwildlife–habitat relationships. To date, the technology has fostered several novel approaches to characterizing avian habitat, but has been limited by the lack of detailed LiDAR-habitat attributes relevant to species across a continuum of spatial grain sizes and habitat requirements. We demonstrate a novel three-step approach for using LiDAR data to evaluate habitat based on multiple habitat attributes and accounting for their influence at multiple grain sizes using federally endangered red-cockaded woodpecker (RCW; Picoides borealis) foraging habitat data fromthe Savannah River Site (SRS) in South Carolina, USA. First,we used high density LiDAR data (10 returns/m2) to predict detailed forest attributes at 20-mresolution across the entire SRS using a complementary application of nonlinear seemingly unrelated regression andmultiple linear regressionmodels. Next,we expanded on previous applications of LiDAR by constructing 95% joint prediction confidence intervals to quantify prediction error at various spatial aggregations and habitat thresholds to determine a biologically and statistically meaningful grain size. Finally,we used aggregations of 20-m cells and associated confidence interval boundaries to demonstrate a newapproach to produce maps of RCWforaging habitat conditions based on the guidelines described in the species\u27 recovery plan. Predictive power (R2) of regression models developed to populate raster layers ranged from 0.34 to 0.81, and prediction error decreased as aggregate size increased, but minimal reductions in prediction error were observed beyond 0.64-ha (4 × 4 20-m cells) aggregates. Mapping habitat quality while accounting for prediction error provided a robust method to determine the potential range of habitat conditions and specific attributes that were limiting in terms of the amount of suitable habitat. The sequential steps of our analytical approach provide a useful framework to extract detailed and reliable habitat attributes for a forest-dwelling habitat specialist, broadening the potential to apply LiDAR in conservation and management of wildlife populations. A zipped folder of Google maps is attached below as a related file