Ensemble Kalman filter for neural network based one-shot inversion

We study the use of novel techniques arising in machine learning for inverse problems. Our approach replaces the complex forward model by a neural network, which is trained simultaneously in a one-shot sense when estimating the unknown parameters from data, i.e. the neural network is trained only for the unknown parameter. By establishing a link to the Bayesian approach to inverse problems, an algorithmic framework is developed which ensures the feasibility of the parameter estimate w.r. to the forward model. We propose an efficient, derivative-free optimization method based on variants of the ensemble Kalman inversion. Numerical experiments show that the ensemble Kalman filter for neural network based one-shot inversion is a promising direction combining optimization and machine learning techniques for inverse problems

Well Posedness and Convergence Analysis of the Ensemble Kalman Inversion

The ensemble Kalman inversion is widely used in practice to estimate unknown parameters from noisy measurement data. Its low computational costs, straightforward implementation, and non-intrusive nature makes the method appealing in various areas of application. We present a complete analysis of the ensemble Kalman inversion with perturbed observations for a fixed ensemble size when applied to linear inverse problems. The well-posedness and convergence results are based on the continuous time scaling limits of the method. The resulting coupled system of stochastic differential equations allows to derive estimates on the long-time behaviour and provides insights into the convergence properties of the ensemble Kalman inversion. We view the method as a derivative free optimization method for the least-squares misfit functional, which opens up the perspective to use the method in various areas of applications such as imaging, groundwater flow problems, biological problems as well as in the context of the training of neural networks

Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models

The ensemble Kalman inversion (EKI) is a particle based method which has been introduced as the application of the ensemble Kalman filter to inverse problems. In practice it has been widely used as derivative-free optimization method in order to estimate unknown parameters from noisy measurement data. For linear forward models the EKI can be viewed as gradient flow preconditioned by a certain sample covariance matrix. Through the preconditioning the resulting scheme remains in a finite dimensional subspace of the original high-dimensional (or even infinite dimensional) parameter space and can be viewed as optimizer restricted to this subspace. For general nonlinear forward models the resulting EKI flow can only be viewed as gradient flow in approximation. In this paper we discuss the effect of applying a sample covariance as preconditioning matrix and quantify the gradient flow structure of the EKI by controlling the approximation error through the spread in the particle system. The ensemble collapse on the one side leads to an accurate gradient approximation, but on the other side to degeneration in the preconditioning sample covariance matrix. In order to ensure convergence as optimization method we derive lower as well as upper bounds on the ensemble collapse. Furthermore, we introduce covariance inflation without breaking the subspace property intending to reduce the collapse rate of the ensemble such that the convergence rate improves. In a numerical experiment we apply EKI to a nonlinear elliptic boundary-value problem and illustrate the dependence of EKI as derivative-free optimizer on the choice of the initial ensemble

Particle based sampling and optimization methods for inverse problems

In this thesis, we present and analyse several ensemble based methods for sampling as well as optimization in inverse problems. Firstly we examine the ensemble Kalman inversion, which has been originally introduced as a sampling method for Bayesian inverse problems, but can also be viewed as derivative free optimization method. Furthermore, we present various transformed methods of the ensemble Kalman inversion, which allow to incorporate box-constraints as well as regularization for the underlying optimization problem. In addition, we also consider a more general class of particle based sampling methods, such as the ensemble Kalman sampler, which is based on an interacting Langevin dynamics, a particle system resulting from an Gaussian approximation, as well as a kernelized Fokker--Planck based particle system. In the last part of this work, we discuss machine learning applications in inverse problems. Here, we consider data-driven regularization, where the regularization parameter will be chosen by solving a bilevel optimization problem. Moreover, we consider an incorporation of neural networks into inverse problems, which will act as a model-informed surrogate for the complex forward model and will be trained with the unknown parameter in a one-shot fashion

On the Incorporation of Box-Constraints for Ensemble Kalman Inversion

The Bayesian approach to inverse problems is widely used in practice to infer unknown parameters from noisy observations. In this framework, the ensemble Kalman inversion has been successfully applied for the quantification of uncertainties in various areas of applications. In recent years, a complete analysis of the method has been developed for linear inverse problems adopting an optimization viewpoint. However, many applications require the incorporation of additional constraints on the parameters, e.g. arising due to physical constraints. We propose a new variant of the ensemble Kalman inversion to include box constraints on the unknown parameters motivated by the theory of projected preconditioned gradient flows. Based on the continuous time limit of the constrained ensemble Kalman inversion, we discuss a complete convergence analysis for linear forward problems. We adopt techniques from filtering which are crucial in order to improve the performance and establish a correct descent, such as variance inflation. These benefits are highlighted through a number of numerical examples on various inverse problems based on partial differential equations

Consistency analysis of bilevel data-driven learning in inverse problems

One fundamental problem when solving inverse problems is how to find regularization parameters. This article considers solving this problem using data-driven bilevel optimization, i.e. we consider the adaptive learning of the regularization parameter from data by means of optimization. This approach can be interpreted as solving an empirical risk minimization problem, and we analyze its performance in the large data sample size limit for general nonlinear problems. We demonstrate how to implement our framework on linear inverse problems, where we can further show the inverse accuracy does not depend on the ambient space dimension. To reduce the associated computational cost, online numerical schemes are derived using the stochastic gradient descent method. We prove convergence of these numerical schemes under suitable assumptions on the forward problem. Numerical experiments are presented illustrating the theoretical results and demonstrating the applicability and efficiency of the proposed approaches for various linear and nonlinear inverse problems, including Darcy flow, the eikonal equation, and an image denoising example

Adaptive multilevel subset simulation with selective refinement

In this work we propose an adaptive multilevel version of subset simulation to estimate the probability of rare events for complex physical systems. Given a sequence of nested failure domains of increasing size, the rare event probability is expressed as a product of conditional probabilities. The proposed new estimator uses different model resolutions and varying numbers of samples across the hierarchy of nested failure sets. In order to dramatically reduce the computational cost, we construct the intermediate failure sets such that only a small number of expensive high-resolution model evaluations are needed, whilst the majority of samples can be taken from inexpensive low-resolution simulations. A key idea in our new estimator is the use of a posteriori error estimators combined with a selective mesh refinement strategy to guarantee the critical subset property that may be violated when changing model resolution from one failure set to the next. The efficiency gains and the statistical properties of the estimator are investigated both theoretically via shaking transformations, as well as numerically. On a model problem from subsurface flow, the new multilevel estimator achieves gains of more than a factor 200 over standard subset simulation for a practically relevant relative error of 25%

The tumor suppressor CIC directly regulates MAPK pathway genes via histone deacetylation

Abstract Oligodendrogliomas are brain tumors accounting for approximately 10% of all central nervous system cancers. CIC is a transcription factor that is mutated in most patients with oligodendrogliomas; these mutations are believed to be a key oncogenic event in such cancers. Analysis of the Drosophila melanogaster ortholog of CIC, Capicua, indicates that CIC loss phenocopies activation of the EGFR/RAS/MAPK pathway, and studies in mammalian cells have demonstrated a role for CIC in repressing the transcription of the PEA3 subfamily of ETS transcription factors. Here, we address the mechanism by which CIC represses transcription and assess the functional consequences of CIC inactivation. Genome-wide binding patterns of CIC in several cell types revealed that CIC target genes were enriched for MAPK effector genes involved in cell-cycle regulation and proliferation. CIC binding to target genes was abolished by high MAPK activity, which led to their transcriptional activation. CIC interacted with the SIN3 deacetylation complex and, based on our results, we suggest that CIC functions as a transcriptional repressor through the recruitment of histone deacetylases. Independent single amino acid substitutions found in oligodendrogliomas prevented CIC from binding its target genes. Taken together, our results show that CIC is a transcriptional repressor of genes regulated by MAPK signaling, and that ablation of CIC function leads to increased histone acetylation levels and transcription at these genes, ultimately fueling mitogen-independent tumor growth. Significance: Inactivation of CIC inhibits its direct repression of MAPK pathway genes, leading to their increased expression and mitogen-independent growth. Graphical Abstract: http://cancerres.aacrjournals.org/content/canres/78/15/4114/F1.large.jpg. Cancer Res; 78(15); 4114–25. ©2018 AACR.</jats:p

Quality of life in patients with personality disorders seen at an ordinary psychiatric outpatient clinic

BACKGROUND: Epidemiological studies have found reduced health-related quality of life (QoL) in patients with personality disorders (PDs), but few clinical studies have examined QoL in PDs, and none of them are from an ordinary psychiatric outpatient clinic (POC). We wanted to examine QoL in patients with PDs seen at a POC, to explore the associations of QoL with established psychiatric measures, and to evaluate QoL as an outcome measure in PD patients. METHODS: 72 patients with PDs at a POC filled in the MOS Short Form 36 (SF-36), and two established psychiatric self-rating measures. A national norm sample was compared on the SF-36. An independent psychiatrist diagnosed PDs and Axis-I disorders by structured interviews and rated the Global Assessment of Functioning (GAF). All measurements were repeated in the 39 PD patients that attended the 2 years follow-up examination. RESULTS: PD patients showed high co-morbidity with other PDs and Axis I mental disorders, and they scored significantly lower on all the SF-36 dimensions than age- and gender-adjusted norms. Adjustment for co-morbid Axis I disorders had some influence, however. The SF-36 mental health, vitality, and social functioning were significantly associated with the GAF and the self-rated psychiatric measures. Significant changes at follow-up were found in the psychiatric measures, but only on the mental health and role-physical of the SF-36. CONCLUSION: Patients with PDs seen for treatment at a POC have globally poor QoL. Both physical and mental dimensions of the SF-36 are correlated with established psychiatric measures in such patients, but significant changes in these measures are only partly associated with changes in the SF-36 dimensions