An Analysis of Galerkin Proper Orthogonal Decomposition for Subdiffusion

In this work, we develop a novel Galerkin-L1-POD scheme for the subdiffusion model with a Caputo fractional derivative of order $\alpha\in (0,1)$ in time, which is often used to describe anomalous diffusion processes in heterogeneous media. The nonlocality of the fractional derivative requires storing all the solutions from time zero. The proposed scheme is based on continuous piecewise linear finite elements, L1 time stepping, and proper orthogonal decomposition (POD). By constructing an effective reduced-order scheme using problem-adapted basis functions, it can significantly reduce the computational complexity and storage requirement. We shall provide a complete error analysis of the scheme under realistic regularity assumptions by means of a novel energy argument. Extensive numerical experiments are presented to verify the convergence analysis and the efficiency of the proposed scheme.Comment: 25 pp, 5 figure

A Finite Element Method With Singularity Reconstruction for Fractional Boundary Value Problems

We consider a two-point boundary value problem involving a Riemann-Liouville fractional derivative of order \al\in (1,2) in the leading term on the unit interval $(0,1)$. Generally the standard Galerkin finite element method can only give a low-order convergence even if the source term is very smooth due to the presence of the singularity term x^{\al-1} in the solution representation. In order to enhance the convergence, we develop a simple singularity reconstruction strategy by splitting the solution into a singular part and a regular part, where the former captures explicitly the singularity. We derive a new variational formulation for the regular part, and establish that the Galerkin approximation of the regular part can achieve a better convergence order in the $L^2(0,1)$, H^{\al/2}(0,1) and $L^\infty(0,1)$-norms than the standard Galerkin approach, with a convergence rate for the recovered singularity strength identical with the $L^2(0,1)$ error estimate. The reconstruction approach is very flexible in handling explicit singularity, and it is further extended to the case of a Neumann type boundary condition on the left end point, which involves a strong singularity x^{\al-2}. Extensive numerical results confirm the theoretical study and efficiency of the proposed approach.Comment: 23 pp. ESAIM: Math. Model. Numer. Anal., to appea

Adaptive Reconstruction for Electrical Impedance Tomography with a Piecewise Constant Conductivity

In this work we propose and analyze a numerical method for electrical impedance tomography of recovering a piecewise constant conductivity from boundary voltage measurements. It is based on standard Tikhonov regularization with a Modica-Mortola penalty functional and adaptive mesh refinement using suitable a posteriori error estimators of residual type that involve the state, adjoint and variational inequality in the necessary optimality condition and a separate marking strategy. We prove the convergence of the adaptive algorithm in the following sense: the sequence of discrete solutions contains a subsequence convergent to a solution of the continuous necessary optimality system. Several numerical examples are presented to illustrate the convergence behavior of the algorithm.Comment: 26 pages, 12 figure

A new approach to nonlinear constrained Tikhonov regularization

We present a novel approach to nonlinear constrained Tikhonov regularization from the viewpoint of optimization theory. A second-order sufficient optimality condition is suggested as a nonlinearity condition to handle the nonlinearity of the forward operator. The approach is exploited to derive convergence rates results for a priori as well as a posteriori choice rules, e.g., discrepancy principle and balancing principle, for selecting the regularization parameter. The idea is further illustrated on a general class of parameter identification problems, for which (new) source and nonlinearity conditions are derived and the structural property of the nonlinearity term is revealed. A number of examples including identifying distributed parameters in elliptic differential equations are presented.Comment: 21 pages, to appear in Inverse Problem

On the Regularizing Property of Stochastic Gradient Descent

Stochastic gradient descent is one of the most successful approaches for solving large-scale problems, especially in machine learning and statistics. At each iteration, it employs an unbiased estimator of the full gradient computed from one single randomly selected data point. Hence, it scales well with problem size and is very attractive for truly massive dataset, and holds significant potentials for solving large-scale inverse problems. In the recent literature of machine learning, it was empirically observed that when equipped with early stopping, it has regularizing property. In this work, we rigorously establish its regularizing property (under \textit{a priori} early stopping rule), and also prove convergence rates under the canonical sourcewise condition, for minimizing the quadratic functional for linear inverse problems. This is achieved by combining tools from classical regularization theory and stochastic analysis. Further, we analyze the preasymptotic weak and strong convergence behavior of the algorithm. The theoretical findings shed insights into the performance of the algorithm, and are complemented with illustrative numerical experiments.Comment: 22 pages, better presentatio

Expectation Propagation for Nonlinear Inverse Problems -- with an Application to Electrical Impedance Tomography

In this paper, we study a fast approximate inference method based on expectation propagation for exploring the posterior probability distribution arising from the Bayesian formulation of nonlinear inverse problems. It is capable of efficiently delivering reliable estimates of the posterior mean and covariance, thereby providing an inverse solution together with quantified uncertainties. Some theoretical properties of the iterative algorithm are discussed, and the efficient implementation for an important class of problems of projection type is described. The method is illustrated with one typical nonlinear inverse problem, electrical impedance tomography with complete electrode model, under sparsity constraints. Numerical results for real experimental data are presented, and compared with that by Markov chain Monte Carlo. The results indicate that the method is accurate and computationally very efficient.Comment: Journal of Computational Physics, to appea

Lagrange optimality system for a class of nonsmooth convex optimization

In this paper, we revisit the augmented Lagrangian method for a class of nonsmooth convex optimization. We present the Lagrange optimality system of the augmented Lagrangian associated with the problems, and establish its connections with the standard optimality condition and the saddle point condition of the augmented Lagrangian, which provides a powerful tool for developing numerical algorithms. We apply a linear Newton method to the Lagrange optimality system to obtain a novel algorithm applicable to a variety of nonsmooth convex optimization problems arising in practical applications. Under suitable conditions, we prove the nonsingularity of the Newton system and the local convergence of the algorithm.Comment: 19 page

Numerical methods for time-fractional evolution equations with nonsmooth data: a concise overview

Over the past few decades, there has been substantial interest in evolution equations that involving a fractional-order derivative of order $\alpha\in(0,1)$ in time, due to their many successful applications in engineering, physics, biology and finance. Thus, it is of paramount importance to develop and to analyze efficient and accurate numerical methods for reliably simulating such models, and the literature on the topic is vast and fast growing. The present paper gives a concise overview on numerical schemes for the subdiffusion model with nonsmooth problem data, which are important for the numerical analysis of many problems arising in optimal control, inverse problems and stochastic analysis. We focus on the following aspects of the subdiffusion model: regularity theory, Galerkin finite element discretization in space, time-stepping schemes (including convolution quadrature and L1 type schemes), and space-time variational formulations, and compare the results with that for standard parabolic problems. Further, these aspects are showcased with illustrative numerical experiments and complemented with perspectives and pointers to relevant literature.Comment: 24 pages, 3 figure