Non-local control in the conduction coefficients: well posedness and convergence to the local limit

We consider a problem of optimal distribution of conductivities in a system governed by a non-local diffusion law. The problem stems from applications in optimal design and more specifically topology optimization. We propose a novel parametrization of non-local material properties. With this parametrization the non-local diffusion law in the limit of vanishing non-local interaction horizons converges to the famous and ubiquitously used generalized Laplacian with SIMP (Solid Isotropic Material with Penalization) material model. The optimal control problem for the limiting local model is typically ill-posed and does not attain its infimum without additional regularization. Surprisingly, its non-local counterpart attains its global minima in many practical situations, as we demonstrate in this work. In spite of this qualitatively different behaviour, we are able to partially characterize the relationship between the non-local and the local optimal control problems. We also complement our theoretical findings with numerical examples, which illustrate the viability of our approach to optimal design practitioners

A Total Fractional-Order Variation Model for Image Restoration with Non-homogeneous Boundary Conditions and its Numerical Solution

To overcome the weakness of a total variation based model for image restoration, various high order (typically second order) regularization models have been proposed and studied recently. In this paper we analyze and test a fractional-order derivative based total $\alpha$-order variation model, which can outperform the currently popular high order regularization models. There exist several previous works using total $\alpha$-order variations for image restoration; however first no analysis is done yet and second all tested formulations, differing from each other, utilize the zero Dirichlet boundary conditions which are not realistic (while non-zero boundary conditions violate definitions of fractional-order derivatives). This paper first reviews some results of fractional-order derivatives and then analyzes the theoretical properties of the proposed total $\alpha$-order variational model rigorously. It then develops four algorithms for solving the variational problem, one based on the variational Split-Bregman idea and three based on direct solution of the discretise-optimization problem. Numerical experiments show that, in terms of restoration quality and solution efficiency, the proposed model can produce highly competitive results, for smooth images, to two established high order models: the mean curvature and the total generalized variation.Comment: 26 page

On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization

Conventional wisdom in deep learning states that increasing depth improves expressiveness but complicates optimization. This paper suggests that, sometimes, increasing depth can speed up optimization. The effect of depth on optimization is decoupled from expressiveness by focusing on settings where additional layers amount to overparameterization - linear neural networks, a well-studied model. Theoretical analysis, as well as experiments, show that here depth acts as a preconditioner which may accelerate convergence. Even on simple convex problems such as linear regression with $\ell_p$ loss, $p>2$, gradient descent can benefit from transitioning to a non-convex overparameterized objective, more than it would from some common acceleration schemes. We also prove that it is mathematically impossible to obtain the acceleration effect of overparametrization via gradients of any regularizer.Comment: Published at the International Conference on Machine Learning (ICML) 201

A piecewise linear FEM for an optimal control problem of fractional operators: error analysis on curved domains

We propose and analyze a new discretization technique for a linear-quadratic optimal control problem involving the fractional powers of a symmetric and uniformly elliptic second oder operator; control constraints are considered. Since these fractional operators can be realized as the Dirichlet-to-Neumann map for a nonuniformly elliptic equation, we recast our problem as a nonuniformly elliptic optimal control problem. The rapid decay of the solution to this problem suggests a truncation that is suitable for numerical approximation. We propose a fully discrete scheme that is based on piecewise linear functions on quasi-uniform meshes to approximate the optimal control and first-degree tensor product functions on anisotropic meshes for the optimal state variable. We provide an a priori error analysis that relies on derived Holder and Sobolev regularity estimates for the optimal variables and error estimates for an scheme that approximates fractional diffusion on curved domains; the latter being an extension of previous available results. The analysis is valid in any dimension. We conclude by presenting some numerical experiments that validate the derived error estimates

External optimal control of fractional parabolic PDEs

In this paper we introduce a new notion of optimal control, or source identification in inverse, problems with fractional parabolic PDEs as constraints. This new notion allows a source/control placement outside the domain where the PDE is fulfilled. We tackle the Dirichlet, the Neumann and the Robin cases. For the fractional elliptic PDEs this has been recently investigated by the authors in \cite{HAntil_RKhatri_MWarma_2018a}. The need for these novel optimal control concepts stems from the fact that the classical PDE models only allow placing the source/control either on the boundary or in the interior where the PDE is satisfied. However, the nonlocal behavior of the fractional operator now allows placing the control in the exterior. We introduce the notions of weak and very-weak solutions to the parabolic Dirichlet problem. We present an approach on how to approximate the parabolic Dirichlet solutions by the parabolic Robin solutions (with convergence rates). A complete analysis for the Dirichlet and Robin optimal control problems has been discussed. The numerical examples confirm our theoretical findings and further illustrate the potential benefits of nonlocal models over the local ones.Comment: arXiv admin note: text overlap with arXiv:1811.0451

Design of generalized fractional order gradient descent method

This paper focuses on the convergence problem of the emerging fractional order gradient descent method, and proposes three solutions to overcome the problem. In fact, the general fractional gradient method cannot converge to the real extreme point of the target function, which critically hampers the application of this method. Because of the long memory characteristics of fractional derivative, fixed memory principle is a prior choice. Apart from the truncation of memory length, two new methods are developed to reach the convergence. The one is the truncation of the infinite series, and the other is the modification of the constant fractional order. Finally, six illustrative examples are performed to illustrate the effectiveness and practicability of proposed methods.Comment: 8 pages, 16 figure