Solving Inverse Problems with Reinforcement Learning

In this paper, we formally introduce, with rigorous derivations, the use of reinforcement learning to the field of inverse problems by designing an iterative algorithm, called REINFORCE-IP, for solving a general type of non-linear inverse problem. By choosing specific probability models for the action-selection rule, we connect our approach to the conventional regularization methods of Tikhonov regularization and iterative regularization. For the numerical implementation of our approach, we parameterize the solution-searching rule with the help of neural networks and iteratively improve the parameter using a reinforcement-learning algorithm~-- REINFORCE. Under standard assumptions we prove the almost sure convergence of the parameter to a locally optimal value. Our work provides two typical examples (non-linear integral equations and parameter-identification problems in partial differential equations) of how reinforcement learning can be applied in solving non-linear inverse problems. Our numerical experiments show that REINFORCE-IP is an efficient algorithm that can escape from local minimums and identify multi-solutions for inverse problems with non-uniqueness.Comment: 33 pages, 10 figure

Monodromy Transform Approach to Solution of Some Field Equations in General Relativity and String Theory

A monodromy transform approach, presented in this communication, provides a general base for solution of space-time symmetry reductions of Einstein equations in all known integrable cases, which include vacuum, electrovacuum, massless Weyl spinor field and stiff matter fluid, as well as some string theory induced gravity models. It was found a special finite set of functional parameters, defined as the monodromy data for the fundamental solution of associated spectral problem. Similarly to the scattering data in the inverse scattering transform, these monodromy data can be used for characterization of any local solution of the field equations. A "direct" and "inverse" problems of such monodromy transform admit unambiguous solutions. For the linear singular integral equation with a scalar (i.e. non-matrix) kernel, which solves the inverse problem of this monodromy transform, an equivalent regularization -- a Fredholm linear integral equation of the second kind is constrcuted in several convenient forms. For arbitrary choice of the monodromy data a simple iterative method leads to an effective construction of the solution in terms of homogeneously convergent functional series.Comment: 7 pages, 1 figure, LaTeX2e; as submitted to the Proceedings of the Workshop "Nonlinearity Integrability and All That. Twenty Years After NEEDS'79", Gallipoli (Lecce), Italy - July 1 - July 10, 1999

An Alternative Lattice Field Theory Formulation Inspired by Lattice Supersymmetry

We propose an unconventional formulation of lattice field theories which is quite general, although originally motivated by the quest of exact lattice supersymmetry. Two long standing problems have a solution in this context: 1) Each degree of freedom on the lattice corresponds to $2^d$ degrees of freedom in the continuum, but all these doublers have (in the case of fermions) the same chirality and can be either identified, thus removing the degeneracy, or, in some theories with extended supersymmetry, identified with different members of the same supermultiplet. 2) The derivative operator, defined on the lattice as a suitable periodic function of the lattice momentum, is an addittive and conserved quantity, thus assuring that the Leibnitz rule is satisfied. This implies that the product of two fields on the lattice is replaced by a non-local "star product" which is however in general non-associative. Associativity of the "star product" poses strong restrictions on the form of the lattice derivative operator (which becomes the inverse gudermannian function of the lattice momentum) and has the consequence that the degrees of freedom of the lattice theory and of the continuum theory are in one-to-one correspondence, so that the two theories are eventually equivalent. Regularization of the ultraviolet divergences on the lattice is not associated to the lattice spacing, which does not act as a regulator, but may be obtained by a one parameter deformation of the lattice derivative, thus preserving the lattice structure even in the limit of infinite momentum cutoff. However this regularization breaks gauge invariance and a gauge invariant regularization within the lattice formulation is still lacking.Comment: 68 pages, 7 figure

Inverse Problems and Self-similarity in Imaging

This thesis examines the concept of image self-similarity and provides solutions to various associated inverse problems such as resolution enhancement and missing fractal codes. In general, many real-world inverse problems are ill-posed, mainly because of the lack of existence of a unique solution. The procedure of providing acceptable unique solutions to such problems is known as regularization. The concept of image prior, which has been of crucial importance in image modelling and processing, has also been important in solving inverse problems since it algebraically translates to the regularization procedure. Indeed, much recent progress in imaging has been due to advances in the formulation and practice of regularization. This, coupled with progress in optimization and numerical analysis, has yielded much improvement in computational methods of solving inverse imaging problems. Historically, the idea of self-similarity was important in the development of fractal image coding. Here we show that the self-similarity properties of natural images may be used to construct image priors for the purpose of addressing certain inverse problems. Indeed, new trends in the area of non-local image processing have provided a rejuvenated appreciation of image self-similarity and opportunities to explore novel self-similarity-based priors. We first revisit the concept of fractal-based methods and address some open theoretical problems in the area. This includes formulating a necessary and sufficient condition for the contractivity of the block fractal transform operator. We shall also provide some more generalized formulations of fractal-based self-similarity constraints of an image. These formulations can be developed algebraically and also in terms of the set-based method of Projection Onto Convex Sets (POCS). We then revisit the traditional inverse problems of single frame image zooming and multi-frame resolution enhancement, also known as super-resolution. Some ideas will be borrowed from newly developed non-local denoising algorithms in order to formulate self-similarity priors. Understanding the role of scale and choice of examples/samples is also important in these proposed models. For this purpose, we perform an extensive series of numerical experiments and analyze the results. These ideas naturally lead to the method of self-examples, which relies on the regularity properties of natural images at different scales, as a means of solving the single-frame image zooming problem. Furthermore, we propose and investigate a multi-frame super-resolution counterpart which does not require explicit motion estimation among video sequences

On a continuation approach in Tikhonov regularization and its application in piecewise-constant parameter identification

We present a new approach to convexification of the Tikhonov regularization using a continuation method strategy. We embed the original minimization problem into a one-parameter family of minimization problems. Both the penalty term and the minimizer of the Tikhonov functional become dependent on a continuation parameter. In this way we can independently treat two main roles of the regularization term, which are stabilization of the ill-posed problem and introduction of the a priori knowledge. For zero continuation parameter we solve a relaxed regularization problem, which stabilizes the ill-posed problem in a weaker sense. The problem is recast to the original minimization by the continuation method and so the a priori knowledge is enforced. We apply this approach in the context of topology-to-shape geometry identification, where it allows to avoid the convergence of gradient-based methods to a local minima. We present illustrative results for magnetic induction tomography which is an example of PDE constrained inverse problem

Optimization Methods for Inverse Problems

Optimization plays an important role in solving many inverse problems. Indeed, the task of inversion often either involves or is fully cast as a solution of an optimization problem. In this light, the mere non-linear, non-convex, and large-scale nature of many of these inversions gives rise to some very challenging optimization problems. The inverse problem community has long been developing various techniques for solving such optimization tasks. However, other, seemingly disjoint communities, such as that of machine learning, have developed, almost in parallel, interesting alternative methods which might have stayed under the radar of the inverse problem community. In this survey, we aim to change that. In doing so, we first discuss current state-of-the-art optimization methods widely used in inverse problems. We then survey recent related advances in addressing similar challenges in problems faced by the machine learning community, and discuss their potential advantages for solving inverse problems. By highlighting the similarities among the optimization challenges faced by the inverse problem and the machine learning communities, we hope that this survey can serve as a bridge in bringing together these two communities and encourage cross fertilization of ideas.Comment: 13 page