95 research outputs found
Applications of a splitting algorithm to decomposition in convex programming and variational inequalities
Cover title.Includes bibliographical references.Partially supported by the U.S. Army Research Office (Center for Intelligent Control Systems) DAAL03-86-K-0171 Partially supported by the National Science Foundation. NSF-ECS-8519058by Paul Tseng
Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization
Generalized Linear Models (GLMs), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
arise in a range of applications in nonlinear
filtering and regression. Approximate Message Passing (AMP) methods, based on
loopy belief propagation, are a promising class of approaches for approximate
inference in these models. AMP methods are computationally simple, general, and
admit precise analyses with testable conditions for optimality for large i.i.d.
transforms . However, the algorithms can easily diverge for general
. This paper presents a convergent approach to the generalized AMP
(GAMP) algorithm based on direct minimization of a large-system limit
approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a
double-loop procedure, where the outer loop successively linearizes the LSL-BFE
and the inner loop minimizes the linearized LSL-BFE using the Alternating
Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP,
is similar in structure to the original GAMP method, but with an additional
least-squares minimization. It is shown that for strictly convex, smooth
penalties, ADMM-GAMP is guaranteed to converge to a local minima of the
LSL-BFE, thus providing a convergent alternative to GAMP that is stable under
arbitrary transforms. Simulations are also presented that demonstrate the
robustness of the method for non-convex penalties as well
An efficient symmetric primal-dual algorithmic framework for saddle point problems
In this paper, we propose a new primal-dual algorithmic framework for a class
of convex-concave saddle point problems frequently arising from image
processing and machine learning. Our algorithmic framework updates the primal
variable between the twice calculations of the dual variable, thereby appearing
a symmetric iterative scheme, which is accordingly called the {\bf s}ymmetric
{\bf p}r{\bf i}mal-{\bf d}ual {\bf a}lgorithm (SPIDA). It is noteworthy that
the subproblems of our SPIDA are equipped with Bregman proximal regularization
terms, which make SPIDA versatile in the sense that it enjoys an algorithmic
framework covering some existing algorithms such as the classical augmented
Lagrangian method (ALM), linearized ALM, and Jacobian splitting algorithms for
linearly constrained optimization problems. Besides, our algorithmic framework
allows us to derive some customized versions so that SPIDA works as efficiently
as possible for structured optimization problems. Theoretically, under some
mild conditions, we prove the global convergence of SPIDA and estimate the
linear convergence rate under a generalized error bound condition defined by
Bregman distance. Finally, a series of numerical experiments on the matrix
game, basis pursuit, robust principal component analysis, and image restoration
demonstrate that our SPIDA works well on synthetic and real-world datasets.Comment: 32 pages; 5 figure; 7 table
Technical Report: Distributed Asynchronous Large-Scale Mixed-Integer Linear Programming via Saddle Point Computation
We solve large-scale mixed-integer linear programs (MILPs) via distributed
asynchronous saddle point computation. This is motivated by the MILPs being
able to model problems in multi-agent autonomy, e.g., task assignment problems
and trajectory planning with collision avoidance constraints in multi-robot
systems. To solve a MILP, we relax it with a nonlinear program approximation
whose accuracy tightens as the number of agents increases relative to the
number of coupled constraints. Next, we form an equivalent Lagrangian saddle
point problem, and then regularize the Lagrangian in both the primal and dual
spaces to create a regularized Lagrangian that is
strongly-convex-strongly-concave. We then develop a parallelized algorithm to
compute saddle points of the regularized Lagrangian. This algorithm partitions
problems into blocks, which are either scalars or sub-vectors of the primal or
dual decision variables, and it is shown to tolerate asynchrony in the
computations and communications of primal and dual variables. Suboptimality
bounds and convergence rates are presented for convergence to a saddle point.
The suboptimality bound includes (i) the regularization error induced by
regularizing the Lagrangian and (ii) the suboptimality gap between solutions to
the original MILP and its relaxed form. Simulation results illustrate these
theoretical developments in practice, and show that relaxation and
regularization together have only a mild impact on the quality of solution
obtained.Comment: 14 pages, 2 figure
4D imaging in tomography and optical nanoscopy
Diese Dissertation gehört zu den Gebieten mathematische Bildverarbeitung und inverse Probleme. Ein inverses Problem ist die Aufgabe, Modellparameter anhand von gemessenen Daten zu berechnen. Solche Probleme treten in zahlreichen Anwendungen in Wissenschaft und Technik auf, z.B. in medizinischer Bildgebung, Biophysik oder Astronomie. Wir betrachten Rekonstruktionsprobleme mit Poisson Rauschen in der Tomographie und optischen Nanoskopie. Bei letzterer gilt es Bilder ausgehend von verzerrten und verrauschten Messungen zu rekonstruieren, wohingegen in der Positronen-Emissions-Tomographie die Aufgabe in der Visualisierung physiologischer Prozesse eines Patienten besteht. Standardmethoden zur 3D Bildrekonstruktion berücksichtigen keine zeitabhängigen Informationen oder Dynamik, z.B. Herzschlag oder Atmung in der Tomographie oder Zellmigration in der Mikroskopie. Diese Dissertation behandelt Modelle, Analyse und effiziente Algorithmen für 3D und 4D zeitabhängige inverse Probleme. This thesis contributes to the field of mathematical image processing
and inverse problems. An inverse problem is a task, where the values of
some model parameters must be computed from observed data. Such problems
arise in a wide variety of applications in sciences and engineering,
such as medical imaging, biophysics or astronomy. We mainly consider
reconstruction problems with Poisson noise in tomography and optical
nanoscopy. In the latter case, the task is to reconstruct images from
blurred and noisy measurements, whereas in positron emission tomography
the task is to visualize physiological processes of a patient. In 3D
static image reconstruction standard methods do not incorporate
time-dependent information or dynamics, e.g. heart beat or breathing in
tomography or cell motion in microscopy. This thesis is a treatise on
models, analysis and efficient algorithms to solve 3D and 4D
time-dependent inverse problems
- …