201 research outputs found
Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle-Pock algorithm
The primal-dual optimization algorithm developed in Chambolle and Pock (CP),
2011 is applied to various convex optimization problems of interest in computed
tomography (CT) image reconstruction. This algorithm allows for rapid
prototyping of optimization problems for the purpose of designing iterative
image reconstruction algorithms for CT. The primal-dual algorithm is briefly
summarized in the article, and its potential for prototyping is demonstrated by
explicitly deriving CP algorithm instances for many optimization problems
relevant to CT. An example application modeling breast CT with low-intensity
X-ray illumination is presented.Comment: Resubmitted to Physics in Medicine and Biology. Text has been
modified according to referee comments, and typos in the equations have been
correcte
Implementing a smooth exact penalty function for equality-constrained nonlinear optimization
We develop a general equality-constrained nonlinear optimization algorithm
based on a smooth penalty function proposed by Fletcher (1970). Although it was
historically considered to be computationally prohibitive in practice, we
demonstrate that the computational kernels required are no more expensive than
other widely accepted methods for nonlinear optimization. The main kernel
required to evaluate the penalty function and its derivatives is solving a
structured linear system. We show how to solve this system efficiently by
storing a single factorization each iteration when the matrices are available
explicitly. We further show how to adapt the penalty function to the class of
factorization-free algorithms by solving the linear system iteratively. The
penalty function therefore has promise when the linear system can be solved
efficiently, e.g., for PDE-constrained optimization problems where efficient
preconditioners exist. We discuss extensions including handling simple
constraints explicitly, regularizing the penalty function, and inexact
evaluation of the penalty function and its gradients. We demonstrate the merits
of the approach and its various features on some nonlinear programs from a
standard test set, and some PDE-constrained optimization problems
An asymptotically superlinearly convergent semismooth Newton augmented Lagrangian method for Linear Programming
Powerful interior-point methods (IPM) based commercial solvers, such as
Gurobi and Mosek, have been hugely successful in solving large-scale linear
programming (LP) problems. The high efficiency of these solvers depends
critically on the sparsity of the problem data and advanced matrix
factorization techniques. For a large scale LP problem with data matrix
that is dense (possibly structured) or whose corresponding normal matrix
has a dense Cholesky factor (even with re-ordering), these solvers may require
excessive computational cost and/or extremely heavy memory usage in each
interior-point iteration. Unfortunately, the natural remedy, i.e., the use of
iterative methods based IPM solvers, although can avoid the explicit
computation of the coefficient matrix and its factorization, is not practically
viable due to the inherent extreme ill-conditioning of the large scale normal
equation arising in each interior-point iteration. To provide a better
alternative choice for solving large scale LPs with dense data or requiring
expensive factorization of its normal equation, we propose a semismooth Newton
based inexact proximal augmented Lagrangian ({\sc Snipal}) method. Different
from classical IPMs, in each iteration of {\sc Snipal}, iterative methods can
efficiently be used to solve simpler yet better conditioned semismooth Newton
linear systems. Moreover, {\sc Snipal} not only enjoys a fast asymptotic
superlinear convergence but is also proven to enjoy a finite termination
property. Numerical comparisons with Gurobi have demonstrated encouraging
potential of {\sc Snipal} for handling large-scale LP problems where the
constraint matrix has a dense representation or has a dense
factorization even with an appropriate re-ordering.Comment: Due to the limitation "The abstract field cannot be longer than 1,920
characters", the abstract appearing here is slightly shorter than that in the
PDF fil
Privately Estimating a Gaussian: Efficient, Robust and Optimal
In this work, we give efficient algorithms for privately estimating a
Gaussian distribution in both pure and approximate differential privacy (DP)
models with optimal dependence on the dimension in the sample complexity. In
the pure DP setting, we give an efficient algorithm that estimates an unknown
-dimensional Gaussian distribution up to an arbitrary tiny total variation
error using samples while tolerating a
constant fraction of adversarial outliers. Here, is the condition
number of the target covariance matrix. The sample bound matches best
non-private estimators in the dependence on the dimension (up to a
polylogarithmic factor). We prove a new lower bound on differentially private
covariance estimation to show that the dependence on the condition number
in the above sample bound is also tight. Prior to our work, only
identifiability results (yielding inefficient super-polynomial time algorithms)
were known for the problem. In the approximate DP setting, we give an efficient
algorithm to estimate an unknown Gaussian distribution up to an arbitrarily
tiny total variation error using samples while tolerating
a constant fraction of adversarial outliers. Prior to our work, all efficient
approximate DP algorithms incurred a super-quadratic sample cost or were not
outlier-robust. For the special case of mean estimation, our algorithm achieves
the optimal sample complexity of , improving on a bound from prior work. Our pure DP algorithm relies on a recursive
private preconditioning subroutine that utilizes the recent work on private
mean estimation [Hopkins et al., 2022]. Our approximate DP algorithms are based
on a substantial upgrade of the method of stabilizing convex relaxations
introduced in [Kothari et al., 2022]
Fast Quasi-Newton Algorithms for Penalized Reconstruction in Emission Tomography and Further Improvements via Preconditioning
OAPA This paper reports on the feasibility of using a quasi-Newton optimization algorithm, limited-memory Broyden- Fletcher-Goldfarb-Shanno with boundary constraints (L-BFGSB), for penalized image reconstruction problems in emission tomography (ET). For further acceleration, an additional preconditioning technique based on a diagonal approximation of the Hessian was introduced. The convergence rate of L-BFGSB and the proposed preconditioned algorithm (L-BFGS-B-PC) was evaluated with simulated data with various factors, such as the noise level, penalty type, penalty strength and background level. Data of three 18F-FDG patient acquisitions were also reconstructed. Results showed that the proposed L-BFGS-B-PC outperforms L-BFGS-B in convergence rate for all simulated conditions and the patient data. Based on these results, L-BFGSB- PC shows promise for clinical application
- …