340 research outputs found
Semidefinite Programming Relaxations of the Simplified Wasserstein Barycenter Problem: An ADMM Approach
The Simplified Wasserstein Barycenter problem, the problem of picking k points each chosen from a distinct set of n points as to minimize the sum of distances to their barycenter, finds applications in various areas of data science. Despite the simple formulation, it is a hard computational problem. The difficulty comes in the lack of efficient algorithms for approximating the solution. In this thesis, I propose a doubly non-negative relaxation to this problem and apply the alternating direction method of multipliers (ADMM) with intermediate update of multipliers, to efficiently compute tight lower and upper bounds on its optimal value for certain input data distributions. Our empirics show that generically the gap between upper and lower bounds is zero, though problems with symmetries exhibit positive gaps
The Minimization of Piecewise Functions: Pseudo Stationarity
There are many significant applied contexts that require the solution of
discontinuous optimization problems in finite dimensions. Yet these problems
are very difficult, both computationally and analytically. With the functions
being discontinuous and a minimizer (local or global) of the problems, even if
it exists, being impossible to verifiably compute, a foremost question is what
kind of ''stationary solutions'' one can expect to obtain; these solutions
provide promising candidates for minimizers; i.e., their defining conditions
are necessary for optimality. Motivated by recent results on sparse
optimization, we introduce in this paper such a kind of solution, termed
''pseudo B- (for Bouligand) stationary solution'', for a broad class of
discontinuous piecewise continuous optimization problems with objective and
constraint defined by indicator functions of the positive real axis composite
with functions that are possibly nonsmooth. We present two approaches for
computing such a solution. One approach is based on lifting the problem to a
higher dimension via the epigraphical formulation of the indicator functions;
this requires the addition of some auxiliary variables. The other approach is
based on certain continuous (albeit not necessarily differentiable) piecewise
approximations of the indicator functions and the convergence to a pseudo
B-stationary solution of the original problem is established. The conditions
for convergence are discussed and illustrated by an example
Convex relaxations for large-scale graphically structured nonconvex problems with spherical constraints: An optimal transport approach
In this paper we derive a moment relaxation for large-scale nonsmooth
optimization problems with graphical structure and spherical constraints. In
contrast to classical moment relaxations for global polynomial optimization
that suffer from the curse of dimensionality we exploit the partially separable
structure of the optimization problem to reduce the dimensionality of the
search space. Leveraging optimal transport and Kantorovich--Rubinstein duality
we decouple the problem and derive a tractable dual subspace approximation of
the infinite-dimensional problem using spherical harmonics. This allows us to
tackle possibly nonpolynomial optimization problems with spherical constraints
and geodesic coupling terms. We show that the duality gap vanishes in the limit
by proving that a Lipschitz continuous dual multiplier on a unit sphere can be
approximated as closely as desired in terms of a Lipschitz continuous
polynomial. The formulation is applied to sphere-valued imaging problems with
total variation regularization and graph-based simultaneous localization and
mapping (SLAM). In imaging tasks our approach achieves small duality gaps for a
moderate degree. In graph-based SLAM our approach often finds solutions which
after refinement with a local method are near the ground truth solution
Constrained Optimization of Rank-One Functions with Indicator Variables
Optimization problems involving minimization of a rank-one convex function
over constraints modeling restrictions on the support of the decision variables
emerge in various machine learning applications. These problems are often
modeled with indicator variables for identifying the support of the continuous
variables. In this paper we investigate compact extended formulations for such
problems through perspective reformulation techniques. In contrast to the
majority of previous work that relies on support function arguments and
disjunctive programming techniques to provide convex hull results, we propose a
constructive approach that exploits a hidden conic structure induced by
perspective functions. To this end, we first establish a convex hull result for
a general conic mixed-binary set in which each conic constraint involves a
linear function of independent continuous variables and a set of binary
variables. We then demonstrate that extended representations of sets associated
with epigraphs of rank-one convex functions over constraints modeling indicator
relations naturally admit such a conic representation. This enables us to
systematically give perspective formulations for the convex hull descriptions
of these sets with nonlinear separable or non-separable objective functions,
sign constraints on continuous variables, and combinatorial constraints on
indicator variables. We illustrate the efficacy of our results on sparse
nonnegative logistic regression problems
Proximal Galerkin: A structure-preserving finite element method for pointwise bound constraints
The proximal Galerkin finite element method is a high-order, low iteration
complexity, nonlinear numerical method that preserves the geometric and
algebraic structure of bound constraints in infinite-dimensional function
spaces. This paper introduces the proximal Galerkin method and applies it to
solve free boundary problems, enforce discrete maximum principles, and develop
scalable, mesh-independent algorithms for optimal design. The paper leads to a
derivation of the latent variable proximal point (LVPP) algorithm: an
unconditionally stable alternative to the interior point method. LVPP is an
infinite-dimensional optimization algorithm that may be viewed as having an
adaptive barrier function that is updated with a new informative prior at each
(outer loop) optimization iteration. One of the main benefits of this algorithm
is witnessed when analyzing the classical obstacle problem. Therein, we find
that the original variational inequality can be replaced by a sequence of
semilinear partial differential equations (PDEs) that are readily discretized
and solved with, e.g., high-order finite elements. Throughout this work, we
arrive at several unexpected contributions that may be of independent interest.
These include (1) a semilinear PDE we refer to as the entropic Poisson
equation; (2) an algebraic/geometric connection between high-order
positivity-preserving discretizations and certain infinite-dimensional Lie
groups; and (3) a gradient-based, bound-preserving algorithm for two-field
density-based topology optimization. The complete latent variable proximal
Galerkin methodology combines ideas from nonlinear programming, functional
analysis, tropical algebra, and differential geometry and can potentially lead
to new synergies among these areas as well as within variational and numerical
analysis
Conditional graph entropy as an alternating minimization problem
Conditional graph entropy is known to be the minimal rate for a natural
functional compression problem with side information at the receiver. In this
paper we show that it can be formulated as an alternating minimization problem,
which gives rise to a simple iterative algorithm for numerically computing
(conditional) graph entropy. This also leads to a new formula which shows that
conditional graph entropy is part of a more general framework: the solution of
an optimization problem over a convex corner. In the special case of graph
entropy (i.e., unconditioned version) this was known due to Csisz\'ar,
K\"orner, Lov\'asz, Marton, and Simonyi. In that case the role of the convex
corner was played by the so-called vertex packing polytope. In the conditional
version it is a more intricate convex body but the function to minimize is the
same. Furthermore, we describe a dual problem that leads to an optimality check
and an error bound for the iterative algorithm
Evolutionary Algorithms for Resource Constrained Project Scheduling Problems
The resource constrained project scheduling problems (RCPSPs) are well-known challenging research problems that require efficient solutions to meet the planning need of many practical high-value projects. RCPSPs are usually solved using optimization problem-solving approaches. In recent years, evolutionary algorithms (EAs) have been extensively employed to solve optimization problems, including RCPSPs. Despite that numerous EAs have been developed for solving various RCPSPs, there is no single algorithm that is consistently effective across a wide range of problems. In this context, this thesis aims to propose a few new algorithms for solving different RCPSPs that include singular-resource and multiple-resource problems with single and multiple objectives.
In general, RCPSPs are solved with an assumption that its activities are homogeneous, where all activities require all resource types. However, many activities are often singular, requiring only a single resource to complete an activity. Even though the existing algorithms that were developed for multi-resource problems, can solve this RCPSP variant with minor modifications, they are computationally expensive because they include some unnecessary resource constraints in the optimization process. In this thesis, at first, a problem with singular resource and single objective is considered. A heuristic-embedded genetic algorithm (GA) has been proposed for solving this problem, and it's effectiveness has been investigated. To enhance the performance of this algorithm, three heuristics are proposed and integrated with it. As there are no test problems available for singular resource problems, new benchmark problems are generated by modifying the existing multi-resource RCPSPs test set. As compared with experimental results of one of the modified algorithms and an exact solver, it was shown that the proposed algorithm achieved a better quality of solution while requiring a significantly smaller computational budget.
The proposed algorithm is then extended to make it suitable for solving multi-resource cases with a single objective, which are known as traditional RCPSPs. A self-adaptive GA is developed for this problem. The proposed self-adaptive component of the algorithm selects an appropriate genetic operator based on their performance as the evolution progresses and increases. To judge the performance of this algorithm, small to large-scale problem instances have been solved from the PSP Library and the results are compared with state-of-the-art algorithms. Based on the experimental results, it was found that the proposed algorithm was able to obtain much better solutions than the non-self-adaptive GA. Furthermore, the proposed approach outperformed the state-of-the-art algorithms.
In practice, cost of some resources varies with the day of the week or specific days in the month or year. To consider these day dependent costs, a new cost function is developed that is integrated with the usual cost fitness function in a multi-objective version of RCPSPs. Completion time is considered as the second objective. A heuristic-embedded self-adaptive multi-objective GA is proposed for both singular and multi-resource problems. In this algorithm, the selection mechanism is based on crowding distance and a reference point. A customized mutation operator is also introduced. The experimental results show that the proposed variant, with reference points-based selection, outperformed the variant, with crowding distance-based selection.
In many situations, resource availability varies with time, such as time of the day and in some particular days. A dynamic multi-operators-based GA is proposed to deal with this variant. Along with the genetic operators, two local search methods are also included in the self-adaptive mechanism. The proposed approach has been validated using both large-scale singular and multi-resource problem instances with a single objective. Its experimental results demonstrate the efficiency of the proposed dynamic multi-operator-based approach.
In summary, the proposed algorithms can solve different variants of RCPSPs that cover a broad spectrum of project scheduling problems, with significantly less computational tim
A Kernel-Density-Estimator Minimizing Movement Scheme for Diffusion Equations
The mathematical theory of a novel variational approximation scheme for
general second and fourth order partial differential equations
\begin{equation}\label{eq: A} \partial_t u -
\nabla\cdot\Big(u\nabla\frac{\delta\phi}{\delta
u}(u)\Big|\nabla\frac{\delta\phi}{\delta u}(u)\Big|^{q-2}\Big) \ = \ 0,
\quad\quad u\geq0, \end{equation} , is developed
Nonlocal Graph-PDEs and Riemannian Gradient Flows for Image Labeling
In this thesis, we focus on the image labeling problem which is the task of performing unique
pixel-wise label decisions to simplify the image while reducing its redundant information. We
build upon a recently introduced geometric approach for data labeling by assignment flows
[
APSS17
] that comprises a smooth dynamical system for data processing on weighted graphs.
Hereby we pursue two lines of research that give new application and theoretically-oriented
insights on the underlying segmentation task.
We demonstrate using the example of Optical Coherence Tomography (OCT), which is the
mostly used non-invasive acquisition method of large volumetric scans of human retinal tis-
sues, how incorporation of constraints on the geometry of statistical manifold results in a novel
purely data driven
geometric
approach for order-constrained segmentation of volumetric data
in any metric space. In particular, making diagnostic analysis for human eye diseases requires
decisive information in form of exact measurement of retinal layer thicknesses that has be done
for each patient separately resulting in an demanding and time consuming task. To ease the
clinical diagnosis we will introduce a fully automated segmentation algorithm that comes up
with a high segmentation accuracy and a high level of built-in-parallelism. As opposed to many
established retinal layer segmentation methods, we use only local information as input without
incorporation of additional global shape priors. Instead, we achieve physiological order of reti-
nal cell layers and membranes including a new formulation of ordered pair of distributions in an
smoothed energy term. This systematically avoids bias pertaining to global shape and is hence
suited for the detection of anatomical changes of retinal tissue structure. To access the perfor-
mance of our approach we compare two different choices of features on a data set of manually
annotated
3
D OCT volumes of healthy human retina and evaluate our method against state of
the art in automatic retinal layer segmentation as well as to manually annotated ground truth
data using different metrics.
We generalize the recent work [
SS21
] on a variational perspective on assignment flows and
introduce a novel nonlocal partial difference equation (G-PDE) for labeling metric data on graphs.
The G-PDE is derived as nonlocal reparametrization of the assignment flow approach that was
introduced in
J. Math. Imaging & Vision
58(2), 2017. Due to this parameterization, solving the
G-PDE numerically is shown to be equivalent to computing the Riemannian gradient flow with re-
spect to a nonconvex potential. We devise an entropy-regularized difference-of-convex-functions
(DC) decomposition of this potential and show that the basic geometric Euler scheme for inte-
grating the assignment flow is equivalent to solving the G-PDE by an established DC program-
ming scheme. Moreover, the viewpoint of geometric integration reveals a basic way to exploit
higher-order information of the vector field that drives the assignment flow, in order to devise a
novel accelerated DC programming scheme. A detailed convergence analysis of both numerical
schemes is provided and illustrated by numerical experiments
- …