987 research outputs found
Algorithms for the continuous nonlinear resource allocation problem---new implementations and numerical studies
Patriksson (2008) provided a then up-to-date survey on the
continuous,separable, differentiable and convex resource allocation problem
with a single resource constraint. Since the publication of that paper the
interest in the problem has grown: several new applications have arisen where
the problem at hand constitutes a subproblem, and several new algorithms have
been developed for its efficient solution. This paper therefore serves three
purposes. First, it provides an up-to-date extension of the survey of the
literature of the field, complementing the survey in Patriksson (2008) with
more then 20 books and articles. Second, it contributes improvements of some of
these algorithms, in particular with an improvement of the pegging (that is,
variable fixing) process in the relaxation algorithm, and an improved means to
evaluate subsolutions. Third, it numerically evaluates several relaxation
(primal) and breakpoint (dual) algorithms, incorporating a variety of pegging
strategies, as well as a quasi-Newton method. Our conclusion is that our
modification of the relaxation algorithm performs the best. At least for
problem sizes up to 30 million variables the practical time complexity for the
breakpoint and relaxation algorithms is linear
Regularized Optimal Transport and the Rot Mover's Distance
This paper presents a unified framework for smooth convex regularization of
discrete optimal transport problems. In this context, the regularized optimal
transport turns out to be equivalent to a matrix nearness problem with respect
to Bregman divergences. Our framework thus naturally generalizes a previously
proposed regularization based on the Boltzmann-Shannon entropy related to the
Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We
call the regularized optimal transport distance the rot mover's distance in
reference to the classical earth mover's distance. We develop two generic
schemes that we respectively call the alternate scaling algorithm and the
non-negative alternate scaling algorithm, to compute efficiently the
regularized optimal plans depending on whether the domain of the regularizer
lies within the non-negative orthant or not. These schemes are based on
Dykstra's algorithm with alternate Bregman projections, and further exploit the
Newton-Raphson method when applied to separable divergences. We enhance the
separable case with a sparse extension to deal with high data dimensions. We
also instantiate our proposed framework and discuss the inherent specificities
for well-known regularizers and statistical divergences in the machine learning
and information geometry communities. Finally, we demonstrate the merits of our
methods with experiments using synthetic data to illustrate the effect of
different regularizers and penalties on the solutions, as well as real-world
data for a pattern recognition application to audio scene classification
Indefinite Knapsack Separable Quadratic Programming: Methods and Applications
Quadratic programming (QP) has received significant consideration due to an extensive list of applications. Although polynomial time algorithms for the convex case have been developed, the solution of large scale QPs is challenging due to the computer memory and speed limitations. Moreover, if the QP is nonconvex or includes integer variables, the problem is NP-hard. Therefore, no known algorithm can solve such QPs efficiently. Alternatively, row-aggregation and diagonalization techniques have been developed to solve QP by a sub-problem, knapsack separable QP (KSQP), which has a separable objective function and is constrained by a single knapsack linear constraint and box constraints.
KSQP can therefore be considered as a fundamental building-block to solve the general QP and is an important class of problems for research. For the convex KSQP, linear time algorithms are available. However, if some quadratic terms or even only one term is negative in KSQP, the problem is known to be NP-hard, i.e. it is notoriously difficult to solve.
The main objective of this dissertation is to develop efficient algorithms to solve general KSQP. Thus, the contributions of this dissertation are five-fold. First, this dissertation includes comprehensive literature review for convex and nonconvex KSQP by considering their computational efficiencies and theoretical complexities. Second, a new algorithm with quadratic time worst-case complexity is developed to globally solve the nonconvex KSQP, having open box constraints. Third, the latter global solver is utilized to develop a new bounding algorithm for general KSQP. Fourth, another new algorithm is developed to find a bound for general KSQP in linear time complexity. Fifth, a list of comprehensive applications for convex KSQP is introduced, and direct applications of indefinite KSQP are described and tested with our newly developed methods.
Experiments are conducted to compare the performance of the developed algorithms with that of local, global, and commercial solvers such as IBM CPLEX using randomly generated problems in the context of certain applications. The experimental results show that our proposed methods are superior in speed as well as in the quality of solutions
A Parametric Non-Convex Decomposition Algorithm for Real-Time and Distributed NMPC
A novel decomposition scheme to solve parametric non-convex programs as they
arise in Nonlinear Model Predictive Control (NMPC) is presented. It consists of
a fixed number of alternating proximal gradient steps and a dual update per
time step. Hence, the proposed approach is attractive in a real-time
distributed context. Assuming that the Nonlinear Program (NLP) is
semi-algebraic and that its critical points are strongly regular, contraction
of the sequence of primal-dual iterates is proven, implying stability of the
sub-optimality error, under some mild assumptions. Moreover, it is shown that
the performance of the optimality-tracking scheme can be enhanced via a
continuation technique. The efficacy of the proposed decomposition method is
demonstrated by solving a centralised NMPC problem to control a DC motor and a
distributed NMPC program for collaborative tracking of unicycles, both within a
real-time framework. Furthermore, an analysis of the sub-optimality error as a
function of the sampling period is proposed given a fixed computational power.Comment: 16 pages, 9 figure
Distributed Interior-point Method for Loosely Coupled Problems
In this paper, we put forth distributed algorithms for solving loosely
coupled unconstrained and constrained optimization problems. Such problems are
usually solved using algorithms that are based on a combination of
decomposition and first order methods. These algorithms are commonly very slow
and require many iterations to converge. In order to alleviate this issue, we
propose algorithms that combine the Newton and interior-point methods with
proximal splitting methods for solving such problems. Particularly, the
algorithm for solving unconstrained loosely coupled problems, is based on
Newton's method and utilizes proximal splitting to distribute the computations
for calculating the Newton step at each iteration. A combination of this
algorithm and the interior-point method is then used to introduce a distributed
algorithm for solving constrained loosely coupled problems. We also provide
guidelines on how to implement the proposed methods efficiently and briefly
discuss the properties of the resulting solutions.Comment: Submitted to the 19th IFAC World Congress 201
- …