66 research outputs found
Better Unrelated Machine Scheduling for Weighted Completion Time via Random Offsets from Non-Uniform Distributions
In this paper we consider the classic scheduling problem of minimizing total
weighted completion time on unrelated machines when jobs have release times,
i.e, using the three-field notation. For this
problem, a 2-approximation is known based on a novel convex programming (J. ACM
2001 by Skutella). It has been a long standing open problem if one can improve
upon this 2-approximation (Open Problem 8 in J. of Sched. 1999 by Schuurman and
Woeginger). We answer this question in the affirmative by giving a
1.8786-approximation. We achieve this via a surprisingly simple linear
programming, but a novel rounding algorithm and analysis. A key ingredient of
our algorithm is the use of random offsets sampled from non-uniform
distributions.
We also consider the preemptive version of the problem, i.e, . We again use the idea of sampling offsets from non-uniform
distributions to give the first better than 2-approximation for this problem.
This improvement also requires use of a configuration LP with variables for
each job's complete schedules along with more careful analysis. For both
non-preemptive and preemptive versions, we break the approximation barrier of 2
for the first time.Comment: 24 pages. To apper in FOCS 201
Branching strategies for mixed-integer programs containing logical constraints and decomposable structure
Decision-making optimisation problems can include discrete selections, e.g. selecting a route, arranging non-overlapping items or designing a network of items. Branch-and-bound (B&B), a widely applied divide-and-conquer framework, often solves such problems by considering a continuous approximation, e.g. replacing discrete variable domains by a continuous superset. Such approximations weaken the logical relations, e.g. for discrete variables corresponding to Boolean variables. Branching in B&B reintroduces logical relations by dividing the search space. This thesis studies designing B&B branching strategies, i.e. how to divide the search space, for optimisation problems that contain both a logical and a continuous structure.
We begin our study with a large-scale, industrially-relevant optimisation problem where the objective consists of machine-learnt gradient-boosted trees (GBTs) and convex penalty functions. GBT functions contain if-then queries which introduces a logical structure to this problem. We propose decomposition-based rigorous bounding strategies and an iterative heuristic that can be embedded into a B&B algorithm. We approach branching with two strategies: a pseudocost initialisation and strong branching that target the structure of GBT and convex penalty aspects of the optimisation objective, respectively. Computational tests show that our B&B approach outperforms state-of-the-art solvers in deriving rigorous bounds on optimality.
Our second project investigates how satisfiability modulo theories (SMT) derived unsatisfiable cores may be utilised in a B&B context. Unsatisfiable cores are subsets of constraints that explain an infeasible result. We study two-dimensional bin packing (2BP) and develop a B&B algorithm that branches on SMT unsatisfiable cores. We use the unsatisfiable cores to derive cuts that break 2BP symmetries. Computational results show that our B&B algorithm solves 20% more instances when compared with commercial solvers on the tested instances.
Finally, we study convex generalized disjunctive programming (GDP), a framework that supports logical variables and operators. Convex GDP includes disjunctions of mathematical constraints, which motivate branching by partitioning the disjunctions. We investigate separation by branching, i.e. eliminating solutions that prevent rigorous bound improvement, and propose a greedy algorithm for building the branches. We propose three scoring methods for selecting the next branching disjunction. We also analyse how to leverage infeasibility to expedite the B&B search. Computational results show that our scoring methods can reduce the number of explored B&B nodes by an order of magnitude when compared with scoring methods proposed in literature. Our infeasibility analysis further reduces the number of explored nodes.Open Acces
Active-set prediction for interior point methods
This research studies how to efficiently predict optimal active constraints of an inequality
constrained optimization problem, in the context of Interior Point Methods (IPMs).
We propose a framework based on shifting/perturbing the inequality constraints of the
problem.
Despite being a class of powerful tools for solving Linear Programming (LP) problems,
IPMs are well-known to encounter difficulties with active-set prediction due essentially
to their construction. When applied to an inequality constrained optimization
problem, IPMs generate iterates that belong to the interior of the set determined by
the constraints, thus avoiding/ignoring the combinatorial aspect of the solution. This
comes at the cost of difficulty in predicting the optimal active constraints that would
enable termination, as well as increasing ill-conditioning of the solution process. We
show that, existing techniques for active-set prediction, however, suffer from difficulties
in making an accurate prediction at the early stage of the iterative process of IPMs;
when these techniques are ready to yield an accurate prediction towards the end of
a run, as the iterates approach the solution set, the IPMs have to solve increasingly
ill-conditioned and hence difficult, subproblems.
To address this challenging question, we propose the use of controlled perturbations.
Namely, in the context of LP problems, we consider perturbing the inequality constraints
(by a small amount) so as to enlarge the feasible set. We show that if the perturbations
are chosen judiciously, the solution of the original problem lies on or close to the central
path of the perturbed problem. We solve the resulting perturbed problem(s) using a
path-following IPM while predicting on the way the active set of the original LP problem;
we find that our approach is able to accurately predict the optimal active set of the
original problem before the duality gap for the perturbed problem gets too small.
Furthermore, depending on problem conditioning, this prediction can happen sooner
than predicting the active set for the perturbed problem or for the original one if no
perturbations are used. Proof-of-concept algorithms are presented and encouraging
preliminary numerical experience is also reported when comparing activity prediction
for the perturbed and unperturbed problem formulations.
We also extend the idea of using controlled perturbations to enhance the capabilities
of optimal active-set prediction for IPMs for convex Quadratic Programming (QP) problems.
QP problems share many properties of LP, and based on these properties, some
results require more care; furthermore, encouraging preliminary numerical experience
is also presented for the QP case
Algorithms for the solution of the quadratic programming problem
The purpose of this dissertation was to provide a review of the theory of Optimization, in particular quadratic programming, and the algorithms suitable for solving both convex and non-convex quadratic programming problems. Optimization problems arise in a wide variety of fields and many can be effectively modeled with linear equations. However, there are problems for which linear models are not sufficient thus creating a need for non-linear systems. This dissertation includes a literature study of the formal theory necessary for understanding optimization and an investigation of the algorithms available for solving a special class of the non-linear programming problem, namely the quadratic programming problem. It was not the intention of this dissertation to discuss all possible algorithms for solving the quadratic programming problem, therefore certain algorithms for convex and non-convex quadratic programming problems were selected for a detailed discussion in the dissertation. Some of the algorithms were selected arbitrarily, because limited information was available comparing the efficiency of the various algorithms. Algorithms available for solving general non-linear programming problems were also included and briefly discussed as they can be used to solve quadratic programming problems. A number of algorithms were then selected for evaluation, depending on the frequency of use in practice and the availability of software implementing these algorithms. The evaluation included a theoretical and quantitative comparison of the algorithms. The quantitative results were analyzed and discussed and it was shown that the results supported the theoretical comparison. It was also shown that it is difficult to conclude that one algorithm is better than another as the efficiency of an algorithm greatly depends on the size of the problem, the complexity of an algorithm and many other implementation issues. Optimization problems arise continuously in a wide range of fields and thus create the need for effective methods of solving them. This dissertation provides the fundamental theory necessary for the understanding of optimization problems, with particular reference to quadratic programming problems and the algorithms that solve such problems. Keywords: Quadratic Programming, Quadratic Programming Algorithms, Optimization, Non-linear Programming, Convex, Non-convex
Algorithms for the solution of the quadratic programming problem
The purpose of this dissertation was to provide a review of the theory of Optimization, in particular quadratic programming, and the algorithms suitable for solving both convex and non-convex quadratic programming problems. Optimization problems arise in a wide variety of fields and many can be effectively modeled with linear equations. However, there are problems for which linear models are not sufficient thus creating a need for non-linear systems. This dissertation includes a literature study of the formal theory necessary for understanding optimization and an investigation of the algorithms available for solving a special class of the non-linear programming problem, namely the quadratic programming problem. It was not the intention of this dissertation to discuss all possible algorithms for solving the quadratic programming problem, therefore certain algorithms for convex and non-convex quadratic programming problems were selected for a detailed discussion in the dissertation. Some of the algorithms were selected arbitrarily, because limited information was available comparing the efficiency of the various algorithms. Algorithms available for solving general non-linear programming problems were also included and briefly discussed as they can be used to solve quadratic programming problems. A number of algorithms were then selected for evaluation, depending on the frequency of use in practice and the availability of software implementing these algorithms. The evaluation included a theoretical and quantitative comparison of the algorithms. The quantitative results were analyzed and discussed and it was shown that the results supported the theoretical comparison. It was also shown that it is difficult to conclude that one algorithm is better than another as the efficiency of an algorithm greatly depends on the size of the problem, the complexity of an algorithm and many other implementation issues. Optimization problems arise continuously in a wide range of fields and thus create the need for effective methods of solving them. This dissertation provides the fundamental theory necessary for the understanding of optimization problems, with particular reference to quadratic programming problems and the algorithms that solve such problems. Keywords: Quadratic Programming, Quadratic Programming Algorithms, Optimization, Non-linear Programming, Convex, Non-convex
Distributed Optimization with Application to Power Systems and Control
Mathematical optimization techniques are among the most successful tools for controlling technical systems optimally with feasibility guarantees. Yet, they are often centralized—all data has to be collected in one central and computationally powerful entity. Methods from distributed optimization overcome this limitation. Classical approaches, however, are often not applicable due to non-convexities. This work develops one of the first frameworks for distributed non-convex optimization
Lecture Notes on Applied Optimization
Paaßen B, Artelt A, Hammer B. Lecture Notes on Applied Optimization. Faculty of Technology, Bielefeld University; 2019.These lecture notes cover theory and algorithms for optimization from an application perspective. With respect to theory we cover basic definitions of optimization problems and their solutions, necessary and sufficient conditions of optimality, convex problems and optimality under convexity, Lagrange- and Wolfe dual forms, as well as Karush-Kuhn-Tucker conditions of optimality. With respect to algorithms we cover analytical optimization; numeric optimization, especially (conjugate) gradient descent, (pseudo-)Newton, trust region, log-barrier, penalty, and projection methods; probabilistic optimization, especially expectation maximization and max-product; linear and quadratic programming; and heuristics, especially the Nelder-Mead algorithm, CMA-ES, Bayesian optimization, hill climbing, simulated annealing, tabu search, branch-and-cut, and ant colony optimization. As such, this document provides a comprehensive overview of the most important optimization techniques for a wide range of application domains as well as their theoretical foundations
Learning neural ordinary differential equations for optimal control
Ce mémoire rassemble des éléments d'optimisation,
d'apprentissage profond et de contrôle optimal afin de répondre
aux problématiques
d'apprentissage et de planification
dans le contexte des systèmes dynamiques en temps continu.
Deux approches générales sont explorées.
D'abord, une approche basée sur la méthode du
maximum de vraisemblance
est présentée.
Ici, les trajectoires ``d'entrainement'' sont
échantillonnées depuis
la dynamique réelle, et à partir de celles-ci un modèle
de prédiction des états observés
est appris.
Une fois que l'apprentissage est terminé,
le modèle est utilisé pour la planification,
en utilisant la dynamique de l'environnement
et une fonction de coût pour construire un
programme non linéaire, qui est
par la suite résolu pour trouver une séquence
de contrôle optimal.
Ensuite, une approche de bout en bout
est proposée, dans laquelle la tâche d'apprentissage de modèle
dynamique et celle de planification se déroulent simultanément.
Ceci est illustré
dans le cadre d'un problème d'apprentissage par imitation,
où le modèle est mis à jour
en rétropropageant le signal de perte à travers
l'algorithme de planification. Grâce au fait que l'entrainement
est effectué de bout en bout, cette technique pourrait
constituer un sous-module de réseau de neurones
de plus grande taille, et pourrait être utilisée pour
fournir un biais inductif en faveur des comportements optimaux
dans le contexte de systèmes dynamiques en temps continu.
Ces méthodes sont toutes les deux conçues
pour fonctionner
avec des modèles d'équations différentielles ordinaires
paramétriques et neuronaux.
Également, inspiré par des applications réelles pertinentes,
un large recueil de systèmes dynamiques
et d'optimiseurs de trajectoire, nommé Myriad,
est implémenté; les algorithmes sont
testés et comparés sur une variété
de domaines de
la suite Myriad.This thesis brings together elements of optimization,
deep learning and optimal control to study the challenge of
learning and planning in continuous-time
dynamical systems. Two general
approaches are explored. First, a maximum likelihood
approach is
presented, in which training trajectories are sampled
from the true dynamics, and a model
is learned to accurately predict the state observations.
After training is completed, the learned model
is then used for planning,
by using the dynamics and cost function to construct a
nonlinear program, which can be solved to find a sequence
of optimal controls.
Second, a fully end-to-end approach
is proposed, in which the tasks of model learning and
planning are performed simultaneously. This is demonstrated
in an imitation learning setting, in which the model is updated
by backpropagating the loss signal through the planning
algorithm itself. Importantly, because it can be trained
in an end-to-end fashion, this technique can be included
as a sub-module of a larger neural network, and used to
provide an inductive bias towards behaving optimally
in a continuous-time dynamical system.
Both the maximum likelihood and end-to-end methods
are designed to work
with parametric and neural ordinary
differential equation models.
Inspired by relevant real-world applications,
a large repository of dynamical systems
and trajectory optimizers, named Myriad,
is also implemented.
The algorithms are
tested and compared on a variety
of domains within
the Myriad suite
- …