Nonconvex and mixed integer multiobjective optimization with an application to decision uncertainty

Multiobjective optimization problems commonly arise in different fields like economics or engineering. In general, when dealing with several conflicting objective functions, there is an infinite number of optimal solutions which cannot usually be determined analytically. This thesis presents new branch-and-bound-based approaches for computing the globally optimal solutions of multiobjective optimization problems of various types. New algorithms are proposed for smooth multiobjective nonconvex optimization problems with convex constraints as well as for multiobjective mixed integer convex optimization problems. Both algorithms guarantee a certain accuracy of the computed solutions, and belong to the first deterministic algorithms within their class of optimization problems. Additionally, a new approach to compute a covering of the optimal solution set of multiobjective optimization problems with decision uncertainty is presented. The three new algorithms are tested numerically. The results are evaluated in this thesis as well. The branch-and-bound based algorithms deal with box partitions and use selection rules, discarding tests and termination criteria. The discarding tests are the most important aspect, as they give criteria whether a box can be discarded as it does not contain any optimal solution. We present discarding tests which combine techniques from global single objective optimization with outer approximation techniques from multiobjective convex optimization and with the concept of local upper bounds from multiobjective combinatorial optimization. The new discarding tests aim to find appropriate lower bounds of subsets of the image set in order to compare them with known upper bounds numerically.Multikriterielle Optimierungprobleme sind in diversen Anwendungsgebieten wie beispielsweise in den Wirtschafts- oder Ingenieurwissenschaften zu finden. Da hierbei mehrere konkurrierende Zielfunktionen auftreten, ist die Lösungsmenge eines derartigen Optimierungsproblems im Allgemeinen unendlich groß und kann meist nicht in analytischer Form berechnet werden. In dieser Dissertation werden neue Branch-and-Bound basierte Algorithmen zur Lösung verschiedener Klassen von multikriteriellen Optimierungsproblemen entwickelt und vorgestellt. Der Branch-and-Bound Ansatz ist eine typische Methode der globalen Optimierung. Einer der neuen Algorithmen löst glatte multikriterielle nichtkonvexe Optimierungsprobleme mit konvexen Nebenbedingungen, während ein zweiter zur Lösung multikriterieller gemischt-ganzzahliger konvexer Optimierungsprobleme dient. Beide Algorithmen garantieren eine gewisse Genauigkeit der berechneten Lösungen und gehören damit zu den ersten deterministischen Algorithmen ihrer Art. Zusätzlich wird ein Algorithmus zur Berechnung einer Überdeckung der Lösungsmenge multikriterieller Optimierungsprobleme mit Entscheidungsunsicherheit vorgestellt. Alle drei Algorithmen wurden numerisch getestet. Die Ergebnisse werden ebenfalls in dieser Arbeit ausgewertet. Die neuen Algorithmen arbeiten alle mit Boxunterteilungen und nutzen Auswahlregeln, sowie Verwerfungs- und Terminierungskriterien. Dabei spielen gute Verwerfungskriterien eine zentrale Rolle. Diese entscheiden, ob eine Box verworfen werden kann, da diese sicher keine Optimallösung enthält. Die neuen Verwerfungskriterien nutzen Methoden aus der globalen skalarwertigen Optimierung, Approximationstechniken aus der multikriteriellen konvexen Optimierung sowie ein Konzept aus der kombinatorischen Optimierung. Dabei werden stets untere Schranken der Bildmengen konstruiert, die mit bisher berechneten oberen Schranken numerisch verglichen werden können

Enhancing numerical constraint propagation using multiple inclusion representations

Building tight and conservative enclosures of the solution set is of crucial importance in the design of efficient complete solvers for numerical constraint satisfaction problems (NCSPs). This paper proposes a novel generic algorithm enabling the cooperative use, during constraint propagation, of multiple enclosure techniques. The new algorithm brings into the constraint propagation framework the strength of techniques coming from different areas such as interval arithmetic, affine arithmetic, and mathematical programming. It is based on the directed acyclic graph (DAG) representation of NCSPs whose flexibility and expressiveness facilitates the design of fine-grained combination strategies for general factorable systems. The paper presents several possible combination strategies for creating practical instances of the generic algorithm. The experiments reported on a particular instance using interval constraint propagation, interval arithmetic, affine arithmetic, and linear programming illustrate the flexibility and efficiency of the approac

Human-Machine Collaborative Optimization via Apprenticeship Scheduling

Coordinating agents to complete a set of tasks with intercoupled temporal and resource constraints is computationally challenging, yet human domain experts can solve these difficult scheduling problems using paradigms learned through years of apprenticeship. A process for manually codifying this domain knowledge within a computational framework is necessary to scale beyond the ``single-expert, single-trainee" apprenticeship model. However, human domain experts often have difficulty describing their decision-making processes, causing the codification of this knowledge to become laborious. We propose a new approach for capturing domain-expert heuristics through a pairwise ranking formulation. Our approach is model-free and does not require enumerating or iterating through a large state space. We empirically demonstrate that this approach accurately learns multifaceted heuristics on a synthetic data set incorporating job-shop scheduling and vehicle routing problems, as well as on two real-world data sets consisting of demonstrations of experts solving a weapon-to-target assignment problem and a hospital resource allocation problem. We also demonstrate that policies learned from human scheduling demonstration via apprenticeship learning can substantially improve the efficiency of a branch-and-bound search for an optimal schedule. We employ this human-machine collaborative optimization technique on a variant of the weapon-to-target assignment problem. We demonstrate that this technique generates solutions substantially superior to those produced by human domain experts at a rate up to 9.5 times faster than an optimization approach and can be applied to optimally solve problems twice as complex as those solved by a human demonstrator.Comment: Portions of this paper were published in the Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) in 2016 and in the Proceedings of Robotics: Science and Systems (RSS) in 2016. The paper consists of 50 pages with 11 figures and 4 table

Branching strategies for mixed-integer programs containing logical constraints and decomposable structure

Decision-making optimisation problems can include discrete selections, e.g. selecting a route, arranging non-overlapping items or designing a network of items. Branch-and-bound (B&B), a widely applied divide-and-conquer framework, often solves such problems by considering a continuous approximation, e.g. replacing discrete variable domains by a continuous superset. Such approximations weaken the logical relations, e.g. for discrete variables corresponding to Boolean variables. Branching in B&B reintroduces logical relations by dividing the search space. This thesis studies designing B&B branching strategies, i.e. how to divide the search space, for optimisation problems that contain both a logical and a continuous structure. We begin our study with a large-scale, industrially-relevant optimisation problem where the objective consists of machine-learnt gradient-boosted trees (GBTs) and convex penalty functions. GBT functions contain if-then queries which introduces a logical structure to this problem. We propose decomposition-based rigorous bounding strategies and an iterative heuristic that can be embedded into a B&B algorithm. We approach branching with two strategies: a pseudocost initialisation and strong branching that target the structure of GBT and convex penalty aspects of the optimisation objective, respectively. Computational tests show that our B&B approach outperforms state-of-the-art solvers in deriving rigorous bounds on optimality. Our second project investigates how satisfiability modulo theories (SMT) derived unsatisfiable cores may be utilised in a B&B context. Unsatisfiable cores are subsets of constraints that explain an infeasible result. We study two-dimensional bin packing (2BP) and develop a B&B algorithm that branches on SMT unsatisfiable cores. We use the unsatisfiable cores to derive cuts that break 2BP symmetries. Computational results show that our B&B algorithm solves 20% more instances when compared with commercial solvers on the tested instances. Finally, we study convex generalized disjunctive programming (GDP), a framework that supports logical variables and operators. Convex GDP includes disjunctions of mathematical constraints, which motivate branching by partitioning the disjunctions. We investigate separation by branching, i.e. eliminating solutions that prevent rigorous bound improvement, and propose a greedy algorithm for building the branches. We propose three scoring methods for selecting the next branching disjunction. We also analyse how to leverage infeasibility to expedite the B&B search. Computational results show that our scoring methods can reduce the number of explored B&B nodes by an order of magnitude when compared with scoring methods proposed in literature. Our infeasibility analysis further reduces the number of explored nodes.Open Acces

Enhancing numerical constraint propagation using multiple inclusion representations

Building tight and conservative enclosures of the solution set is of crucial importance in the design of efficient complete solvers for numerical constraint satisfaction problems (NCSPs). This paper proposes a novel generic algorithm enabling the cooperative use, during constraint propagation, of multiple enclosure techniques. The new algorithm brings into the constraint propagation framework the strength of techniques coming from different areas such as interval arithmetic, affine arithmetic, and mathematical programming. It is based on the directed acyclic graph (DAG) representation of NCSPs whose flexibility and expressiveness facilitates the design of fine-grained combination strategies for general factorable systems. The paper presents several possible combination strategies for creating practical instances of the generic algorithm. The experiments reported on a particular instance using interval constraint propagation, interval arithmetic, affine arithmetic, and linear programming illustrate the flexibility and efficiency of the approach

Geometric-based Optimization Algorithms for Cable Routing and Branching in Cluttered Environments

The need for designing lighter and more compact systems often leaves limited space for planning routes for the connectors that enable interactions among the system’s components. Finding optimal routes for these connectors in a densely populated environment left behind at the detail design stage has been a challenging problem for decades. A variety of deterministic as well as heuristic methods has been developed to address different instances of this problem. While the focus of the deterministic methods is primarily on the optimality of the final solution, the heuristics offer acceptable solutions, especially for such problems, in a reasonable amount of time without guaranteeing to find optimal solutions. This study is an attempt to furthering the efforts in deterministic optimization methods to tackle the routing problem in two and three dimensions by focusing on the optimality of final solutions. The objective of this research is twofold. First, a mathematical framework is proposed for the optimization of the layout of wiring connectors in planar cluttered environments. The problem looks at finding the optimal tree network that spans multiple components to be connected with the aim of minimizing the overall length of the connectors while maximizing their common length (for maintainability and traceability of connectors). The optimization problem is formulated as a bi-objective problem and two solution methods are proposed: (1) to solve for the optimal locations of a known number of breakouts (where the connectors branch out) using mixed-binary optimization and visibility notion and (2) to find the minimum length tree that spans multiple components of the system and generates the optimal layout using the previously-developed convex hull based routing. The computational performance of these methods in solving a variety of problems is further evaluated. Second, the problem of finding the shortest route connecting two given nodes in a 3D cluttered environment is considered and addressed through deterministically generating a graphical representation of the collision-free space and searching for the shortest path on the found graph. The method is tested on sample workspaces with scattered convex polyhedra and its computational performance is evaluated. The work demonstrates the NP-hardness aspect of the problem which becomes quickly intractable as added components or increase in facets are considered

Multidisciplinary Design Optimization for Space Applications

Multidisciplinary Design Optimization (MDO) has been increasingly studied in aerospace engineering with the main purpose of reducing monetary and schedule costs. The traditional design approach of optimizing each discipline separately and manually iterating to achieve good solutions is substituted by exploiting the interactions between the disciplines and concurrently optimizing every subsystem. The target of the research was the development of a flexible software suite capable of concurrently optimizing the design of a rocket propellant launch vehicle for multiple objectives. The possibility of combining the advantages of global and local searches have been exploited in both the MDO architecture and in the selected and self developed optimization methodologies. Those have been compared according to computational efficiency and performance criteria. Results have been critically analyzed to identify the most suitable optimization approach for the targeted MDO problem

Sparse Plus Low Rank Matrix Decomposition: A Discrete Optimization Approach

We study the Sparse Plus Low-Rank decomposition problem (SLR), which is the problem of decomposing a corrupted data matrix into a sparse matrix of perturbations plus a low-rank matrix containing the ground truth. SLR is a fundamental problem in Operations Research and Machine Learning which arises in various applications, including data compression, latent semantic indexing, collaborative filtering, and medical imaging. We introduce a novel formulation for SLR that directly models its underlying discreteness. For this formulation, we develop an alternating minimization heuristic that computes high-quality solutions and a novel semidefinite relaxation that provides meaningful bounds for the solutions returned by our heuristic. We also develop a custom branch-and-bound algorithm that leverages our heuristic and convex relaxations to solve small instances of SLR to certifiable (near) optimality. Given an input $n$-by-$n$ matrix, our heuristic scales to solve instances where $n=10000$ in minutes, our relaxation scales to instances where $n=200$ in hours, and our branch-and-bound algorithm scales to instances where $n=25$ in minutes. Our numerical results demonstrate that our approach outperforms existing state-of-the-art approaches in terms of rank, sparsity, and mean-square error while maintaining a comparable runtime