A branch-and-cut method for the bi-objective bi-dimensional knapsack problem

International audienceMulti-objective multi-dimensional knapsack problems (pOmDKP) are widely used to represent practical problems as capital budgeting or allocating processors. It aims to select a subset of n items such that the sum of weight of the selected items does not exceed the capacity on any of the m dimensions, while maximizing p objective functions. Each item has a weight on each dimension and a profit for each objective function. This problem is known for being particularly difficult as soon as the number of dimensions exceeds one, even in its single-objective version.There are many published papers focusing on the exact solution of multi-objective single-dimensional knapsack. The solutions methods are often two-phases methods. The second phase is either a branch-and-bound method (as in [1] for the bi-objective case or in [2] for the three-objective case), either a dynamic programming method [3], or a dedicated ranking method [2].Only a few works have studied exactly the multi-objective multi-dimensional case. Concerning the single-objective multi-dimensional knapsack problem, many works have investigated cutting inequalities to speed-up the computation of solution [4].In this work we are interested in the exact solution of the bi-objective bi-dimensional knapsack problem (2O2DKP), using a branch-and-cut method. A branch-and-cut method is a combination of a cutting plane method and a branch-and-bound method. According to its name, one of the main component of a branch-and-bound method aims at computing bounds of the problem. Convex relaxation has been a key component for successful bi-objective branch-and-bound algorithm (see for example [5]). It defines indeed a tight upper bound set, which can be computed easily if the single-objective version of the problem can be solved in (pseudo-)polynomial time. However, this is not the case for 2O2DKP. On the contrary, the linear relaxation remains relatively easy to compute, but the resulting bound set is less tight, which makes more difficult the exploration of nodes and leads to larger search-trees. To improve the quality of the upper bound set based on linear relaxation, we introduce cover inequalities at each node of the branch-and-bound method, turning it to a branch-and-cut method. Cover inequalities consist of cuts defined for single-objective binary problems [6]. A cover is a set of objects such that the sum of the weights associated to these objects exceeds the capacity. In [6], the authors remark that computing all possible cover inequalities would be time-consuming and even impossible to implement. Instead, they consider the optimal solution of the linear relaxation and solve a smaller binary problem to find a cover inequality that is violated. In the bi-objective context, the linear relaxation is described by a set of extreme points, which are associated to efficient solutions. Moreover, each of these efficient solutions may be fractional and have a different subset of fractional variables. The generation of cover inequalities is therefore more complex, particularly to get a good tradeoff between quality of the improved upper bound set defined and computational time. This leads to numerous strategies to generate cover inequalities. This presentation will describe the mechanisms used in the multi-objective branch-and-cut method that we have developed (separation procedure, bound sets, generation of cover inequalities...). These strategies have been then experimentally validated. [1] Visée, M., Teghem, J., Pirlot, M., Ulungu, E. L., March 1998. Two-phases method and branch and bound procedures to solve the bi–objective knapsack problem. Journal of Global Optimization 12, 139–155. [2] Jorge, J., May 2010. Nouvelles propositions pour la résolution exacte du sac à dos multi-objectif unidimensionnel en variables binaires. Thèse, Université de Nantes.[3] Delort, C., Spanjaard, O., 2010. Using bound sets in multiobjective optimization: Application to the biobjective binary knapsack problem. In: Festa, P. (Ed.), SEA. Vol 6049 of Lecture Notes in Computer Science. Springer, 253-265.[4] Osorio, M. A., Glover, F., Hammer, P., 2002. Cutting and surrogate constraint analysis for improved multidimensional knapsack solutions. Annals of Operations Research 117 (1-4), 71–93.[5] Sourd F. and Spanjaard O., 2008. A multi-objective branch-and bound framework: Application to the biobjective spanning tree problem. INFORMS Journal on Computing, 20:472-484.[6] Crowder, H., Johnson, E. L., Padberg, M. W., 1983. Solving large-scale zero-one linear programming problems. Operations Research 31 (5), 803–834

Xqx Based Modeling For General Integer Programming Problems

We present a new way to model general integer programming (IP) problems with in- equality and equality constraints using XQX. We begin with the definition of IP problems folloby their practical applications, and then present the existing XQX based models to handle such problems. We then present our XQX model for general IP problems (including binary IP) with equality and inequality constraints, and also show how this model can be applied to problems with just inequality constraints. We then present the local optima based solution procedure for our XQX model. We also present new theorems and their proofs for our XQX model. Next, we present a detailed literature survey on the 0-1 multidimensional knapsack problem (MDKP) and apply our XQX model using our simple heuristic procedure to solve benchmark problems. The 0-1 MDKP is a binary IP problem with inequality con- straints and variables with binary values. We apply our XQX model using a heuristics we developed on 0-1 MDKP problems of various sizes and found that our model can handle any problem sizes and can provide reasonable quality results in reasonable time. Finally, we apply our XQX model developed for general integer programming problems on the general multi-dimensional knapsack problems. The general MDKP is a general IP problem with inequality constraints where the variables are positive integers. We apply our XQX model on GMDKP problems of various sizes and find that it can provide reasonable quality results in reasonable time. We also find that it can handle problems of any size and provide fea- sible and good quality solutions irrespective of the starting solutions. We conclude with a discussion of some issues related with our XQX model

New Heuristics For The 0-1 Multi-dimensional Knapsack Problems

This dissertation introduces new heuristic methods for the 0-1 multi-dimensional knapsack problem (0-1 MKP). 0-1 MKP can be informally stated as the problem of packing items into a knapsack while staying within the limits of different constraints (dimensions). Each item has a profit level assigned to it. They can be, for instance, the maximum weight that can be carried, the maximum available volume, or the maximum amount that can be afforded for the items. One main assumption is that we have only one item of each type, hence the problem is binary (0-1). The single dimensional version of the 0-1 MKP is the uni-dimensional single knapsack problem which can be solved in pseudo-polynomial time. However the 0-1 MKP is a strongly NP-Hard problem. Reduced cost values are rarely used resources in 0-1 MKP heuristics; using reduced cost information we introduce several new heuristics and also some improvements to past heuristics. We introduce two new ordering strategies, decision variable importance (DVI) and reduced cost based ordering (RCBO). We also introduce a new greedy heuristic concept which we call the sliding concept and a sub-branch of the sliding concept which we call sliding enumeration . We again use the reduced cost values within the sliding enumeration heuristic. RCBO is a brand new ordering strategy which proved useful in several methods such as improving Pirkul\u27s MKHEUR, a triangular distribution based probabilistic approach, and our own sliding enumeration. We show how Pirkul\u27s shadow price based ordering strategy fails to order the partial variables. We present a possible fix to this problem since there tends to be a high number of partial variables in hard problems. Therefore, this insight will help future researchers solve hard problems with more success. Even though sliding enumeration is a trivial method it found optima in less than a few seconds for most of our problems. We present different levels of sliding enumeration and discuss potential improvements to the method. Finally, we also show that in meta-heuristic approaches such as Drexl\u27s simulated annealing where random numbers are abundantly used, it would be better to use better designed probability distributions instead of random numbers

Methods for integer programming

Imperial Users onl

A Random Forest Assisted Evolutionary Algorithm for Data-Driven Constrained Multi-Objective Combinatorial Optimization of Trauma Systems for publication

Many real-world optimization problems can be solved by using the data-driven approach only, simply because no analytic objective functions are available for evaluating candidate solutions. In this work, we address a class of expensive datadriven constrained multi-objective combinatorial optimization problems, where the objectives and constraints can be calculated only on the basis of large amount of data. To solve this class of problems, we propose to use random forests and radial basis function networks as surrogates to approximate both objective and constraint functions. In addition, logistic regression models are introduced to rectify the surrogate-assisted fitness evaluations and a stochastic ranking selection is adopted to further reduce the influences of the approximated constraint functions. Three variants of the proposed algorithm are empirically evaluated on multi-objective knapsack benchmark problems and two realworld trauma system design problems. Experimental results demonstrate that the variant using random forest models as the surrogates are effective and efficient in solving data-driven constrained multi-objective combinatorial optimization problems

Co-evolutionary Hybrid Bi-level Optimization

Multi-level optimization stems from the need to tackle complex problems involving multiple decision makers. Two-level optimization, referred as ``Bi-level optimization'', occurs when two decision makers only control part of the decision variables but impact each other (e.g., objective value, feasibility). Bi-level problems are sequential by nature and can be represented as nested optimization problems in which one problem (the ``upper-level'') is constrained by another one (the ``lower-level''). The nested structure is a real obstacle that can be highly time consuming when the lower-level is $\mathcal{NP}-hard$. Consequently, classical nested optimization should be avoided. Some surrogate-based approaches have been proposed to approximate the lower-level objective value function (or variables) to reduce the number of times the lower-level is globally optimized. Unfortunately, such a methodology is not applicable for large-scale and combinatorial bi-level problems. After a deep study of theoretical properties and a survey of the existing applications being bi-level by nature, problems which can benefit from a bi-level reformulation are investigated. A first contribution of this work has been to propose a novel bi-level clustering approach. Extending the well-know ``uncapacitated k-median problem'', it has been shown that clustering can be easily modeled as a two-level optimization problem using decomposition techniques. The resulting two-level problem is then turned into a bi-level problem offering the possibility to combine distance metrics in a hierarchical manner. The novel bi-level clustering problem has a very interesting property that enable us to tackle it with classical nested approaches. Indeed, its lower-level problem can be solved in polynomial time. In cooperation with the Luxembourg Centre for Systems Biomedicine (LCSB), this new clustering model has been applied on real datasets such as disease maps (e.g. Parkinson, Alzheimer). Using a novel hybrid and parallel genetic algorithm as optimization approach, the results obtained after a campaign of experiments have the ability to produce new knowledge compared to classical clustering techniques combining distance metrics in a classical manner. The previous bi-level clustering model has the advantage that the lower-level can be solved in polynomial time although the global problem is by definition $\mathcal{NP}$-hard. Therefore, next investigations have been undertaken to tackle more general bi-level problems in which the lower-level problem does not present any specific advantageous properties. Since the lower-level problem can be very expensive to solve, the focus has been turned to surrogate-based approaches and hyper-parameter optimization techniques with the aim of approximating the lower-level problem and reduce the number of global lower-level optimizations. Adapting the well-know bayesian optimization algorithm to solve general bi-level problems, the expensive lower-level optimizations have been dramatically reduced while obtaining very accurate solutions. The resulting solutions and the number of spared lower-level optimizations have been compared to the bi-level evolutionary algorithm based on quadratic approximations (BLEAQ) results after a campaign of experiments on official bi-level benchmarks. Although both approaches are very accurate, the bi-level bayesian version required less lower-level objective function calls. Surrogate-based approaches are restricted to small-scale and continuous bi-level problems although many real applications are combinatorial by nature. As for continuous problems, a study has been performed to apply some machine learning strategies. Instead of approximating the lower-level solution value, new approximation algorithms for the discrete/combinatorial case have been designed. Using the principle employed in GP hyper-heuristics, heuristics are trained in order to tackle efficiently the $\mathcal{NP}-hard$ lower-level of bi-level problems. This automatic generation of heuristics permits to break the nested structure into two separated phases: \emph{training lower-level heuristics} and \emph{solving the upper-level problem with the new heuristics}. At this occasion, a second modeling contribution has been introduced through a novel large-scale and mixed-integer bi-level problem dealing with pricing in the cloud, i.e., the Bi-level Cloud Pricing Optimization Problem (BCPOP). After a series of experiments that consisted in training heuristics on various lower-level instances of the BCPOP and using them to tackle the bi-level problem itself, the obtained results are compared to the ``cooperative coevolutionary algorithm for bi-level optimization'' (COBRA). Although training heuristics enables to \emph{break the nested structure}, a two phase optimization is still required. Therefore, the emphasis has been put on training heuristics while optimizing the upper-level problem using competitive co-evolution. Instead of adopting the classical decomposition scheme as done by COBRA which suffers from the strong epistatic links between lower-level and upper-level variables, co-evolving the solution and the mean to get to it can cope with these epistatic link issues. The ``CARBON'' algorithm developed in this thesis is a competitive and hybrid co-evolutionary algorithm designed for this purpose. In order to validate the potential of CARBON, numerical experiments have been designed and results have been compared to state-of-the-art algorithms. These results demonstrate that ``CARBON'' makes possible to address nested optimization efficiently

Optimizing Mean Mission Duration for Multiple-Payload Satellites

This thesis addresses the problem of optimally selecting and specifying satellite payloads for inclusion on a satellite bus to be launched into a constellation. The objective is to select and specify payloads so that the total lifetime utility of the constellation is maximized. The satellite bus is limited by finite power, weight, volume, and cost constraints. This problem is modeled as a classical knapsack problem in one and multiple dimensions, and dynamic programming and binary integer programming formulations are provided to solve the problem. Due to the computational complexity of the problem, the solution techniques include exact methods as well as four heuristic procedures including a greedy heuristic, two norm-based heuristics, and a simulated annealing heuristic. The performance of the exact and heuristic approaches is evaluated on the basis of solution quality and computation time by solving a series of notional and randomly-generated problem instances. The numerical results indicate that, when an exact solution is required for a moderately-sized constellation, the integer programming formulation is most reliable in solving the problem to optimality. However, if the problem size is very large, and near-optimal solutions are acceptable, then the simulated annealing algorithm performs best among the heuristic procedures