A Bayesian Optimisation Approach for Multidimensional Knapsack Problem

This paper considers the application of Bayesian optimisation to the well-known multidimensional knapsack problem which is strongly NP-hard. For the multidimensional knapsack problem with a large number of items and knapsack constraints, a two-level formulation is presented to take advantage of the global optimisation capability of the Bayesian optimisation approach, and the efficiency of integer programming solvers on small problems. The first level makes the decisions about the optimal allocation of knapsack capacities to different item groups, while the second level solves a multidimensional knapsack problem of reduced size for each item group. To accelerate the Bayesian optimisation guided search process, various techniques are proposed including variable domain tightening, initialisation by the Genetic Algorithm, and optimisation landscape smoothing by local search. Computational experiments are carried out on the widely used benchmark instances with up to 100 items and 30 knapsack constraints. The preliminary results demonstrate the effectiveness of the proposed solution approach

A Weight-coded Evolutionary Algorithm for the Multidimensional Knapsack Problem

A revised weight-coded evolutionary algorithm (RWCEA) is proposed for solving multidimensional knapsack problems. This RWCEA uses a new decoding method and incorporates a heuristic method in initialization. Computational results show that the RWCEA performs better than a weight-coded evolutionary algorithm proposed by Raidl (1999) and to some existing benchmarks, it can yield better results than the ones reported in the OR-library.Comment: Submitted to Applied Mathematics and Computation on April 8, 201

Combinatorial optimization and metaheuristics

Today, combinatorial optimization is one of the youngest and most active areas of discrete mathematics. It is a branch of optimization in applied mathematics and computer science, related to operational research, algorithm theory and computational complexity theory. It sits at the intersection of several fields, including artificial intelligence, mathematics and software engineering. Its increasing interest arises for the fact that a large number of scientific and industrial problems can be formulated as abstract combinatorial optimization problems, through graphs and/or (integer) linear programs. Some of these problems have polynomial-time (“efficient”) algorithms, while most of them are NP-hard, i.e. it is not proved that they can be solved in polynomial-time. Mainly, it means that it is not possible to guarantee that an exact solution to the problem can be found and one has to settle for an approximate solution with known performance guarantees. Indeed, the goal of approximate methods is to find “quickly” (reasonable run-times), with “high” probability, provable “good” solutions (low error from the real optimal solution). In the last 20 years, a new kind of algorithm commonly called metaheuristics have emerged in this class, which basically try to combine heuristics in high level frameworks aimed at efficiently and effectively exploring the search space. This report briefly outlines the components, concepts, advantages and disadvantages of different metaheuristic approaches from a conceptual point of view, in order to analyze their similarities and differences. The two very significant forces of intensification and diversification, that mainly determine the behavior of a metaheuristic, will be pointed out. The report concludes by exploring the importance of hybridization and integration methods

Do sophisticated evolutionary algorithms perform better than simple ones?

Evolutionary algorithms (EAs) come in all shapes and sizes. Theoretical investigations focus on simple, bare-bones EAs while applications often use more sophisticated EAs that perform well on the problem at hand. What is often unclear is whether a large degree of algorithm sophistication is necessary, and if so, how much performance is gained by adding complexity to an EA. We address this question by comparing the performance of a wide range of theory-driven EAs, from bare-bones algorithms like the (1+1) EA, a (2+1) GA and simple population-based algorithms to more sophisticated ones like the (1+(λ,λ)) GA and algorithms using fast (heavy-tailed) mutation operators, against sophisticated and highly effective EAs from specific applications. This includes a famous and highly cited Genetic Algorithm for the Multidimensional Knapsack Problem and the Parameterless Population Pyramid for Ising Spin Glasses and MaxSat. While for the Multidimensional Knapsack Problem the sophisticated algorithm performs best, surprisingly, for large Ising and MaxSat instances the simplest algorithm performs best. We also derive conclusions about the usefulness of populations, crossover and fast mutation operators. Empirical results are supported by statistical tests and contrasted against theoretical work in an attempt to link theoretical and empirical results on EAs

A Binary differential search algorithm for the 0-1 multidimensional knapsack problem

The multidimensional knapsack problem (MKP) is known to be NP-hard in operations research and it has a wide range of applications in engineering and management. In this study, we propose a binary differential search method to solve 0-1 MKPs where the stochastic search is guided by a Brownian motion-like random walk. Our proposed method comprises two main operations: discrete solution generation and feasible solution production. Discrete solutions are generated by integrating Brownian motion-like random search with an integer-rounding operation. However, the rounded discrete variables may violate the constraints. Thus, a feasible solution production strategy is used to maintain the feasibility of the rounded discrete variables. To demonstrate the efficiency of our proposed algorithm, we solved various 0-1 MKPs using our proposed algorithm as well as some existing meta-heuristic methods. The numerical results obtained demonstrated that our algorithm performs better than existing meta-heuristic methods. Furthermore, our algorithm has the capacity to solve large-scale 0-1 MKPs

A case study of controlling crossover in a selection hyper-heuristic framework using the multidimensional knapsack problem

Hyper-heuristics are high-level methodologies for solving complex problems that operate on a search space of heuristics. In a selection hyper-heuristic framework, a heuristic is chosen from an existing set of low-level heuristics and applied to the current solution to produce a new solution at each point in the search. The use of crossover low-level heuristics is possible in an increasing number of general-purpose hyper-heuristic tools such as HyFlex and Hyperion. However, little work has been undertaken to assess how best to utilise it. Since a single-point search hyper-heuristic operates on a single candidate solution, and two candidate solutions are required for crossover, a mechanism is required to control the choice of the other solution. The frameworks we propose maintain a list of potential solutions for use in crossover. We investigate the use of such lists at two conceptual levels. First, crossover is controlled at the hyper-heuristic level where no problem-specific information is required. Second, it is controlled at the problem domain level where problem-specific information is used to produce good-quality solutions to use in crossover. A number of selection hyper-heuristics are compared using these frameworks over three benchmark libraries with varying properties for an NP-hard optimisation problem: the multidimensional 0-1 knapsack problem. It is shown that allowing crossover to be managed at the domain level outperforms managing crossover at the hyper-heuristic level in this problem domain. © 2016 Massachusetts Institute of Technolog