Incorporating Memory and Learning Mechanisms Into Meta-RaPS

Due to the rapid increase of dimensions and complexity of real life problems, it has become more difficult to find optimal solutions using only exact mathematical methods. The need to find near-optimal solutions in an acceptable amount of time is a challenge when developing more sophisticated approaches. A proper answer to this challenge can be through the implementation of metaheuristic approaches. However, a more powerful answer might be reached by incorporating intelligence into metaheuristics. Meta-RaPS (Metaheuristic for Randomized Priority Search) is a metaheuristic that creates high quality solutions for discrete optimization problems. It is proposed that incorporating memory and learning mechanisms into Meta-RaPS, which is currently classified as a memoryless metaheuristic, can help the algorithm produce higher quality results. The proposed Meta-RaPS versions were created by taking different perspectives of learning. The first approach taken is Estimation of Distribution Algorithms (EDA), a stochastic learning technique that creates a probability distribution for each decision variable to generate new solutions. The second Meta-RaPS version was developed by utilizing a machine learning algorithm, Q Learning, which has been successfully applied to optimization problems whose output is a sequence of actions. In the third Meta-RaPS version, Path Relinking (PR) was implemented as a post-optimization method in which the new algorithm learns the good attributes by memorizing best solutions, and follows them to reach better solutions. The fourth proposed version of Meta-RaPS presented another form of learning with its ability to adaptively tune parameters. The efficiency of these approaches motivated us to redesign Meta-RaPS by removing the improvement phase and adding a more sophisticated Path Relinking method. The new Meta-RaPS could solve even the largest problems in much less time while keeping up the quality of its solutions. To evaluate their performance, all introduced versions were tested using the 0-1 Multidimensional Knapsack Problem (MKP). After comparing the proposed algorithms, Meta-RaPS PR and Meta-RaPS Q Learning appeared to be the algorithms with the best and worst performance, respectively. On the other hand, they could all show superior performance than other approaches to the 0-1 MKP in the literature

Combinatorial optimization and metaheuristics

Today, combinatorial optimization is one of the youngest and most active areas of discrete mathematics. It is a branch of optimization in applied mathematics and computer science, related to operational research, algorithm theory and computational complexity theory. It sits at the intersection of several fields, including artificial intelligence, mathematics and software engineering. Its increasing interest arises for the fact that a large number of scientific and industrial problems can be formulated as abstract combinatorial optimization problems, through graphs and/or (integer) linear programs. Some of these problems have polynomial-time (“efficient”) algorithms, while most of them are NP-hard, i.e. it is not proved that they can be solved in polynomial-time. Mainly, it means that it is not possible to guarantee that an exact solution to the problem can be found and one has to settle for an approximate solution with known performance guarantees. Indeed, the goal of approximate methods is to find “quickly” (reasonable run-times), with “high” probability, provable “good” solutions (low error from the real optimal solution). In the last 20 years, a new kind of algorithm commonly called metaheuristics have emerged in this class, which basically try to combine heuristics in high level frameworks aimed at efficiently and effectively exploring the search space. This report briefly outlines the components, concepts, advantages and disadvantages of different metaheuristic approaches from a conceptual point of view, in order to analyze their similarities and differences. The two very significant forces of intensification and diversification, that mainly determine the behavior of a metaheuristic, will be pointed out. The report concludes by exploring the importance of hybridization and integration methods

Xqx Based Modeling For General Integer Programming Problems

We present a new way to model general integer programming (IP) problems with in- equality and equality constraints using XQX. We begin with the definition of IP problems folloby their practical applications, and then present the existing XQX based models to handle such problems. We then present our XQX model for general IP problems (including binary IP) with equality and inequality constraints, and also show how this model can be applied to problems with just inequality constraints. We then present the local optima based solution procedure for our XQX model. We also present new theorems and their proofs for our XQX model. Next, we present a detailed literature survey on the 0-1 multidimensional knapsack problem (MDKP) and apply our XQX model using our simple heuristic procedure to solve benchmark problems. The 0-1 MDKP is a binary IP problem with inequality con- straints and variables with binary values. We apply our XQX model using a heuristics we developed on 0-1 MDKP problems of various sizes and found that our model can handle any problem sizes and can provide reasonable quality results in reasonable time. Finally, we apply our XQX model developed for general integer programming problems on the general multi-dimensional knapsack problems. The general MDKP is a general IP problem with inequality constraints where the variables are positive integers. We apply our XQX model on GMDKP problems of various sizes and find that it can provide reasonable quality results in reasonable time. We also find that it can handle problems of any size and provide fea- sible and good quality solutions irrespective of the starting solutions. We conclude with a discussion of some issues related with our XQX model

New variants of variable neighbourhood search for 0-1 mixed integer programming and clustering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Many real-world optimisation problems are discrete in nature. Although recent rapid developments in computer technologies are steadily increasing the speed of computations, the size of an instance of a hard discrete optimisation problem solvable in prescribed time does not increase linearly with the computer speed. This calls for the development of new solution methodologies for solving larger instances in shorter time. Furthermore, large instances of discrete optimisation problems are normally impossible to solve to optimality within a reasonable computational time/space and can only be tackled with a heuristic approach. In this thesis the development of so called matheuristics, the heuristics which are based on the mathematical formulation of the problem, is studied and employed within the variable neighbourhood search framework. Some new variants of the variable neighbourhood searchmetaheuristic itself are suggested, which naturally emerge from exploiting the information from the mathematical programming formulation of the problem. However, those variants may also be applied to problems described by the combinatorial formulation. A unifying perspective on modern advances in local search-based metaheuristics, a so called hyper-reactive approach, is also proposed. Two NP-hard discrete optimisation problems are considered: 0-1 mixed integer programming and clustering with application to colour image quantisation. Several new heuristics for 0-1 mixed integer programming problem are developed, based on the principle of variable neighbourhood search. One set of proposed heuristics consists of improvement heuristics, which attempt to find high-quality near-optimal solutions starting from a given feasible solution. Another set consists of constructive heuristics, which attempt to find initial feasible solutions for 0-1 mixed integer programs. Finally, some variable neighbourhood search based clustering techniques are applied for solving the colour image quantisation problem. All new methods presented are compared to other algorithms recommended in literature and a comprehensive performance analysis is provided. Computational results show that the methods proposed either outperform the existing state-of-the-art methods for the problems observed, or provide comparable results. The theory and algorithms presented in this thesis indicate that hybridisation of the CPLEX MIP solver and the VNS metaheuristic can be very effective for solving large instances of the 0-1 mixed integer programming problem. More generally, the results presented in this thesis suggest that hybridisation of exact (commercial) integer programming solvers and some metaheuristic methods is of high interest and such combinations deserve further practical and theoretical investigation. Results also show that VNS can be successfully applied to solving a colour image quantisation problem.Support from the Mathematical Institute, Serbian Academy of Sciences and Arts, are acknowledged for this research

New Heuristics For The 0-1 Multi-dimensional Knapsack Problems

This dissertation introduces new heuristic methods for the 0-1 multi-dimensional knapsack problem (0-1 MKP). 0-1 MKP can be informally stated as the problem of packing items into a knapsack while staying within the limits of different constraints (dimensions). Each item has a profit level assigned to it. They can be, for instance, the maximum weight that can be carried, the maximum available volume, or the maximum amount that can be afforded for the items. One main assumption is that we have only one item of each type, hence the problem is binary (0-1). The single dimensional version of the 0-1 MKP is the uni-dimensional single knapsack problem which can be solved in pseudo-polynomial time. However the 0-1 MKP is a strongly NP-Hard problem. Reduced cost values are rarely used resources in 0-1 MKP heuristics; using reduced cost information we introduce several new heuristics and also some improvements to past heuristics. We introduce two new ordering strategies, decision variable importance (DVI) and reduced cost based ordering (RCBO). We also introduce a new greedy heuristic concept which we call the sliding concept and a sub-branch of the sliding concept which we call sliding enumeration . We again use the reduced cost values within the sliding enumeration heuristic. RCBO is a brand new ordering strategy which proved useful in several methods such as improving Pirkul\u27s MKHEUR, a triangular distribution based probabilistic approach, and our own sliding enumeration. We show how Pirkul\u27s shadow price based ordering strategy fails to order the partial variables. We present a possible fix to this problem since there tends to be a high number of partial variables in hard problems. Therefore, this insight will help future researchers solve hard problems with more success. Even though sliding enumeration is a trivial method it found optima in less than a few seconds for most of our problems. We present different levels of sliding enumeration and discuss potential improvements to the method. Finally, we also show that in meta-heuristic approaches such as Drexl\u27s simulated annealing where random numbers are abundantly used, it would be better to use better designed probability distributions instead of random numbers

Hybrid tabu search – strawberry algorithm for multidimensional knapsack problem

Multidimensional Knapsack Problem (MKP) has been widely used to model real-life combinatorial problems. It is also used extensively in experiments to test the performances of metaheuristic algorithms and their hybrids. For example, Tabu Search (TS) has been successfully hybridized with other techniques, including particle swarm optimization (PSO) algorithm and the two-stage TS algorithm to solve MKP. In 2011, a new metaheuristic known as Strawberry algorithm (SBA) was initiated. Since then, it has been vastly applied to solve engineering problems. However, SBA has never been deployed to solve MKP. Therefore, a new hybrid of TS-SBA is proposed in this study to solve MKP with the objective of maximizing the total profit. The Greedy heuristics by ratio was employed to construct an initial solution. Next, the solution was enhanced by using the hybrid TS-SBA. The parameters setting to run the hybrid TS-SBA was determined by using a combination of Factorial Design of Experiments and Decision Tree Data Mining methods. Finally, the hybrid TS-SBA was evaluated using an MKP benchmark problem. It consisted of 270 test problems with different sizes of constraints and decision variables. The findings revealed that on average the hybrid TS-SBA was able to increase 1.97% profit of the initial solution. However, the best-known solution from past studies seemed to outperform the hybrid TS-SBA with an average difference of 3.69%. Notably, the novel hybrid TS-SBA proposed in this study may facilitate decisionmakers to solve real applications of MKP. It may also be applied to solve other variants of knapsack problems (KPs) with minor modifications

Generic parallel implementations for Tabu Search

Tabu Search (TS) is a meta-heuristic for solving combinatorial optimization problems. A review of existing implementations for TS reveals that, on the one hand, these implementations are ad hoc and, on the other hand, most of them run in a sequential setting. Indeed, the reported parallel implementations are few as compared to the sequential implementations. Due to increase in computing resources, especially in LAN environments, it is quite desirable to obtain parallel implementations of TS for solving problems arising in fields others than computer science, such as biology, control theory, etc., in which researchers and practitioners are less familiar with parallel programming. In this work we present a generic implementation of TS able to be run in sequential and parallel settings. The key point in our approach is the design and implementation in C++ of an algorithmic skeleton for TS embedding its main flow as well as several parallel implementations for the method. This is achieved through a separation of concerns: elements related to TS are provided by the skeleton, whereas the problem-dependent elements are expected to be provided by the user according to a fixed interface using purely sequential constructs. Thus, the skeleton has a unique interface but is expected to have many instantiations for concrete problems, all of them being able to run in a straightforward way using different parallel implementations. In order to assess the effectiveness of our approach, we have applied it to several NP-hard combinatorial optimization problems. We have considered developing time, flexibility and easiness of use, quality of solutions and computation efficiency. We have observed that our approach allows fast developing of problem instantiations. Moreover, the skeleton allows the user to configure and implement in different ways internal methods related to TS. Furthermore, the results obtained by our generic parallel implementations are efficient and report good quality results compared to the ones reported by ad hoc implementations. We exemplify our approach through the application to the 0-1 Multidimensional Knapsack problem. The experimental results obtained for standard benchmarks of this problem show that, in spite of the genericity and flexibility of our implementation, the resulting program provides high quality solutions very close to the optimal ones.Postprint (published version