Online 3D Bin Packing with Constrained Deep Reinforcement Learning

We solve a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP). In our problem, the agent has limited information about the items to be packed into the bin, and an item must be packed immediately after its arrival without buffering or readjusting. The item's placement also subjects to the constraints of collision avoidance and physical stability. We formulate this online 3D-BPP as a constrained Markov decision process. To solve the problem, we propose an effective and easy-to-implement constrained deep reinforcement learning (DRL) method under the actor-critic framework. In particular, we introduce a feasibility predictor to predict the feasibility mask for the placement actions and use it to modulate the action probabilities output by the actor during training. Such supervisions and transformations to DRL facilitate the agent to learn feasible policies efficiently. Our method can also be generalized e.g., with the ability to handle lookahead or items with different orientations. We have conducted extensive evaluation showing that the learned policy significantly outperforms the state-of-the-art methods. A user study suggests that our method attains a human-level performance

Hybrid quantum-classical heuristic for the bin packing problem

Optimization problems is one of the most challenging applications of quantum computers, as well as one of the most relevants. As a consequence, it has attracted huge efforts to obtain a speedup over classical algorithms using quantum resources. Up to now, many problems of different nature have been addressed through the perspective of this revolutionary computation paradigm, but there are still many open questions. In this work, a hybrid classical-quantum approach is presented for dealing with the one-dimensional Bin Packing Problem (1dBPP). The algorithm comprises two modules, each one designed for being executed in different computational ecosystems. First, a quantum subroutine seeks a set of feasible bin configurations of the problem at hand. Secondly, a classical computation subroutine builds complete solutions to the problem from the subsets given by the quantum subroutine. Being a hybrid solver, we have called our method H-BPP. To test our algorithm, we have built 18 different 1dBPP instances as a benchmarking set, in which we analyse the fitness, the number of solutions and the performance of the QC subroutine. Based on these figures of merit we verify that H-BPP is a valid technique to address the 1dBPP.QUANTEK project (ELKARTEK program from the Basque Government, expedient no. KK-2021/00070) Spanish Ramón y Cajal Grant RYC-2020-030503- I QMiCS (820505) and OpenSuperQ (820363) of the EU Flagship on Quantum Technologies EU FET Open project Quromorphic (828826) and EPIQUS (899368

DeepACO: Neural-enhanced Ant Systems for Combinatorial Optimization

Ant Colony Optimization (ACO) is a meta-heuristic algorithm that has been successfully applied to various Combinatorial Optimization Problems (COPs). Traditionally, customizing ACO for a specific problem requires the expert design of knowledge-driven heuristics. In this paper, we propose DeepACO, a generic framework that leverages deep reinforcement learning to automate heuristic designs. DeepACO serves to strengthen the heuristic measures of existing ACO algorithms and dispense with laborious manual design in future ACO applications. As a neural-enhanced meta-heuristic, DeepACO consistently outperforms its ACO counterparts on eight COPs using a single neural model and a single set of hyperparameters. As a Neural Combinatorial Optimization method, DeepACO performs better than or on par with problem-specific methods on canonical routing problems. Our code is publicly available at https://github.com/henry-yeh/DeepACO.Comment: Accepted at NeurIPS 202

Models and advanced optimization algorithms for the integrated management of logistics operations

Tese de Doutoramento em Engenharia Industrial e de Sistemas.In this thesis, we propose a set of algorithms regarding real combinatorial optimization problems in the context of transportation of goods. These problems consist in the combination of the vehicle routing problem with the two-dimensional bin-packing problem, which is also known as the vehicle routing problem with two-dimensional loading constraints. We also analyzed two related problems, namely the elementary shortest path and the vehicle routing problem with mixed linehauls and backhauls. In both problems, two-dimensional loading constraints are explicitly considered. Two column generation based approaches are proposed for the vehicle routing problem with two-dimensional constraints. The rst one relies on a branch-and-price algorithm with di erent branching schemes. A family of dual valid inequalities is also de ned, aiming to accelerate the convergence of the algorithm. The second approach is based on a set of di erent heuristics strategies, which are applied to the reformulated model. The elementary shortest path problem with two-dimensional constraints is addressed due to its importance in solving the subproblem of the column generation algorithms. To the best of our knowledge, we contribute with the rst approach for this problem, through di erent constructive strategies to achieve feasible solutions, and a variable neighborhood search algorithm in order to search for improved solutions. In what concerns the vehicle routing problem with mixed linehaul and backhauls and two-dimensional loading constraints, di erent variable neighborhood search algorithms are proposed. These algorithms explored various neighborhood structures, being some of those developed based on the features of the problem. All the proposed methods were implemented and experimentally tested. An exhaustive set of computational tests was conducted, using, for this purpose, a large group of benchmark instances. In some cases, a large set of benchmark instances was adapted in order asses the quality of the proposed models. All the obtained results are presented and discussed.Nesta tese, propomos um conjunto de algoritmos sobre problemas reais de otimiza c~ao combinat oria no contexto do transporte de bens. Estes problemas consistem na combina c~ao do problema de planeamento de rotas de ve culos com o problema de empacotamento bidimensional, que tamb em e conhecido como o problema de planeamento de rotas de ve culos com restri c~oes de carregamento bidimensional. Analisamos tamb em dois problemas relacionados, nomeadamente o problema de caminho mais curto e o problema de planeamento de rotas ve culos com entregas e recolhas indiferenciadas. Em ambos os problemas, s~ao explicitamente consideradas restri c~oes de carregamento bidimensional. Duas abordagens baseadas em gera c~ao de colunas s~ao propostas para o problema de planeamento de rotas de ve culos com restri c~oes de carregamento bidimensional. O primeiro baseia-se num algoritmo de parti c~ao e gera c~ao de colunas com diferentes estrat egias de parti c~ao. Uma fam lia de desigualdades duais v alidas e tamb em apresentada, com o objetivo de acelerar a converg^encia do algoritmo. A segunda abordagem baseia-se num conjunto de diferentes estrat egias heur sticas, que s~ao aplicadas ao modelo reformulado. O problema do caminho mais curto com restri c~oes de carregamento bidimensional e abordado devido a sua import^ancia na resolu c~ao do subproblema dos aos algoritmos de gera c~ao de colunas. De acordo com o nosso conhecimento, contribu mos com a primeira abordagem para este problema, atrav es de diferentes estrat egias construtivas para obter solu c~oes v alidas, e um algoritmo de pesquisa em vizinhan ca vari avel, com o objetivo de encontrar solu c~oes de melhor qualidade. No que concerne ao problema de planeamento de rotas de ve culos com entregas e recolhas indiferenciadas, diferentes algoritmos de pesquisa em vizinhan ca vari avel s~ao propostos. Estes algoritmos exploram v arias estruturas de vizinhan ca, sendo algumas destas desenvolvidas com base nas caracter sticas do problema. Todos os m etodos propostos foram implementados e testados experimentalmente. Um extenso conjunto de testes computacionais foi efetuado, utilizando um grande grupo de inst^ancias descritas na literatura. Em alguns casos, um grande conjunto de inst^ancias descritas na literatura foi adaptado com o objetivo de avaliar a qualidade dos m etodos propostos

Branching strategies for mixed-integer programs containing logical constraints and decomposable structure

Decision-making optimisation problems can include discrete selections, e.g. selecting a route, arranging non-overlapping items or designing a network of items. Branch-and-bound (B&B), a widely applied divide-and-conquer framework, often solves such problems by considering a continuous approximation, e.g. replacing discrete variable domains by a continuous superset. Such approximations weaken the logical relations, e.g. for discrete variables corresponding to Boolean variables. Branching in B&B reintroduces logical relations by dividing the search space. This thesis studies designing B&B branching strategies, i.e. how to divide the search space, for optimisation problems that contain both a logical and a continuous structure. We begin our study with a large-scale, industrially-relevant optimisation problem where the objective consists of machine-learnt gradient-boosted trees (GBTs) and convex penalty functions. GBT functions contain if-then queries which introduces a logical structure to this problem. We propose decomposition-based rigorous bounding strategies and an iterative heuristic that can be embedded into a B&B algorithm. We approach branching with two strategies: a pseudocost initialisation and strong branching that target the structure of GBT and convex penalty aspects of the optimisation objective, respectively. Computational tests show that our B&B approach outperforms state-of-the-art solvers in deriving rigorous bounds on optimality. Our second project investigates how satisfiability modulo theories (SMT) derived unsatisfiable cores may be utilised in a B&B context. Unsatisfiable cores are subsets of constraints that explain an infeasible result. We study two-dimensional bin packing (2BP) and develop a B&B algorithm that branches on SMT unsatisfiable cores. We use the unsatisfiable cores to derive cuts that break 2BP symmetries. Computational results show that our B&B algorithm solves 20% more instances when compared with commercial solvers on the tested instances. Finally, we study convex generalized disjunctive programming (GDP), a framework that supports logical variables and operators. Convex GDP includes disjunctions of mathematical constraints, which motivate branching by partitioning the disjunctions. We investigate separation by branching, i.e. eliminating solutions that prevent rigorous bound improvement, and propose a greedy algorithm for building the branches. We propose three scoring methods for selecting the next branching disjunction. We also analyse how to leverage infeasibility to expedite the B&B search. Computational results show that our scoring methods can reduce the number of explored B&B nodes by an order of magnitude when compared with scoring methods proposed in literature. Our infeasibility analysis further reduces the number of explored nodes.Open Acces

Optimised search heuristics: combining metaheuristics and exact methods to solve scheduling problems

Tese dout., Matemática, Investigação Operacional, Universidade do Algarve, 2009Scheduling problems have many real life applications, from automotive industry to air traffic control. These problems are defined by the need of processing a set of jobs on a shared set of resources. For most scheduling problems there is no known deterministic procedure that can solve them in polynomial time. This is the reason why researchers study methods that can provide a good solution in a reasonable amount of time. Much attention was given to the mathematical formulation of scheduling problems and the algebraic characterisation of the space of feasible solutions when exact algorithms were being developed; but exact methods proved inefficient to solve real sized instances. Local search based heuristics were developed that managed to quickly find good solutions, starting from feasible solutions produced by constructive heuristics. Local search algorithms have the disadvantage of stopping at the first local optimum they find when searching the feasible region. Research evolved to the design of metaheuristics, procedures that guide the search beyond the entrapment of local optima. Recently a new class of hybrid procedures, that combine local search based (meta) heuristics and exact algorithms of the operations research field, have been designed to find solutions for combinatorial optimisation problems, scheduling problems included. In this thesis we study the algebraic structure of scheduling problems; we address the existent hybrid procedures that combine exact methods with metaheuristics and produce a mapping of type of combination versus application and finally we develop new innovative metaheuristics and apply them to solve scheduling problems. These new methods developed include some combinatorial optimisation algorithms as components to guide the search in the solution space using the knowledge of the algebraic structure of the problem being solved. Namely we develop two new methods: a simple method that combines a GRASP procedure with a branch-and-bound algorithm; and a more elaborated procedure that combines the verification of the violation of valid inequalities with a tabu search. We focus on the job-shop scheduling problem

Exact and Heuristic Hybrid Approaches for Scheduling and Clustering Problems

This thesis deals with the design of exact and heuristic algorithms for scheduling and clustering combinatorial optimization problems. All the works are linked by the fact that all the presented methods arebasically hybrid algorithms, that mix techniques used in the world of combinatorial optimization. The algorithms are all efficient in practice, but the one presented in Chapter 4, that has mostly theoretical interest. Chapter 2 presents practical solution algorithms based on an ILP model for an energy scheduling combinatorial problem that arises in a smart building context. Chapter 3 presents a new cutting stock problem and introduce a mathematical formulation and a heuristic solution approach based on a heuristic column generation scheme. Chapter 4 provides an exact exponential algorithm, whose importance is only theoretical so far, for a classical scheduling problem: the Single Machine Total Tardiness Problem. The relevant aspect is that the designed algorithm has the best worst case complexity for the problem, that has been studied for several decades. Furthermore, such result is based on a new technique, called Branch and Merge, that avoids the solution of several equivalent sub-problems in a branching algorithm that requires polynomial space. As a consequence, such technique embeds in a branching algorithm ideas coming from other traditional computer science techniques such as dynamic programming and memorization, but keeping the space requirement polynomial. Chapter 5 provides an exact approach based on semidefinite programming and a matheuristic approach based on a quadratic solver for a fractional clustering combinatorial optimization problem, called Max-Mean Dispersion Problem. The matheuristic approach has the peculiarity of using a non-linear MIP solver. The proposed exact approach uses a general semidefinite programming relaxation and it is likely to be extended to other combinatorial problems with a fractional formulation. Chapter 6 proposes practical solution methods for a real world clustering problem arising in a smart city context. The solution algorithm is based on the solution of a Set Cover model via a commercial ILP solver. As a conclusion, the main contribution of this thesis is given by several approaches of practical or theoretical interest, for two classes of important combinatorial problems: clustering and scheduling. All the practical methods presented in the thesis are validated by extensive computational experiments, that compare the proposed methods with the ones available in the state of the art