Local search inequalities

3noWe describe a general method for deriving new inequalities for integer programming formulations of combinatorial optimization problems. The inequalities, motivated by local search algorithms, are valid for all optimal solutions but not necessarily for all feasible solutions. These local search inequalities can help in either pruning the search tree at some nodes or in improving the bound of the LP relaxations.openopenLancia, Giuseppe; Rinaldi, Franca; Serafini, PaoloLancia, Giuseppe; Rinaldi, Franca; Serafini, Paol

Integer Programming: Optimization and Evaluation Are Equivalent

Link to conference publication published by Springer: http://dx.doi.org/10.1007/978-3-642-03367-4We show that if one can find the optimal value of an integer linear programming problem in polynomial time, then one can find an optimal solution in polynomial time. We also present a proper generalization to (general) integer programs and to local search problems of the well-known result that optimization and augmentation are equivalent for 0/1-integer programs. Among other things, our results imply that PLS-complete problems cannot have “near-exact” neighborhoods, unless PLS = P.United States. Office of Naval Research (ONR grant N00014-01208-1-0029

New Characterizations and Efficient Local Search for General Integer Linear Programming

Integer linear programming (ILP) models a wide range of practical combinatorial optimization problems and has significant impacts in industry and management sectors. This work proposes new characterizations of ILP with the concept of boundary solutions. Motivated by the new characterizations, we develop an efficient local search solver, which is the first local search solver for general ILP validated on a large heterogeneous problem dataset. We propose a new local search framework that switches between three modes, namely Search, Improve, and Restore modes. We design tailored operators adapted to different modes, thus improving the quality of the current solution according to different situations. For the Search and Restore modes, we propose an operator named tight move, which adaptively modifies variables' values, trying to make some constraint tight. For the Improve mode, an efficient operator lift move is proposed to improve the quality of the objective function while maintaining feasibility. Putting these together, we develop a local search solver for integer linear programming called Local-ILP. Experiments conducted on the MIPLIB dataset show the effectiveness of our solver in solving large-scale hard integer linear programming problems within a reasonably short time. Local-ILP is competitive and complementary to the state-of-the-art commercial solver Gurobi and significantly outperforms the state-of-the-art non-commercial solver SCIP. Moreover, our solver establishes new records for 6 MIPLIB open instances. The theoretical analysis of our algorithm is also presented, which shows our algorithm could avoid visiting unnecessary regions and also maintain good connectivity of targeted solutions.Comment: 36 pages, 2 figures, 7 table

Optimization bounds from the branching dual

We present a general method for obtaining strong bounds for discrete optimization problems that is based on a concept of branching duality. It can be applied when no useful integer programming model is available, and we illustrate this with the minimum bandwidth problem. The method strengthens a known bound for a given problem by formulating a dual problem whose feasible solutions are partial branching trees. It solves the dual problem with a “worst-bound” local search heuristic that explores neighboring partial trees. After proving some optimality properties of the heuristic, we show that it substantially improves known combinatorial bounds for the minimum bandwidth problem with a modest amount of computation. It also obtains significantly tighter bounds than depth-first and breadth-first branching, demonstrating that the dual perspective can lead to better branching strategies when the object is to find valid bounds.Accepted manuscrip

Random Search Plus: A more effective random search for machine learning hyperparameters optimization

Machine learning hyperparameter optimization has always been the key to improve model performance. There are many methods of hyperparameter optimization. The popular methods include grid search, random search, manual search, Bayesian optimization, population-based optimization, etc. Random search occupies less computations than the grid search, but at the same time there is a penalty for accuracy. However, this paper proposes a more effective random search method based on the traditional random search and hyperparameter space separation. This method is named random search plus. This thesis empirically proves that random search plus is more effective than random search. There are some case studies to do a comparison between them, which consists of four different machine learning algorithms including K-NN, K-means, Neural Networks and Support Vector Machine as optimization objects with three different size datasets including Iris flower, Pima Indians diabetes and MNIST handwritten dataset. Compared to traditional random search, random search plus can find a better hyperparameters or do an equivalent optimization as random search but with less time at most cases. With a certain hyperparameter space separation strategy, it can only need 10% time of random search to do an equivalent optimization or it can increase both the accuracy of supervised leanings and the silhouette coefficient of a supervised learning by 5%-30% in a same runtime as random search. The distribution of the best hyperparameters searched by the two methods in the hyperparameters space shows that random search plus is more global than random search. The thesis also discusses about some future works like the feasibility of using genetic algorithm to improve the local optimization ability of random search plus, space division of non-integer hyperparameters, etc

Parallelization of Dial-a-Ride Using Tabu Search

Dial-A-Ride is a transport system heavily constrained by following fleet size, vehicle capacity, and a fixed number of requests (pickup and drop-off points) with time windows. It is often modelled as Integer Programming, various solutions are proposed using heuristics. One such heuristic is Tabu Search . Tabu Search is very CPU intensive with its process of search, therefore many modern computing techniques like using GPUs have been employed to make it efficient. As with many other greedy algorithms, the local optima is not always the same as the global optima, so it is not possible to go past the local optima using greedy techniques for this problem. It is often modelled as Integer Programming, with the search space being very big, there are proven to not be so efficient. So, many heuristics have been proposed in the past, one such heuristic is Tabu Search . The local search of this heuristic uses memory to keep track of recent moves made and tries to avoid them for specified iterations (marks as Tabu) and also continues to explore the neighbourhood search space even with the degradation optimization function value, thus helping the algorithm to go past the local optima towards global optima. This thesis focuses on limitations of parallelizing DARP-TS for multi-core CPU, discussing major factors like (i) number of good moves in the neighbourhood and how we can estimate a value for N\_SIZE (number of parallel moves to make in each iteration), (ii) difference between a CPU core and a GPU core in the context of thread scheduling, memory layout and memory limitations, (iii) proposes few data-structures to keep the memory allocations low thus reducing the time for garbage collection and (iv) proposes an incremental way of calculating the value of optimization function in the local search phase which helps in mapping the execution and evaluation of N\_SIZE moves in each iteration onto the multiple CPU cores

The multiple team formation problem using sociometry

The Team Formation problem (TFP) has become a well-known problem in the OR literature over the last few years. In this problem, the allocation of multiple individuals that match a required set of skills as a group must be chosen to maximise one or several social positive attributes. Specifically, the aim of the current research is two-fold. First, two new dimensions of the TFP are added by considering multiple projects and fractions of people's dedication. This new problem is named the Multiple Team Formation Problem (MTFP). Second, an optimization model consisting in a quadratic objective function, linear constraints and integer variables is proposed for the problem. The optimization model is solved by three algorithms: a Constraint Programming approach provided by a commercial solver, a Local Search heuristic and a Variable Neighbourhood Search metaheuristic. These three algorithms constitute the first attempt to solve the MTFP, being a variable neighbourhood local search metaheuristic the most efficient in almost all cases. Applications of this problem commonly appear in real-life situations, particularly with the current and ongoing development of social network analysis. Therefore, this work opens multiple paths for future research

Domain-independent local search for linear integer optimization

Integer and combinatorial optimization problems constitute a major challenge for algorithmics. They arise when a large number of discrete organizational decisions have to be made, subject to constraints and optimization criteria. This thesis describes and investigates new domain-independent local search strategies for linear integer optimization. We introduce WSAT(OIP), an integer local search method which operates on an algebraic problem representation. WSAT(OIP) generalizes Walksat, a successful local search procedure for propositional satisfiability (SAT), to more expressive constraint systems. For this purpose, we introduce over-constrained integer programs (OIPs), a constraint class which is closely related to integer programs. OIP allows for a natural generalization of the principles of SAT local search to integer optimization. Further, it will be shown that OIPs are a special case of integer linear programs and permit combinations with linear programming for bound computation, initialization by rounding, search space reduction, and feasibility testing. The representation is similar enough to integer programs to make use of existing algebraic modeling languages as front-end to a local search solver. To improve performance on realistic problems, WSAT(OIP) incorporates strategies from Tabu Search. We experimentally investigate WSAT(OIP) for a variety of realistic integer optimization problems from the domains of time tabling, sports scheduling, radar surveillance, course assignment, and capacitated production planning. The experimental design examines efficiency, scaling (with increasing problem size and constrainedness), and robustness. The results demonstrate that integer local search can outperform or compete with state-of-the-art integer programming (IP) branch-and-bound and constraint programming (CP) approaches to these problems in finding near-optimal solutions. Key findings of our empirical study include that integer local search is able to solve difficult constraint problems from time-tabling and sports scheduling when cast into a 0-1 representation, which are beyond the scope of IP branch-and-bound strategies and for which devising robust constraint programs is a non-trivial task. For several realistic optimization problems (0-1 integer and finite domain) we show that integer local search exhibits graceful runtime scaling with increasing problem size and constrainedness. It can therefore significantly outperform IP branch-and-bound strategies on large or tightly constrained problems in finding near-optimal solutions. The problems under consideration are mostly beyond the limitations of a previous general-purpose simulated annealing strategy for 0-1 integer programs.Ganzzahlige und kombinatorische Optimierungsprobleme stellen eine schwierige Herausforderung im Gebiet der Algorithmen dar. Sie treten auf, wenn eine große Anzahl diskreter organisatorischer Entscheidungen unter Berücksichtigung von Constraints und Optimierungskriterien zu treffen sind. Diese Arbeit beschreibt und untersucht neue, domänenunabhängige Strategien der lokalen Suche zur ganzzahligen linearen Optimierung. Wir beschreiben WSAT(OIP), eine Strategie "ganzzahliger lokaler Suche';, die auf einer algebraischen Problemrepräsentation operiert. WSAT(OIP) verallgemeinert Walksat, eine erfolgreiche Prozedur lokaler Suche für das Erfüllbarkeitsproblem der Aussagenlogik (SAT), auf ausdrucksstärkere Constraint-Systeme. Für diesen Zweck führen wir die Klasse der "Over-constrained Integer Programs';(OIPs) ein, eine Constraint-Klasse, die eng mit ganzzahligen Programmen verwandt ist. OIPs erlauben einerseits eine natürliche Verallgemeinerung der Prinzipien von lokaler Suche für SAT. Andererseits sind sie ein Spezialfall der ganzzahligen linearen Programme und ermöglichen die Kombination mit linearer Programmierung zur Berechnung von Schranken, Initialisierung durch Rundung, Suchraum-Reduktion und für Gültigkeits-Tests. OIPs sind ganzzahligen Programmen ähnlich, so daß existierende algebraische Modellierungssprachen als Eingabeschnittstelle für einen Problemlöser benutzt werden können, der auf lokaler Suche basiert. Um die Performanz auf realistischen Problemen zu verbessern, ist WSAT(OIP) mit Strategien der Tabu-Suche ausgestattet. Wir führen eine experimentelle Untersuchung von WSAT(OIP) auf einer Reihe von realistischen ganzzahligen Constraint- und Optimierungsproblemen durch. Die Probleme stammen aus den Domänen Zeitplan-Erstellung, Sport-Ablaufplanung, Radar- Überwachung, Kurs-Zuteilung und Produktions-Planung. Das experimentelle Design untersucht Effizienz, Skalierung mit zunehmender Problemgröße und stärkeren Constraints sowie Robustheit. Die Ergebnisse zeigen, daß ganzzahlige lokale Suche bezüglich Performanz auf diesen Problemklassen zeitgemäße Ansätze der ganzzahligen Programmierung und der Constraint-Programmierung beim Finden nahe-optimaler Lösungen schlägt oder mit ihnen konkurriert. Kernergebnisse der empirischen Untersuchung sind, daß ganzzahlige lokale Suche in der Lage ist, schwierige Constraint-Probleme der Zeitplan-Erstellung und Sport-Ablaufplanung in einer 0-1 Repräsentation zu lösen, die außerhalb der Grenzen der ganzzahligen linearen Programmierung liegen, und für die die Entwicklung eines robustes Constraint-Programms eine nicht-triviale Aufgabe darstellt. Für mehrere realistische Optimierungsprobleme (ganzzahlig 0-1 und endliche Bereiche)zeigen wir, daß ganzzahlige lokale Suche eine günstige Skalierung der Laufzeit mit zunehmender Problemgröße und Constrainedness aufweist. Dadurch zeigt das Verfahren auf großen Problemen und auf Problemen mit starken Constraints deutlich bessere Performanz für das Finden nahe-Lösungen als die Branch-and-Bound Strategie der ganzzahligen Programmierung. Die untersuchten Probleme liegen zumeist außerhalb der Grenzen einer existierenden Simulated Annealing Strategie für allgemeine lineare 0-1 Programme

Domain-independent local search for linear integer optimization

Integer and combinatorial optimization problems constitute a major challenge for algorithmics. They arise when a large number of discrete organizational decisions have to be made, subject to constraints and optimization criteria. This thesis describes and investigates new domain-independent local search strategies for linear integer optimization. We introduce WSAT(OIP), an integer local search method which operates on an algebraic problem representation. WSAT(OIP) generalizes Walksat, a successful local search procedure for propositional satisfiability (SAT), to more expressive constraint systems. For this purpose, we introduce over-constrained integer programs (OIPs), a constraint class which is closely related to integer programs. OIP allows for a natural generalization of the principles of SAT local search to integer optimization. Further, it will be shown that OIPs are a special case of integer linear programs and permit combinations with linear programming for bound computation, initialization by rounding, search space reduction, and feasibility testing. The representation is similar enough to integer programs to make use of existing algebraic modeling languages as front-end to a local search solver. To improve performance on realistic problems, WSAT(OIP) incorporates strategies from Tabu Search. We experimentally investigate WSAT(OIP) for a variety of realistic integer optimization problems from the domains of time tabling, sports scheduling, radar surveillance, course assignment, and capacitated production planning. The experimental design examines efficiency, scaling (with increasing problem size and constrainedness), and robustness. The results demonstrate that integer local search can outperform or compete with state-of-the-art integer programming (IP) branch-and-bound and constraint programming (CP) approaches to these problems in finding near-optimal solutions. Key findings of our empirical study include that integer local search is able to solve difficult constraint problems from time-tabling and sports scheduling when cast into a 0-1 representation, which are beyond the scope of IP branch-and-bound strategies and for which devising robust constraint programs is a non-trivial task. For several realistic optimization problems (0-1 integer and finite domain) we show that integer local search exhibits graceful runtime scaling with increasing problem size and constrainedness. It can therefore significantly outperform IP branch-and-bound strategies on large or tightly constrained problems in finding near-optimal solutions. The problems under consideration are mostly beyond the limitations of a previous general-purpose simulated annealing strategy for 0-1 integer programs.Ganzzahlige und kombinatorische Optimierungsprobleme stellen eine schwierige Herausforderung im Gebiet der Algorithmen dar. Sie treten auf, wenn eine große Anzahl diskreter organisatorischer Entscheidungen unter Berücksichtigung von Constraints und Optimierungskriterien zu treffen sind. Diese Arbeit beschreibt und untersucht neue, domänenunabhängige Strategien der lokalen Suche zur ganzzahligen linearen Optimierung. Wir beschreiben WSAT(OIP), eine Strategie "ganzzahliger lokaler Suche\u27;, die auf einer algebraischen Problemrepräsentation operiert. WSAT(OIP) verallgemeinert Walksat, eine erfolgreiche Prozedur lokaler Suche für das Erfüllbarkeitsproblem der Aussagenlogik (SAT), auf ausdrucksstärkere Constraint-Systeme. Für diesen Zweck führen wir die Klasse der "Over-constrained Integer Programs\u27;(OIPs) ein, eine Constraint-Klasse, die eng mit ganzzahligen Programmen verwandt ist. OIPs erlauben einerseits eine natürliche Verallgemeinerung der Prinzipien von lokaler Suche für SAT. Andererseits sind sie ein Spezialfall der ganzzahligen linearen Programme und ermöglichen die Kombination mit linearer Programmierung zur Berechnung von Schranken, Initialisierung durch Rundung, Suchraum-Reduktion und für Gültigkeits-Tests. OIPs sind ganzzahligen Programmen ähnlich, so daß existierende algebraische Modellierungssprachen als Eingabeschnittstelle für einen Problemlöser benutzt werden können, der auf lokaler Suche basiert. Um die Performanz auf realistischen Problemen zu verbessern, ist WSAT(OIP) mit Strategien der Tabu-Suche ausgestattet. Wir führen eine experimentelle Untersuchung von WSAT(OIP) auf einer Reihe von realistischen ganzzahligen Constraint- und Optimierungsproblemen durch. Die Probleme stammen aus den Domänen Zeitplan-Erstellung, Sport-Ablaufplanung, Radar- Überwachung, Kurs-Zuteilung und Produktions-Planung. Das experimentelle Design untersucht Effizienz, Skalierung mit zunehmender Problemgröße und stärkeren Constraints sowie Robustheit. Die Ergebnisse zeigen, daß ganzzahlige lokale Suche bezüglich Performanz auf diesen Problemklassen zeitgemäße Ansätze der ganzzahligen Programmierung und der Constraint-Programmierung beim Finden nahe-optimaler Lösungen schlägt oder mit ihnen konkurriert. Kernergebnisse der empirischen Untersuchung sind, daß ganzzahlige lokale Suche in der Lage ist, schwierige Constraint-Probleme der Zeitplan-Erstellung und Sport-Ablaufplanung in einer 0-1 Repräsentation zu lösen, die außerhalb der Grenzen der ganzzahligen linearen Programmierung liegen, und für die die Entwicklung eines robustes Constraint-Programms eine nicht-triviale Aufgabe darstellt. Für mehrere realistische Optimierungsprobleme (ganzzahlig 0-1 und endliche Bereiche)zeigen wir, daß ganzzahlige lokale Suche eine günstige Skalierung der Laufzeit mit zunehmender Problemgröße und Constrainedness aufweist. Dadurch zeigt das Verfahren auf großen Problemen und auf Problemen mit starken Constraints deutlich bessere Performanz für das Finden nahe-Lösungen als die Branch-and-Bound Strategie der ganzzahligen Programmierung. Die untersuchten Probleme liegen zumeist außerhalb der Grenzen einer existierenden Simulated Annealing Strategie für allgemeine lineare 0-1 Programme