A Maximum Satisfiability Based Approach to Bi-Objective Boolean Optimization

Many real-world problem settings give rise to NP-hard combinatorial optimization problems. This results in a need for non-trivial algorithmic approaches for finding optimal solutions to such problems. Many such approaches—ranging from probabilistic and meta-heuristic algorithms to declarative programming—have been presented for optimization problems with a single objective. Less work has been done on approaches for optimization problems with multiple objectives. We present BiOptSat, an exact declarative approach for finding so-called Pareto-optimal solutions to bi-objective optimization problems. A bi-objective optimization problem arises for example when learning interpretable classifiers and the size, as well as the classification error of the classifier should be taken into account as objectives. Using propositional logic as a declarative programming language, we seek to extend the progress and success in maximum satisfiability (MaxSAT) solving to two objectives. BiOptSat can be viewed as an instantiation of the lexicographic method and makes use of a single SAT solver that is preserved throughout the entire search procedure. It allows for solving three tasks for bi-objective optimization: finding a single Pareto-optimal solution, finding one representative solution for each Pareto point, and enumerating all Pareto-optimal solutions. We provide an open-source implementation of five variants of BiOptSat, building on different algorithms proposed for MaxSAT. Additionally, we empirically evaluate these five variants, comparing their runtime performance to that of three key competing algorithmic approaches. The empirical comparison in the contexts of learning interpretable decision rules and bi-objective set covering shows practical benefits of our approach. Furthermore, for the best-performing variant of BiOptSat, we study the effects of proposed refinements to determine their effectiveness

MaxSAT-Based Bi-Objective Boolean Optimization

Peer reviewe

Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Feature Selection in Software Product Lines

Software design is a process of trading off competing objectives. If the user objective space is rich, then we should use optimizers that can fully exploit that richness. For example, this study configures software product lines (expressed as feature models) using various search-based software engineering methods. Our main result is that as we increase the number of optimization objectives, the methods in widespread use (e.g. NSGA-II, SPEA2) perform much worse than IBEA (Indicator-Based Evolutionary Algorithm). IBEA works best since it makes most use of user preference knowledge. Hence it does better on the standard measures (hypervolume and spread) but it also generates far more products with 0 violations of domain constraints. We also present significant improvements to IBEA\u27s performance by employing three strong heuristic techniques that we call PUSH, PULL, and seeding. The PUSH technique forces the evolutionary search to respect certain rules and dependencies defined by the feature models, while the PULL technique gives higher weight to constraint satisfaction as an optimization objective and thus achieves a higher percentage of fully-compliant configurations within shorter runtimes. The seeding technique helps in guiding very large feature models to correct configurations very early in the optimization process. Our conclusion is that the methods we apply in search-based software engineering need to be carefully chosen, particularly when studying complex decision spaces with many optimization objectives. Also, we conclude that search methods must be customized to fit the problem at hand. Specifically, the evolutionary search must respect domain constraints

Model-based symbolic design space exploration at the electronic system level: a systematic approach

In this thesis, a novel, fully systematic approach is proposed that addresses the automated design space exploration at the electronic system level. The problem is formulated as multi-objective optimization problem and is encoded symbolically using Answer Set Programming (ASP). Several specialized solvers are tightly coupled as background theories with the foreground ASP solver under the ASP modulo Theories (ASPmT) paradigm. By utilizing the ASPmT paradigm, the search is executed entirely systematically and the disparate synthesis steps can be coupled to explore the search space effectively.In dieser Arbeit wird ein vollständig systematischer Ansatz präsentiert, der sich mit der Entwurfsraumexploration auf der elektronischen Systemebene befasst. Das Problem wird als multikriterielles Optimierungsproblem formuliert und symbolisch mit Hilfe von Answer Set Programming (ASP) kodiert. Spezialisierte Solver sind im Rahmen des ASP modulo Theories (ASPmT) Paradigmas als Hintergrundtheorien eng mit dem ASP Solver gekoppelt. Durch die Verwendung von ASPmT wird die Suche systematisch ausgeführt und die individuellen Schritte können gekoppelt werden, um den Suchraum effektiv zu durchsuchen

Branching strategies for mixed-integer programs containing logical constraints and decomposable structure

Decision-making optimisation problems can include discrete selections, e.g. selecting a route, arranging non-overlapping items or designing a network of items. Branch-and-bound (B&B), a widely applied divide-and-conquer framework, often solves such problems by considering a continuous approximation, e.g. replacing discrete variable domains by a continuous superset. Such approximations weaken the logical relations, e.g. for discrete variables corresponding to Boolean variables. Branching in B&B reintroduces logical relations by dividing the search space. This thesis studies designing B&B branching strategies, i.e. how to divide the search space, for optimisation problems that contain both a logical and a continuous structure. We begin our study with a large-scale, industrially-relevant optimisation problem where the objective consists of machine-learnt gradient-boosted trees (GBTs) and convex penalty functions. GBT functions contain if-then queries which introduces a logical structure to this problem. We propose decomposition-based rigorous bounding strategies and an iterative heuristic that can be embedded into a B&B algorithm. We approach branching with two strategies: a pseudocost initialisation and strong branching that target the structure of GBT and convex penalty aspects of the optimisation objective, respectively. Computational tests show that our B&B approach outperforms state-of-the-art solvers in deriving rigorous bounds on optimality. Our second project investigates how satisfiability modulo theories (SMT) derived unsatisfiable cores may be utilised in a B&B context. Unsatisfiable cores are subsets of constraints that explain an infeasible result. We study two-dimensional bin packing (2BP) and develop a B&B algorithm that branches on SMT unsatisfiable cores. We use the unsatisfiable cores to derive cuts that break 2BP symmetries. Computational results show that our B&B algorithm solves 20% more instances when compared with commercial solvers on the tested instances. Finally, we study convex generalized disjunctive programming (GDP), a framework that supports logical variables and operators. Convex GDP includes disjunctions of mathematical constraints, which motivate branching by partitioning the disjunctions. We investigate separation by branching, i.e. eliminating solutions that prevent rigorous bound improvement, and propose a greedy algorithm for building the branches. We propose three scoring methods for selecting the next branching disjunction. We also analyse how to leverage infeasibility to expedite the B&B search. Computational results show that our scoring methods can reduce the number of explored B&B nodes by an order of magnitude when compared with scoring methods proposed in literature. Our infeasibility analysis further reduces the number of explored nodes.Open Acces

Automatic Algorithm Design for Hybrid Flowshop Scheduling Problems

[EN] Industrial production scheduling problems are challenges that researchers have been trying to solve for decades. Many practical scheduling problems such as the hybrid flowshop are ATP-hard. As a result, researchers resort to metaheuristics to obtain effective and efficient solutions. The traditional design process of metaheuristics is mainly manual, often metaphor-based, biased by previous experience and prone to producing overly tailored methods that only work well on the tested problems and objectives. In this paper, we use an Automatic Algorithm Design (AAD) methodology to eliminate these limitations. AAD is capable of composing algorithms from components with minimal human intervention. We test the proposed MD for three different optimization objectives in the hybrid flowshop. Comprehensive computational and statistical testing demonstrates that automatically designed algorithms outperform specifically tailored state-of-the-art methods for the tested objectives in most cases.Pedro Alfaro-Fernandez and Ruben Ruiz are partially supported by the Spanish Ministry of Science, Innovation, and Universities, under the project "OPTEP-Port Terminal Operations Optimization" (No. RTI2018-094940-B-I00) financed with FEDER funds and under grants BES-2013-064858 and EEBB-I-15-10089. This work was supported by the COMEX project (P7/36) within the Interuniversity Attraction Poles Programme of the Belgian Science Policy Office. Thomas Stiitzle acknowledges support from the Belgian F.R.S.-FNRS, of which he is a Research Director.Alfaro-Fernandez, P.; Ruiz García, R.; Pagnozzi, F.; Stützle, T. (2020). Automatic Algorithm Design for Hybrid Flowshop Scheduling Problems. European Journal of Operational Research. 282(3):835-845. https://doi.org/10.1016/j.ejor.2019.10.004S8358452823Bożejko, W., Gnatowski, A., Niżyński, T., Affenzeller, M., & Beham, A. (2018). Local Optima Networks in Solving Algorithm Selection Problem for TSP. Advances in Intelligent Systems and Computing, 83-93. doi:10.1007/978-3-319-91446-6_9Bożejko, W., Pempera, J., & Smutnicki, C. (2013). Parallel tabu search algorithm for the hybrid flow shop problem. Computers & Industrial Engineering, 65(3), 466-474. doi:10.1016/j.cie.2013.04.007Burke, E. K., Hyde, M. R., & Kendall, G. (2012). Grammatical Evolution of Local Search Heuristics. IEEE Transactions on Evolutionary Computation, 16(3), 406-417. doi:10.1109/tevc.2011.2160401Cahon, S., Melab, N., & Talbi, E.-G. (2004). ParadisEO: A Framework for the Reusable Design of Parallel and Distributed Metaheuristics. Journal of Heuristics, 10(3), 357-380. doi:10.1023/b:heur.0000026900.92269.ecCarlier, J., & Neron, E. (2000). An Exact Method for Solving the Multi-Processor Flow-Shop. RAIRO - Operations Research, 34(1), 1-25. doi:10.1051/ro:2000103Chung, T.-P., & Liao, C.-J. (2013). An immunoglobulin-based artificial immune system for solving the hybrid flow shop problem. Applied Soft Computing, 13(8), 3729-3736. doi:10.1016/j.asoc.2013.03.006Cui, Z., & Gu, X. (2015). An improved discrete artificial bee colony algorithm to minimize the makespan on hybrid flow shop problems. Neurocomputing, 148, 248-259. doi:10.1016/j.neucom.2013.07.056Ding, J.-Y., Song, S., Gupta, J. N. D., Zhang, R., Chiong, R., & Wu, C. (2015). An improved iterated greedy algorithm with a Tabu-based reconstruction strategy for the no-wait flowshop scheduling problem. Applied Soft Computing, 30, 604-613. doi:10.1016/j.asoc.2015.02.006Dubois-Lacoste, J., López-Ibáñez, M., & Stützle, T. (2011). A hybrid TP+PLS algorithm for bi-objective flow-shop scheduling problems. Computers & Operations Research, 38(8), 1219-1236. doi:10.1016/j.cor.2010.10.008Dubois-Lacoste, J., Pagnozzi, F., & Stützle, T. (2017). An iterated greedy algorithm with optimization of partial solutions for the makespan permutation flowshop problem. Computers & Operations Research, 81, 160-166. doi:10.1016/j.cor.2016.12.021Gupta, J. N. D. (1988). Two-Stage, Hybrid Flowshop Scheduling Problem. Journal of the Operational Research Society, 39(4), 359-364. doi:10.1057/jors.1988.63Gupta, J. N. D., & Stafford, E. F. (2006). Flowshop scheduling research after five decades. European Journal of Operational Research, 169(3), 699-711. doi:10.1016/j.ejor.2005.02.001Hidri, L., & Haouari, M. (2011). Bounding strategies for the hybrid flow shop scheduling problem. Applied Mathematics and Computation, 217(21), 8248-8263. doi:10.1016/j.amc.2011.02.108Hutter, F., Hoos, H. H., Leyton-Brown, K., & Stuetzle, T. (2009). ParamILS: An Automatic Algorithm Configuration Framework. Journal of Artificial Intelligence Research, 36, 267-306. doi:10.1613/jair.2861Johnson, S. M. (1954). Optimal two- and three-stage production schedules with setup times included. Naval Research Logistics Quarterly, 1(1), 61-68. doi:10.1002/nav.3800010110Khalouli, S., Ghedjati, F., & Hamzaoui, A. (2010). A meta-heuristic approach to solve a JIT scheduling problem in hybrid flow shop. Engineering Applications of Artificial Intelligence, 23(5), 765-771. doi:10.1016/j.engappai.2010.01.008KhudaBukhsh, A. R., Xu, L., Hoos, H. H., & Leyton-Brown, K. (2016). SATenstein: Automatically building local search SAT solvers from components. Artificial Intelligence, 232, 20-42. doi:10.1016/j.artint.2015.11.002Li, J., Pan, Q., & Wang, F. (2014). A hybrid variable neighborhood search for solving the hybrid flow shop scheduling problem. Applied Soft Computing, 24, 63-77. doi:10.1016/j.asoc.2014.07.005Liao, C.-J., Tjandradjaja, E., & Chung, T.-P. (2012). An approach using particle swarm optimization and bottleneck heuristic to solve hybrid flow shop scheduling problem. Applied Soft Computing, 12(6), 1755-1764. doi:10.1016/j.asoc.2012.01.011Lopez-Ibanez, M., & Stutzle, T. (2012). The Automatic Design of Multiobjective Ant Colony Optimization Algorithms. IEEE Transactions on Evolutionary Computation, 16(6), 861-875. doi:10.1109/tevc.2011.2182651López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016). The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives, 3, 43-58. doi:10.1016/j.orp.2016.09.002Marichelvam, M. K., Prabaharan, T., & Yang, X. S. (2014). A Discrete Firefly Algorithm for the Multi-Objective Hybrid Flowshop Scheduling Problems. IEEE Transactions on Evolutionary Computation, 18(2), 301-305. doi:10.1109/tevc.2013.2240304Marichelvam, M. K., Prabaharan, T., & Yang, X. S. (2014). Improved cuckoo search algorithm for hybrid flow shop scheduling problems to minimize makespan. Applied Soft Computing, 19, 93-101. doi:10.1016/j.asoc.2014.02.005Marichelvam, M. K., Prabaharan, T., Yang, X. S., & Geetha, M. (2013). Solving hybrid flow shop scheduling problems using bat algorithm. International Journal of Logistics Economics and Globalisation, 5(1), 15. doi:10.1504/ijleg.2013.054428Mascia, F., López-Ibáñez, M., Dubois-Lacoste, J., & Stützle, T. (2014). Grammar-based generation of stochastic local search heuristics through automatic algorithm configuration tools. Computers & Operations Research, 51, 190-199. doi:10.1016/j.cor.2014.05.020Nawaz, M., Enscore, E. E., & Ham, I. (1983). A heuristic algorithm for the m-machine, n-job flow-shop sequencing problem. Omega, 11(1), 91-95. doi:10.1016/0305-0483(83)90088-9Pan, Q.-K., & Dong, Y. (2014). An improved migrating birds optimisation for a hybrid flowshop scheduling with total flowtime minimisation. Information Sciences, 277, 643-655. doi:10.1016/j.ins.2014.02.152Pan, Q.-K., Ruiz, R., & Alfaro-Fernández, P. (2017). Iterated search methods for earliness and tardiness minimization in hybrid flowshops with due windows. Computers & Operations Research, 80, 50-60. doi:10.1016/j.cor.2016.11.022Pan, Q.-K., Wang, L., Li, J.-Q., & Duan, J.-H. (2014). A novel discrete artificial bee colony algorithm for the hybrid flowshop scheduling problem with makespan minimisation. Omega, 45, 42-56. doi:10.1016/j.omega.2013.12.004Rajendran, C., & Ziegler, H. (1997). An efficient heuristic for scheduling in a flowshop to minimize total weighted flowtime of jobs. European Journal of Operational Research, 103(1), 129-138. doi:10.1016/s0377-2217(96)00273-1Ruiz, R., & Stützle, T. (2007). A simple and effective iterated greedy algorithm for the permutation flowshop scheduling problem. European Journal of Operational Research, 177(3), 2033-2049. doi:10.1016/j.ejor.2005.12.009Ruiz, R., & Vázquez-Rodríguez, J. A. (2010). The hybrid flow shop scheduling problem. European Journal of Operational Research, 205(1), 1-18. doi:10.1016/j.ejor.2009.09.024Sörensen, K. (2013). Metaheuristics-the metaphor exposed. International Transactions in Operational Research, 22(1), 3-18. doi:10.1111/itor.12001Vignier, A., Billaut, J.-C., & Proust, C. (1999). Les problèmes d’ordonnancement de type flow-shop hybride : état de l’art. RAIRO - Operations Research, 33(2), 117-183. doi:10.1051/ro:1999108Wang, S., Wang, L., Liu, M., & Xu, Y. (2013). An enhanced estimation of distribution algorithm for solving hybrid flow-shop scheduling problem with identical parallel machines. The International Journal of Advanced Manufacturing Technology, 68(9-12), 2043-2056. doi:10.1007/s00170-013-4819-yXu, Y., Wang, L., Wang, S., & Liu, M. (2013). An effective shuffled frog-leaping algorithm for solving the hybrid flow-shop scheduling problem with identical parallel machines. Engineering Optimization, 45(12), 1409-1430. doi:10.1080/0305215x.2012.73778

MATEDA: A suite of EDA programs in Matlab

This paper describes MATEDA-2.0, a suite of programs in Matlab for estimation of distribution algorithms. The package allows the optimization of single and multi-objective problems with estimation of distribution algorithms (EDAs) based on undirected graphical models and Bayesian networks. The implementation is conceived for allowing the incorporation by the user of different combinations of selection, learning, sampling, and local search procedures. Other included methods allow the analysis of the structures learned by the probabilistic models, the visualization of particular features of these structures and the use of the probabilistic models as fitness modeling tools