A portfolio-based analysis method for competition results

Nguyen Dang: is a Leverhulme Early Career Fellow.Competitions such as the MiniZinc Challenges or the SAT competitions have been very useful sources for comparing performance of different solving approaches and for advancing the state-of-the-arts of the fields. Traditional competition setting often focuses on producing a ranking between solvers based on their average performance across a wide range of benchmark problems and instances. While this is a sensible way to assess the relative performance of solvers, such ranking does not necessarily reflect the full potential of a solver, especially when we want to utilise a portfolio of solvers instead of a single one for solving a new problem. In this paper, I will describe a portfolio-based analysis method which can give complementary insights into the performance of participating solvers in a competition. The method is demonstrated on the results of the MiniZinc Challenges and new insights gained from the portfolio viewpoint are presented.Publisher PD

A Comprehensive Study of k-Portfolios of Recent SAT Solvers

These are the slides for the paper "A Comprehensive Study of k-Portfolios of Recent SAT Solvers", presented at the conference [*SAT 2022*](http://satisfiability.org/SAT22/). You can find the paper [here](https://www.doi.org/10.4230/LIPIcs.SAT.2022.2)

Learning to Optimize Computational Resources: Frugal Training with Generalization Guarantees

Algorithms typically come with tunable parameters that have a considerable impact on the computational resources they consume. Too often, practitioners must hand-tune the parameters, a tedious and error-prone task. A recent line of research provides algorithms that return nearly-optimal parameters from within a finite set. These algorithms can be used when the parameter space is infinite by providing as input a random sample of parameters. This data-independent discretization, however, might miss pockets of nearly-optimal parameters: prior research has presented scenarios where the only viable parameters lie within an arbitrarily small region. We provide an algorithm that learns a finite set of promising parameters from within an infinite set. Our algorithm can help compile a configuration portfolio, or it can be used to select the input to a configuration algorithm for finite parameter spaces. Our approach applies to any configuration problem that satisfies a simple yet ubiquitous structure: the algorithm's performance is a piecewise constant function of its parameters. Prior research has exhibited this structure in domains from integer programming to clustering

Shapley value: its algorithms and application to supply chains

Introduction− Coalitional game theorists have studied the coalition struc-ture and the payoff schemes attributed to such coalition. With respect to the payoff value, there are number ways of obtaining to “best” distribution of the value of the game. The solution concept or payoff value distribution that is canonically held to fairly dividing a coalition’s value is called the Shapley Value. It is probably the most important regulatory payoff scheme in coali-tion games. The reason the Shapley value has been the focus of so much interest is that it represents a distinct approach to the problems of complex strategic interaction that game theory tries to solve. Objective−This study aims to do a brief literature review of the application of Shapley Value for solving problems in different cooperation fields and the importance of studying existing methods to facilitate their calculation. This review is focused on the algorithmic view of cooperative game theory with a special emphasis on supply chains. Additionally, an algorithm for the calcu-lation of the Shapley Value is proposed and numerical examples are used in order to validate the proposed algorithm. Methodology−First of all, the algorithms used to calculate Shapley value were identified. The element forming a supply chain were also identified. The cooperation between the members of the supply chain ways is simulated and the Shapley Value is calculated using the proposed algorithm in order to check its applicability. Results and Conclusions− The algorithmic approach introduced in this paper does not wish to belittle the contributions made so far but intends to provide a straightforward solution for decision problems that involve supply chains. An efficient and feasible way of calculating the Shapley Value when player structures are known beforehand provides the advantage of reducing the amount of effort in calculating all possible coalition structures prior to the Shapley.Introducción: Los teóricos del juego cooperativos han estudiado la estructura de coalición y los esquemas de pago atribuidos a esas coaliciones. En relación al valor del pago, hay varias maneras de obtener la “mejor” distribución del valor del juego. El concepto de solución o la distribución del valor de recompensa que se mantiene canónicamente para dividir justamente el valor de una coalición se llama Valor de Shapley. Es probablemente el esquema de pago más importante en los juegos cooperativos. La razón por la cual el valor de Shapley ha sido el foco de tanto interés es que representa un acercamiento distinto a los problemas de la interacción estratégica compleja que la teoría del juego intenta resolver.Objetivo: Este estudio tiene como objetivo hacer una breve revisión bibliográfica de la aplicación del Valor de Shapley para resolver problemas en diferentes campos de cooperación y la importancia de estudiar los métodos existentes para facilitar su cálculo. Esta revisión se centra en la visión algorítmica de la teoría cooperativa de juegos con un énfasis especial en las cadenas de suministro. Adicionalmente se propone un algoritmo para el cálculo del Valor de Shapley y se utilizan ejemplos numéricos para validar el algoritmo propuesto.Metodología: En primer lugar, se identificaron los algoritmos utilizados para calcular el valor de Shapley. También se identificó los elementos que forman una cadena de suministro. Luego se simula la cooperación entre los miembros de las vías de la cadena de suministro y se calcula el valor de Shapley utilizando el algoritmo propuesto para comprobar su aplicabilidad.Resultados y Conclusiones: El enfoque algorítmico introducido en este documento no pretende menospreciar las contribuciones hechas hasta ahora, pero tiene la intención de proporcionar una solución directa para problemas de decisión que involucran cadenas de suministro. Una manera eficiente y factible de calcular el valor de Shapley cuando las estructuras de jugador se conocen de antemano proporciona la ventaja de reducir la cantidad de esfuerzo en el cálculo de todas las estructuras de coalición posibles antes del Shapley

On the Evaluation of (Meta-)solver Approaches

Meta-solver approaches exploit many individual solvers to potentially build a better solver. To assess the performance of meta-solvers, one can adopt the metrics typically used for individual solvers (e.g., runtime or solution quality) or employ more specific evaluation metrics (e.g., by measuring how close the meta-solver gets to its virtual best performance). In this paper, based on some recently published works, we provide an overview of different performance metrics for evaluating (meta-)solvers by exposing their strengths and weaknesses

El valor de Shapley: sus algoritmos y aplicación en cadenas de suministro

Introduction: Coalitional game theorists have studied the coalition structure and the payoff schemes attributed to such coalition. With respect to the payoff value, there are number ways of obtaining to “best” distribution of the value of the game. The solution concept or payoff value distribution that is canonically held to fairly dividing a coalition’s value is called the Shapley Value. It is probably the most important regulatory payoff scheme in coalition games. The reason the Shapley value has been the focus of so much interest is that it represents a distinct approach to the problems of complex strategic interaction that game theory tries to solve.Objective: This study aims to do a brief literature review of the application of Shapley Value for solving problems in different cooperation fields and the importance of studying existing methods to facilitate their calculation. This review is focused on the algorithmic view of cooperative game theory with a special emphasis on supply chains. Additionally, an algorithm for the calculation of the Shapley Value is proposed and numerical examples are used in order to validate the proposed algorithm.Methodology: First of all, the algorithms used to calculate Shapley value were identified. The element forming a supply chain were also identified. The cooperation between the members of the supply chain ways is simulated and the Shapley Value is calculated using the proposed algorithm in order to check its applicability.Results and Conclusions: The algorithmic approach introduced in this paper does not wish to belittle the contributions made so far but intends to provide a straightforward solution for decision problems that involve supply chains. An efficient and feasible way of calculating the Shapley Value when player structures are known beforehand provides the advantage of reducing the amount of effort in calculating all possible coalition structures prior to the Shapley.Introducción: Los teóricos del juego cooperativos han estudiado la estructura de coalición y los esquemas de pago atribuidos a esas coaliciones. En relación al valor del pago, hay varias maneras de obtener la “mejor” distribución del valor del juego. El concepto de solución o la distribución del valor de recompensa que se mantiene canónicamente para dividir justamente el valor de una coalición se llama Valor de Shapley. Es probablemente el esquema de pago más importante en los juegos cooperativos. La razón por la cual el valor de Shapley ha sido el foco de tanto interés es que representa un acercamiento distinto a los problemas de la interacción estratégica compleja que la teoría del juego intenta resolver.Objetivo: Este estudio tiene como objetivo hacer una breve revisión bibliográfica de la aplicación del Valor de Shapley para resolver problemas en diferentes campos de cooperación y la importancia de estudiar los métodos existentes para facilitar su cálculo. Esta revisión se centra en la visión algorítmica de la teoría cooperativa de juegos con un énfasis especial en las cadenas de suministro. Adicionalmente se propone un algoritmo para el cálculo del Valor de Shapley y se utilizan ejemplos numéricos para validar el algoritmo propuesto.Metodología: En primer lugar, se identificaron los algoritmos utilizados para calcular el valor de Shapley. También se identificó los elementos que forman una cadena de suministro. Luego se simula la cooperación entre los miembros de las vías de la cadena de suministro y se calcula el valor de Shapley utilizando el algoritmo propuesto para comprobar su aplicabilidad.Resultados y Conclusiones: El enfoque algorítmico introducido en este documento no pretende menospreciar las contribuciones hechas hasta ahora, pero tiene la intención de proporcionar una solución directa para problemas de decisión que involucran cadenas de suministro. Una manera eficiente y factible de calcular el valor de Shapley cuando las estructuras de jugador se conocen de antemano proporciona la ventaja de reducir la cantidad de esfuerzo en el cálculo de todas las estructuras de coalición posibles antes del Shapley.

A Game-Theoretic Approach to Share the Costs of Cooperating Healthcare Networks

The growing demand for healthcare services combined with the disarray in the health insurance market in the United States has created a situation where rival health networks are aggressively competing by building duplicate health facilities and providing redundant services in localized geographic regions. Unfortunately, this strategy can reduce the quality of patient care and decrease profits (Kaissi & Charland, 2013). A solution to this problem is for health networks to cooperate and share healthcare equipment, facilities, and personnel. Toward that end, this paper presents a game-theoretic method that can share these costs in a fair, efficient, and repeatable manner

A Comprehensive Study of k-Portfolios of Recent SAT Solvers

Hard combinatorial problems such as propositional satisfiability are ubiquitous. The holy grail are solution methods that show good performance on all problem instances. However, new approaches emerge regularly, some of which are complementary to existing solvers in that they only run faster on some instances but not on many others. While portfolios, i.e., sets of solvers, have been touted as useful, putting together such portfolios also needs to be efficient. In particular, it remains an open question how well portfolios can exploit the complementarity of solvers. This paper features a comprehensive analysis of portfolios of recent SAT solvers, the ones from the SAT Competitions 2020 and 2021. We determine optimal portfolios with exact and approximate approaches and study the impact of portfolio size k on performance. We also investigate how effective off-the-shelf prediction models are for instance-specific solver recommendations. One result is that the portfolios found with an approximate approach are as good as the optimal solution in practice. We also observe that marginal returns decrease very quickly with larger k, and our prediction models do not give way to better performance beyond very small portfolio sizes

Towards Dynamic Algorithm Selection for Numerical Black-Box Optimization: Investigating BBOB as a Use Case

One of the most challenging problems in evolutionary computation is to select from its family of diverse solvers one that performs well on a given problem. This algorithm selection problem is complicated by the fact that different phases of the optimization process require different search behavior. While this can partly be controlled by the algorithm itself, there exist large differences between algorithm performance. It can therefore be beneficial to swap the configuration or even the entire algorithm during the run. Long deemed impractical, recent advances in Machine Learning and in exploratory landscape analysis give hope that this dynamic algorithm configuration~(dynAC) can eventually be solved by automatically trained configuration schedules. With this work we aim at promoting research on dynAC, by introducing a simpler variant that focuses only on switching between different algorithms, not configurations. Using the rich data from the Black Box Optimization Benchmark~(BBOB) platform, we show that even single-switch dynamic Algorithm selection (dynAS) can potentially result in significant performance gains. We also discuss key challenges in dynAS, and argue that the BBOB-framework can become a useful tool in overcoming these