A Backward Algorithm for the Multiprocessor Online Feasibility of Sporadic Tasks

The online feasibility problem (for a set of sporadic tasks) asks whether there is a scheduler that always prevents deadline misses (if any), whatever the sequence of job releases, which is a priori} unknown to the scheduler. In the multiprocessor setting, this problem is notoriously difficult. The only exact test for this problem has been proposed by Bonifaci and Marchetti-Spaccamela: it consists in modelling all the possible behaviours of the scheduler and of the tasks as a graph; and to interpret this graph as a game between the tasks and the scheduler, which are seen as antagonistic players. Then, computing a correct scheduler is equivalent to finding a winning strategy for the `scheduler player', whose objective in the game is to avoid deadline misses. In practice, however this approach is limited by the intractable size of the graph. In this work, we consider the classical attractor algorithm to solve such games, and introduce antichain techniques to optimise its performance in practice and overcome the huge size of the game graph. These techniques are inspired from results from the formal methods community, and exploit the specific structure of the feasibility problem. We demonstrate empirically that our approach allows to dramatically improve the performance of the game solving algorithm.Comment: Long version of a conference paper accepted to ACSD 201

Certified Reinforcement Learning with Logic Guidance

This paper proposes the first model-free Reinforcement Learning (RL) framework to synthesise policies for unknown, and continuous-state Markov Decision Processes (MDPs), such that a given linear temporal property is satisfied. We convert the given property into a Limit Deterministic Buchi Automaton (LDBA), namely a finite-state machine expressing the property. Exploiting the structure of the LDBA, we shape a synchronous reward function on-the-fly, so that an RL algorithm can synthesise a policy resulting in traces that probabilistically satisfy the linear temporal property. This probability (certificate) is also calculated in parallel with policy learning when the state space of the MDP is finite: as such, the RL algorithm produces a policy that is certified with respect to the property. Under the assumption of finite state space, theoretical guarantees are provided on the convergence of the RL algorithm to an optimal policy, maximising the above probability. We also show that our method produces ''best available'' control policies when the logical property cannot be satisfied. In the general case of a continuous state space, we propose a neural network architecture for RL and we empirically show that the algorithm finds satisfying policies, if there exist such policies. The performance of the proposed framework is evaluated via a set of numerical examples and benchmarks, where we observe an improvement of one order of magnitude in the number of iterations required for the policy synthesis, compared to existing approaches whenever available.Comment: This article draws from arXiv:1801.08099, arXiv:1809.0782

Fully Observable Non-deterministic Planning as Assumption-Based Reactive Synthesis

We contribute to recent efforts in relating two approaches to automatic synthesis, namely, automated planning and discrete reactive synthesis. First, we develop a declarative characterization of the standard “fairness” assumption on environments in non-deterministic planning, and show that strong-cyclic plans are correct solution concepts for fair environments. This complements, and arguably completes, the existing foundational work on non-deterministic planning, which focuses on characterizing (and computing) plans enjoying special “structural” properties, namely loopy but closed policy structures. Second, we provide an encoding suitable for reactive synthesis that avoids the naive exponential state space blowup. To do so, special care has to be taken to specify the fairness assumption on the environment in a succinct manner.Fil: D'ippolito, Nicolás Roque. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Investigación en Ciencias de la Computación. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Investigación en Ciencias de la Computación; ArgentinaFil: Rodriguez, Natalia. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación; ArgentinaFil: Sardina, Sebastian. RMIT University; Australi

Skolem Functions for Factored Formulas

Given a propositional formula F(x,y), a Skolem function for x is a function \Psi(y), such that substituting \Psi(y) for x in F gives a formula semantically equivalent to \exists F. Automatically generating Skolem functions is of significant interest in several applications including certified QBF solving, finding strategies of players in games, synthesising circuits and bit-vector programs from specifications, disjunctive decomposition of sequential circuits etc. In many such applications, F is given as a conjunction of factors, each of which depends on a small subset of variables. Existing algorithms for Skolem function generation ignore any such factored form and treat F as a monolithic function. This presents scalability hurdles in medium to large problem instances. In this paper, we argue that exploiting the factored form of F can give significant performance improvements in practice when computing Skolem functions. We present a new CEGAR style algorithm for generating Skolem functions from factored propositional formulas. In contrast to earlier work, our algorithm neither requires a proof of QBF satisfiability nor uses composition of monolithic conjunctions of factors. We show experimentally that our algorithm generates smaller Skolem functions and outperforms state-of-the-art approaches on several large benchmarks.Comment: Full version of FMCAD 2015 conference publicatio

Safe and Optimal Scheduling for Hard and Soft Tasks

We consider a stochastic scheduling problem with both hard and soft tasks on a single machine. Each task is described by a discrete probability distribution over possible execution times, and possible inter-arrival times of the job, and a fixed deadline. Soft tasks also carry a penalty cost to be paid when they miss a deadline. We ask to compute an online and non-clairvoyant scheduler (i.e. one that must take decisions without knowing the future evolution of the system) that is safe and efficient. Safety imposes that deadline of hard tasks are never violated while efficient means that we want to minimise the mean cost of missing deadlines by soft tasks. First, we show that the dynamics of such a system can be modelled as a finite Markov Decision Process (MDP). Second, we show that our scheduling problem is PP-hard and in EXPTime. Third, we report on a prototype tool that solves our scheduling problem by relying on the Storm tool to analyse the corresponding MDP. We show how antichain techniques can be used as a potential heuristic

The Theory of Universal Graphs for Infinite Duration Games

We introduce the notion of universal graphs as a tool for constructing algorithms solving games of infinite duration such as parity games and mean payoff games. In the first part we develop the theory of universal graphs, with two goals: showing an equivalence and normalisation result between different recently introduced related models, and constructing generic value iteration algorithms for any positionally determined objective. In the second part we give four applications: to parity games, to mean payoff games, and to combinations of them (in the form of disjunctions of objectives). For each of these four cases we construct algorithms achieving or improving over the best known time and space complexity.Comment: 43 pages, 10 figure

Natural Strategic Ability

International audienc

Improving parity games in practice

Parity games are infinite-round two-player games played on directed graphs whose nodes are labeled with priorities. The winner of a play is determined by the smallest priority (even or odd) that is encountered infinitely often along the play. In the last two decades, several algorithms for solving parity games have been proposed and implemented in PGSolver, a platform written in OCaml. PGSolver includes the Zielonka’s recursive algorithm (RE, for short) which is known to be the best performing one over random games. Notably, several attempts have been carried out with the aim of improving the performance of RE in PGSolver, but with small advances in practice. In this work, we deeply revisit the implementation of RE by dealing with the use of specific data structures and programming languages such as Scala, Java, C++, and Go. Our empirical evaluation shows that these choices are successful, gaining up to three orders of magnitude in running time over the classic version of the algorithm implemented in PGSolver