Certified Reinforcement Learning with Logic Guidance

This paper proposes the first model-free Reinforcement Learning (RL) framework to synthesise policies for unknown, and continuous-state Markov Decision Processes (MDPs), such that a given linear temporal property is satisfied. We convert the given property into a Limit Deterministic Buchi Automaton (LDBA), namely a finite-state machine expressing the property. Exploiting the structure of the LDBA, we shape a synchronous reward function on-the-fly, so that an RL algorithm can synthesise a policy resulting in traces that probabilistically satisfy the linear temporal property. This probability (certificate) is also calculated in parallel with policy learning when the state space of the MDP is finite: as such, the RL algorithm produces a policy that is certified with respect to the property. Under the assumption of finite state space, theoretical guarantees are provided on the convergence of the RL algorithm to an optimal policy, maximising the above probability. We also show that our method produces ''best available'' control policies when the logical property cannot be satisfied. In the general case of a continuous state space, we propose a neural network architecture for RL and we empirically show that the algorithm finds satisfying policies, if there exist such policies. The performance of the proposed framework is evaluated via a set of numerical examples and benchmarks, where we observe an improvement of one order of magnitude in the number of iterations required for the policy synthesis, compared to existing approaches whenever available.Comment: This article draws from arXiv:1801.08099, arXiv:1809.0782

Abstract Program Slicing: an Abstract Interpretation-based approach to Program Slicing

In the present paper we formally define the notion of abstract program slicing, a general form of program slicing where properties of data are considered instead of their exact value. This approach is applied to a language with numeric and reference values, and relies on the notion of abstract dependencies between program components (statements). The different forms of (backward) abstract slicing are added to an existing formal framework where traditional, non-abstract forms of slicing could be compared. The extended framework allows us to appreciate that abstract slicing is a generalization of traditional slicing, since traditional slicing (dealing with syntactic dependencies) is generalized by (semantic) non-abstract forms of slicing, which are actually equivalent to an abstract form where the identity abstraction is performed on data. Sound algorithms for computing abstract dependencies and a systematic characterization of program slices are provided, which rely on the notion of agreement between program states

Data Mining Earthquake Prediction with Multivariate Adaptive Regression Splines and Peak Ground Acceleration

Earthquake research has not yielded promising results because earthquakes have uncertain data parameters, and one of the methods to overcome the problem of uncertain parameters is the nonparametric method, namely Multivariate Adaptive Regression Splines (MARS). Sumbawa Island is part of the territory of Indonesia and is in the position of three active earth plates, so Sumbawa is prone to earthquake hazards. Therefore, this research is important to do. This study aimed to analyze earthquake hazard prediction on the island of Sumbawa by using the nonparametric MARS and Peak Ground Acceleration (PGA) methods to determine the risk of earthquake hazards. The method used in this study was MARS, which has two completed stages: Forward Stepwise and Backward Stepwise. The results of this study were based on testing and parameter analysis obtained a Mathematical model with 11 basis functions (BF) that contribute to the response variable, namely (BF) 1,2,3,4,5,7,9,11, and the basis functions do not contribute 6, 8, and 10. The predictor variables with the greatest influence were 100% Epicenter Distance and 73.8% Magnitude. The conclusion of this study is based on the highest PGA values in the areas most prone to earthquake hazards in Sumbawa, namely Mapin Kebak, Mapin Rea, Pulau Panjang, and Pulau Saringi

A Review of Formal Methods applied to Machine Learning

We review state-of-the-art formal methods applied to the emerging field of the verification of machine learning systems. Formal methods can provide rigorous correctness guarantees on hardware and software systems. Thanks to the availability of mature tools, their use is well established in the industry, and in particular to check safety-critical applications as they undergo a stringent certification process. As machine learning is becoming more popular, machine-learned components are now considered for inclusion in critical systems. This raises the question of their safety and their verification. Yet, established formal methods are limited to classic, i.e. non machine-learned software. Applying formal methods to verify systems that include machine learning has only been considered recently and poses novel challenges in soundness, precision, and scalability. We first recall established formal methods and their current use in an exemplar safety-critical field, avionic software, with a focus on abstract interpretation based techniques as they provide a high level of scalability. This provides a golden standard and sets high expectations for machine learning verification. We then provide a comprehensive and detailed review of the formal methods developed so far for machine learning, highlighting their strengths and limitations. The large majority of them verify trained neural networks and employ either SMT, optimization, or abstract interpretation techniques. We also discuss methods for support vector machines and decision tree ensembles, as well as methods targeting training and data preparation, which are critical but often neglected aspects of machine learning. Finally, we offer perspectives for future research directions towards the formal verification of machine learning systems

Proving termination through conditional termination

We present a constraint-based method for proving conditional termination of integer programs. Building on this, we construct a framework to prove (unconditional) program termination using a powerful mechanism to combine conditional termination proofs. Our key insight is that a conditional termination proof shows termination for a subset of program execution states which do not need to be considered in the remaining analysis. This facilitates more effective termination as well as non-termination analyses, and allows handling loops with different execution phases naturally. Moreover, our method can deal with sequences of loops compositionally. In an empirical evaluation, we show that our implementation VeryMax outperforms state-of-the-art tools on a range of standard benchmarks.Peer ReviewedPostprint (author's final draft