776 research outputs found
A Review of Formal Methods applied to Machine Learning
We review state-of-the-art formal methods applied to the emerging field of
the verification of machine learning systems. Formal methods can provide
rigorous correctness guarantees on hardware and software systems. Thanks to the
availability of mature tools, their use is well established in the industry,
and in particular to check safety-critical applications as they undergo a
stringent certification process. As machine learning is becoming more popular,
machine-learned components are now considered for inclusion in critical
systems. This raises the question of their safety and their verification. Yet,
established formal methods are limited to classic, i.e. non machine-learned
software. Applying formal methods to verify systems that include machine
learning has only been considered recently and poses novel challenges in
soundness, precision, and scalability.
We first recall established formal methods and their current use in an
exemplar safety-critical field, avionic software, with a focus on abstract
interpretation based techniques as they provide a high level of scalability.
This provides a golden standard and sets high expectations for machine learning
verification. We then provide a comprehensive and detailed review of the formal
methods developed so far for machine learning, highlighting their strengths and
limitations. The large majority of them verify trained neural networks and
employ either SMT, optimization, or abstract interpretation techniques. We also
discuss methods for support vector machines and decision tree ensembles, as
well as methods targeting training and data preparation, which are critical but
often neglected aspects of machine learning. Finally, we offer perspectives for
future research directions towards the formal verification of machine learning
systems
Data-Driven Abstraction-Based Control Synthesis
This paper studies formal synthesis of controllers for continuous-spacesystems with unknown dynamics to satisfy requirements expressed as lineartemporal logic formulas. Formal abstraction-based synthesis schemes rely on aprecise mathematical model of the system to build a finite abstract model,which is then used to design a controller. The abstraction-based schemes arenot applicable when the dynamics of the system are unknown. We propose adata-driven approach that computes the growth bound of the system using afinite number of trajectories. The growth bound together with the sampledtrajectories are then used to construct the abstraction and synthesise acontroller. Our approach casts the computation of the growth bound as a robust convexoptimisation program (RCP). Since the unknown dynamics appear in theoptimisation, we formulate a scenario convex program (SCP) corresponding to theRCP using a finite number of sampled trajectories. We establish a samplecomplexity result that gives a lower bound for the number of sampledtrajectories to guarantee the correctness of the growth bound computed from theSCP with a given confidence. We also provide a sample complexity result for thesatisfaction of the specification on the system in closed loop with thedesigned controller for a given confidence. Our results are founded onestimating a bound on the Lipschitz constant of the system and provideguarantees on satisfaction of both finite and infinite-horizon specifications.We show that our data-driven approach can be readily used as a model-freeabstraction refinement scheme by modifying the formulation of the growth boundand providing similar sample complexity results. The performance of ourapproach is shown on three case studies.<br
Formal Synthesis of Control Strategies for Positive Monotone Systems
We design controllers from formal specifications for positive discrete-time
monotone systems that are subject to bounded disturbances. Such systems are
widely used to model the dynamics of transportation and biological networks.
The specifications are described using signal temporal logic (STL), which can
express a broad range of temporal properties. We formulate the problem as a
mixed-integer linear program (MILP) and show that under the assumptions made in
this paper, which are not restrictive for traffic applications, the existence
of open-loop control policies is sufficient and almost necessary to ensure the
satisfaction of STL formulas. We establish a relation between satisfaction of
STL formulas in infinite time and set-invariance theories and provide an
efficient method to compute robust control invariant sets in high dimensions.
We also develop a robust model predictive framework to plan controls optimally
while ensuring the satisfaction of the specification. Illustrative examples and
a traffic management case study are included.Comment: To appear in IEEE Transactions on Automatic Control (TAC) (2018), 16
pages, double colum
Recommended from our members
Reachability Analysis of Cyber-Physical Systems Using Symbolic-Numeric Techniques
In this thesis, we address the problem of reachability analysis in cyber-physical systems. These are systems engineered by interfacing computational components with the physical world. They provide partially or fully automated safety-critical services in the form of medical devices, autonomous vehicles, avionics and power systems.
We propose techniques to reason about the reachability of such systems, and provide methods for falsifying their safety properties. We model the cyber component as a software program and the physical component as a hybrid dynamical system. Unlike model based analysis, which uses either a purely symbolic or a numerical approach, we argue in favor of using a combination of the two. We justify this by noting that the software program running on a computer is completely specified and has precise semantics. In contrast, the model of the physical system is only an approximation. Hence, we treat the former as a white box, but treat the latter as a black box. Using symbolic methods for the cyber components and numerical methods for hybrid systems, we carefully capture the complex behaviors of software programs and circumvent the difficulty in analyzing complex models developed through first principles. To combine the two techniques, we use a Counterexample Guided Abstraction Refinement (CEGAR) framework. Furthermore, we explore learning techniques like regression and piecewise affine modeling to estimate and represent black box hybrid dynamical systems for the purpose of falsification.
We use prototype implementations to demonstrate the effectiveness of presented ideas. Using non-trivial benchmarks, we compare their performance against the state of the art. We also comment on their applicability and discuss ideas for further improvement
- …