Bicknell, Brett

Butler, Michael

Colley, John

Reis, Jose

Snook, Colin

English

Southampton (e-Prints Soton)

A Practical Approach for Closed SystemsFormal Verication Using Event-BBrett Bicknell1, Jose Reis1, Michael Butler2, John Colley2, and Colin Snook21 Critical Software Technologies Ltd, 2 Venture Road, Southampton Science Park,Southampton, SO16 7NP, United Kingdomjreis@critical-software.co.uk,WWW home page: http://www.critical-software.co.uk2 Universitity of Southampton, Electronics and Computer Science, SO17 1BJ, UnitedKingdomAbstract. Assurance of high integrity systems based on closed systemsis a challenge that becomes diﬃcult to overcome when a classical testingapproach is used; in particular the evidence generated from a classi-cal testing approach may not meet the objectives of rigorous standards.This paper presents a new approach for the formal veriﬁcation of closedsystems, in particular commercial oﬀ the shelf (COTS) products. Theapproach brings together the formal language Event-B, mathematicalproof theory and the Rodin toolset and provides the mechanism for cre-ating abstract models of closed systems and to then verify these systemproperties against operational requirements. From an industrial perspec-tive this approach represents a step change in the use and successfulintegration of closed systems; using formal methods to guarantee theirintegration and functionality. The outcome of the proof of concept willprovide a solution that will increase the level of conﬁdence on complexsystem of system solutions containing closed systems. Moreover, it willsupport the production of safety-cases by providing formal proofs of asystem’s correctness.Keywords: closed systems, COTS, Event-B, Rodin, formal veriﬁcation,virtualisation, VMWare1 ContextClosed Systems (CS) are becoming more common in the safety critical domainand in particular within the military domain. As the name connotes a closedsystem is a solution which is available as a black box where access to internalmodules and design artifacts is not possible. In the context of this paper we willrefer to a closed system as a software based system and of which can refer toeither of the following contexts:1. Systems developed by a third party that are commercially available to thepublic domain as readymade solutions; also known as COTS2. Systems developed by a third party that are not commercially available butdue to legal, export controls or other commercial reasons are not open tothe system integratorThe justiﬁcations provided by system integrators for adopting a CS productas opposed to a bespoke solution are typically:{ Adoption of state of the art technology : The supplier of a CS has thebudget to invest in R&D and innovative methods to provide state of the arttechnology. Furthermore these suppliers tend to invest in and continuouslydevelop the product to stay competitive for longer{ Cost reduction : It is debatable whether a CS product oﬀers a signiﬁcant costreduction or not, because the CS product may not be integrated seamlesslywith the complete end system. If instead there are no signiﬁcant issues withthe integration of the CS, the end cost will be less than if the same solutionwas developed from scratch. Another key factor is that having the producton an open market pushes suppliers to oﬀer competitive prices{ Reliability : In principle CS products have been tested thoroughly by thesupplier, in that the solution is deemed reliable in particular contexts. Thecaveat is that the customer may have conditions that the solution has notbeen tested under and this becomes an issue, in particular when the CS isused in a safety critical systemThe rollout of a CS product in a safety critical domain is not a straight-forward activity as the assumptions made about the CS product may not holdwhen the product is integrated and tested with the rest of the system. It oftenhappens that the CS product does not operate as expected in the speciﬁc envi-ronment or that some of the system requirements are not satisﬁed by the closedsystem. Furthermore the Veriﬁcation and Validation (V&V) of Closed Systemsis particularly diﬃcult to perform because in most cases there is no design or re-quirements documentation available in the public domain, nor has the evidenceof V&V activities performed by the supplier been made public. Determining theconditions and environment under which the CS product was validated is notsimple. The non-existence of this data makes it more diﬃcult to build a case forthe reliability and safety of the end product.Where all this might not be vital in a non-safety critical domain, it is par-ticularly relevant in a safety critical domain where, for instance, the DO-178standard requires evidence of V&V activities to meet speciﬁc objectives.2 Existing Approaches2.1 Integration Testing and PrototypingThe validation of CS based solutions typically starts as soon as the CS product ismade available to the system integrator. First the product is taken through a pro-totyping stage with parts of the system being integrated with the CS products.This stage validates some functional integration aspects; it does not necessarilyaddress safety or reliability properties of the system as the focus is instead onvalidating the functional integration with the end system. The prototyping stagemay lead to the identiﬁcation of issues with the CS that require customizations,of which in some cases the supplier of the CS product can be open to ﬁxing.This stage can also identify limitations which could lead to either rejecting theCS product or the system integrator having to identify workarounds.Once the various elements of the solution are amenable to integration thesystem is formally built and only then can it go through integration testing. Theapproach used for testing is driven by the set of requirements including safetyrequirements, use cases, the test environment and last but not least the timeavailable to perform the testing.There are several risks associated with this approach:{ The identiﬁcation of issues and the need for workarounds might come late inthe lifecycle when there is no other option but to develop bespoke solutions.This impacts on the cost and schedule of the end system{ A limited test coverage as a result of the limited time available for testingand a limited test environment which does not allow for simulating the realin-service environment. This impacts on the conﬁdence in the end solutionand could result in a system deployed with untested states and an impacton the ability to generate a sound safety case{ As safety can be seen as an emergent property of the integrated system[menon2009interim], the safety analysis, which could result in the need forsigniﬁcant re-design, occurs too late in the veriﬁcation process2.2 Fault Injection TestingFault injection testing is another approach that aims to address scenarios wherethere is insuﬃcient evidence to demonstrate that the CS will not fail unac-ceptably or where the CS does not satisfy safety requirements. The primaryobjective of fault injection testing is to understand whether additional fault de-tection, isolation and recovery (FDIR) mechanisms are required or if the existingFDIR mechanisms are satisfactory. An example of a successful application of themethod is described in [MSM+02].The key risks associated with this method are:{ The APIs may not be suﬃciently well documented resulting in additionaleﬀort to eﬀectively customise the test environment and integrate it with thetarget system{ The results produced are constrained by the number of fault scenarios gen-erated and as a consequence the evidence generated may be inadequate todemonstrate the robustness of the target system{ The fault injection tool may not provide all of the interfaces required tointegrate with the equipment under test and as a consequence it may not bepossible to inject faults in the target system{ Potential failures caused by unexpected component interactions in the inte-grated system are not addressed by this approachThere are a number of other approaches that can be used to facilitate the ver-iﬁcation and validation of CS based solutions, such as using pre-existing evidenceor reverse engineering. For further details see [MHM09].3 Solution DescriptionThis paper outlines an approach that we have been applying to a real industrialproject using the Event-B formal method. The system being analysed uses anumber of complex static conﬁgurations and we were able to specify general re-quirements on these conﬁgurations in Event-B, and verify speciﬁc conﬁgurationsusing model checking and mathematical proofs. We also use reﬁnement to mapa high level model of the requirements to a lower level model that represents anabstraction of the CS product. Our experience to date suggests this approachencourages a deeper analysis of the system requirements and how these relate tothe characteristics of the CS product than a less formal approach. The formalmodels and veriﬁcation results also contribute to the safety case for the system.The work on the case study is still in an intermediate stage, yet enough workhas been completed to draw preliminary conclusions on the method and issuesfaced.3.1 Tools DescriptionThe case study uses the Event-B language, which is a formal language built uponset theory and mathematical proof, and is an evolution of the industrial-strengthB method. This is in combination with the Rodin toolset, which is an Eclipse-based open source development environment for Event-B models [ABH+10].Rodin includes proof obligation generation and automated proof tools alongwith a range of model-checking plug-ins. To date, Rodin has mostly been usedin a research environment, yet using real industrial case studies.The extensions to Rodin which are utilised in this case study are UML-B [SB08] - a UML-like graphical front end for Event-B which provides strongsupport for model reﬁnement - and ProB [LB08], a plug-in which supports sim-ulation and model-checking of both Event-B and UML-B models.3.2 Case Study OverviewThe system to be veriﬁed through the case study is a shared computer environ-ment (SCE) which consists of hardware, virtualization middleware and applica-tions. The hardware is composed of multiple servers, and the virtualization isprovided by VMWare - a closed system - which enables multiple applications tobe running on the same server and provides dynamic load-balancing and resourcereallocation across the system.Fig.1. An example of a UML-B model used in the case studyThe abstract design for the system is broken down into logical servers, whichrepresent groupings of applications that should be running on the same server.The hierarchy of the applications is introduced by the classiﬁcation of each ap-plication to a function or sub-function; this allows for functions to span multiplelogical servers if they have sub-functions running on each. In relation to VMWare,each function and sub-function corresponds to a Virtual Appliance (VApp) andcan have multiple Virtual Machines (VMs). Each SCE operates according to aﬁxed number of static conﬁgurations mapping functions and sub-functions tological servers. In this way it is possible to deﬁne conﬁgurations for multipleSCEs, a single SCE (should one or more of the SCEs fail), or minimum safe con-ﬁgurations (so that the critical applications can continue running in the eventof multiple server failures in the same SCE)The properties of the system that can be veriﬁed formally are broken downinto two phases; the static veriﬁcation of each conﬁguration, and the dynamicveriﬁcation of the design requirements with the behavior of VMWare. The staticphase of the veriﬁcation involves proving that the conﬁguration adheres to therequirements within the design - for instance, that a function assigned to a singlelogical server should not have any sub-functions - but also that the memory,processing and network resources assigned to the sum of the functions and sub-functions running on each logical server do not exceed the limits on a singleserver. VMWare can control the memory and processing resources; however, ifthe overall limits for the server are exceeded then it cannot guarantee that allof the applications will run correctly. Perhaps more critically, VMWare has nocontrol over the network usage; so it is imperative that the network resource useis checked before each conﬁguration is run.In each of the static cases the formal veriﬁcation is achieved by modelling therequirements as axioms in the model and then inserting the mappings speciﬁcto the conﬁguration into the model. To verify the static design requirementswe used ProB to check that the axioms are consistent, i.e. that ProB can ﬁndsets and constants that satisfy the axioms. By using ProBs constraint solvingabilities, we discovered several modelling errors when ProB could not ﬁnd aFig.2. Comparison of VMWare and design viewssolution that satisﬁed the axioms. We then veriﬁed that an example conﬁgura-tion (also deﬁned using axioms) was consistent in ProB and therefore satisﬁedthe axiomatic constraints on conﬁgurations such as aﬃnity rules and functionallocations.The advantage of using a model checker in this case is clear; if new conﬁg-urations are designed, or existing conﬁgurations are changed slightly by addingnew functions, then a static check can be simply and quickly performed on theconﬁguration in an automated fashion without having to manually recalculateany of the resource limits or check that the mappings are valid.To verify that an example conﬁguration obeys the resource requirements weused the Rodin provers. The resource rules were expressed as a theorem which, ifthe conﬁguration is valid, should follow from the axioms describing the resourcingrequired by the conﬁguration. This involved time consuming interactive proof,even for a simple conﬁguration. As a result we have proposed a new proof rulewhich will be added to the Rodin provers, thereby decreasing the time neededto prove future conﬁgurations.The dynamic veriﬁcation involves demonstrating that the combination of thesystem and VMWare behaves in such a way that the system design requirementsare not invalidated, as these requirements are not inherent within VMWare. Morespeciﬁcally, there are certain aﬃnity rules within the design, such as groupings ofapplications which have to be running on the same server, which could easily beinvalidated by VMWare if it decides to reallocate applications due to an increasedload on one or more of the servers. The potential issues are even more apparentwhen server failures are considered; in this case entire logical servers have tobe reallocated or the conﬁguration has to be changed, all by VMWare whilstretaining the design requirements and operational capacity of the system. Theformal approach here, although not yet complete, will be to model the dynamicbehavior of VMWare in Event-B, and verify this using the Rodin provers againstFig.3. Further example of a UML-B reﬁnement step in the case studythe existing axioms in the static veriﬁcation representing the design and resourcerequirements.The key limitations to the approach are:{ The complete CS has to be modelled and thus assumptions have to be madeabout its behaviour. This is in fact what determines the scope of the mod-elling; for instance, it is assumed for the dynamic behaviour that the memoryreallocation and low-level processes work reliably. However it is importantto keep in mind that these are the type of issues that, as mentioned earlier,are often veriﬁed during the development of the CS.{ Another known limitation to this approach is inherent to the Event-B lan-guage and is due to the restricted set of requirements that are amenable tothe language. Only a subset of the requirements could be modelled; it is notpossible to model requirements concerning the performance of the system,for instance. This is not to say that this is not possible with other formallanguages, however.{ A further limitation which has become apparent through the work is that therequirements presented are often in an unsuitable form for formal modeling,and consist of a mix of high and low levels. During the case study another,intermediate, requirements document had to be produced to ﬁll the gapbetween the initial requirements and the formal model.The modelling process is reliant on the participation of the customer; at eachstep in the process models were produced, which raised questions, and after moreinformation was gained the models were developed further. A beneﬁt of using aformal method such as Event-B is that it encourages deep requirements analysisdue to the completeness of requirements necessary for formal modeling; beforeand during the modelling missing and inconsistent requirements were found,most of which were corrected or included in the intermediate requirements doc-ument. A further beneﬁt of the Event-B approach described is that it encouragesthe investigation, discovery and modelling of safety constraints much earlier inthe veriﬁcation process than traditional methods.4 Preliminary ConclusionsThe work on the case study is still in an intermediate stage, yet the followingpreliminary conclusions can be drawn:{ Static properties of VMs running in VMWare can be formalised using theEvent-B method and veriﬁed using the Rodin toolset{ Event-B is a suitable method to verify properties which are not managed byVMWare but which are critical and have to be validated before running thevirtualised applications{ The Event-B method oﬀers the mechanisms to abstract speciﬁc details per-tinent to the CS and focus on a sub-set of properties which are complex toverify using a classical testing approach{ Event-B models of VMWare can be reused and expanded with more detailif required{ We found UML-B’s class diagram notation very suitable for modelling thissystem. UML-B allowed us to construct a model very quickly and to explorediﬀerent modelling choices, along with providing a mechanism to visualisethe model and communicate it to othersFurther work is going to be performed to formally verify the design require-ments with the behavior of VMWare.References[ABH+10] J.R. Abrial, M. Butler, S. Hallerstede, T.S. Hoang, F. Mehta, and L. Voisin.Rodin: an open toolset for modelling and reasoning in event-b. InternationalJournal on Software Tools for Technology Transfer (STTT), 12(6):447–466,2010.[LB08] M. Leuschel and M. Butler. Prob: an automated analysis toolset for the bmethod. International Journal on Software Tools for Technology Transfer(STTT), 10(2):185–203, 2008.[MHM09] C. Menon, R. Hawkins, and J. McDermid. Interim standard of best practiceon software in the context of ds 00-56 issue 4. SSEI, University of York,Standard of Best Practice, (1), 2009.[MSM+02] H. Madeira, R.R. Some, F. Moreira, D. Costa, and D. Rennels. Experimen-tal evaluation of a cots system for space applications. In Dependable Systemsand Networks, 2002. DSN 2002. Proceedings. International Conference on,pages 325–330. IEEE, 2002.[SB08] Colin Snook and Michael Butler. Uml-b and event-b: an integration oflanguages and tools. In The IASTED International Conference on SoftwareEngineering - SE2008, February 2008. Event Dates: 12-14th February 2008.

Interim standard of best practice on software in the context of ds 00-56 issue 4. SSEI,

Prob: an automated analysis toolset for the b method.

Rodin: an open toolset for modelling and reasoning in event-b.

Uml-b and event-b: an integration of languages and tools.

A Practical Approach for Closed Systems Formal Verification Using Event-B

A Practical Approach for Closed Systems Formal Verification Using Event-B

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)