Grey-box Testing and Verification of Java/JML

International audienceWe present in this paper the application of constraint solving techniques to the validation and automated test cases generation for Java programs, annotated with JML specifications. The Java/JML code is translated into a constraint representation based on a subset of the set-theory, which is well-suited for modelling object-oriented programs. Symbolic code execution techniques can then be applied to produce test cases, using classical structural test selection criteria, or to detect possible runtime errors, and non-conformances between the Java code and its embedded JML model

Executing Underspecified OCL Operation Contracts with a SAT Solver

Executing formal operation contracts is an important technique for requirements validation and rapid prototyping. Current approaches require additional guidance from the user or exhibit poor performance for underspecified contracts that describe the operation results non-constructively. We present an efficient and fully automatic approach to executing OCL operation contracts which uses a satisfiability (SAT) solver. The operation contract is translated to an arithmetic formula with bounded quantifiers and later to a satisfiability problem. Based on the system state in which the operation is called and the arguments to the operation, an off-the-shelf SAT solver computes a new state that satisfies the postconditions of the operation. An effort is made to keep the changes to the system state as small as possible. We present a tool for generating Java method bodies for operations specified with OCL. The efficiency of our method is confirmed by a comparison with existing approaches

Generating Tests from {B} Specifications and Test Purposes

International audienceThis paper is about generating tests from test purposes, in addition to structural tests. We present a method that re-uses a behavioural model and an abstract test concretization layer developed for structural testing, and relies on additional test purposes. We propose, in the B framework, a process of test generation that uses the symbolic animation mechanisms of LTG (Leirios Test Generator) based on constraint solving, and guided by the test purposes. We build for that a B animable model that is the synchronized product of a behavioural B abstract model and a test purpose described as a labelled transition system. We prove the correctness of this method, and illustrate it by means of the IAS case study. IAS is a smart-card application dedicated to the operations of Identification, Authentication and electronic Signature

The 14th Overture Workshop: Towards Analytical Tool Chains

This report contains the proceedings from the 14th Overture workshop organized in connection with the Formal Methods 2016 symposium. This includes nine papers describing different technological progress in relation to the Overture/VDM tool support and its connection with other tools such as Crescendo, Symphony, INTO-CPS, TASTE and ViennaTalk

A debugging engine for parallel and distributed programs

Dissertação apresentada para a obtenção do Grau de Doutor em Informática pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia.In the last decade a considerable amount of research work has focused on distributed debugging, one of the crucial fields in the parallel software development cycle. The productivity of the software development process strongly depends on the adequate definition of what debugging tools should be provided, and what debugging methodologies and functionalities should these tools support. The work described in this dissertation was initiated in 1995, in the context of two research projects, the SEPP (Software Engineering for Parallel Processing) and HPCTI (High-Performance Computing Tools for Industry), both sponsored by the European Union in the Copernicus programme, which aimed at the design and implementation of an integrated parallel software development environment. In the context of these projects, two independent toolsets have been developed, the GRADE and EDPEPPS parallel software development environments. Our contribution to these projects was in the debugging support. We have designed a debugging engine and developed a prototype, which was integrated the both toolsets (it was the only tool developed in the context of the SEPP and HPCTI projects which achieved such a result). Even after the closing of those research projects, further research work on distributed debugger has been carried on, which conducted to the re-design and re-implementation of the debugging engine. This dissertation describes the debugging engine according to its most up-to-date design and implementation stages. It also reposts some of the experimentalworkmade with both the initial and the current implementations, and how it contributed to validate the design and implementations of the debugging engine

Enhancing System Realisation in Formal Model Development

Software for mission-critical systems is sometimes analysed using formal specification to increase the chances of the system behaving as intended. When sufficient insights into the system have been obtained from the formal analysis, the formal specification is realised in the form of a software implementation. One way to realise the system's software is by automatically generating it from the formal specification -- a technique referred to as code generation. However, in general it is difficult to make guarantees about the correctness of the generated code -- especially while requiring automation of the steps involved in realising the formal specification. This PhD dissertation investigates ways to improve the automation of the steps involved in realising and validating a system based on a formal specification. The approach aims to develop properly designed software tools which support the integration of formal methods tools into the software development life cycle, and which leverage the formal specification in the subsequent validation of the system. The tools developed use a new code generation infrastructure that has been built as part of this PhD project and implemented in the Overture tool -- a formal methods tool that supports the Vienna Development Method. The development of the code generation infrastructure has involved the re-design of the software architecture of Overture. The new architecture brings forth the reuse and extensibility features of Overture to take into account the needs and requirements of software extensions targeting Overture. The tools developed in this PhD project have successfully supported three case studies from externally funded projects. The feedback received from the case study work has further helped improve the code generation infrastructure and the tools built using it

Verification, slicing, and visualization of programs with contracts

Tese de doutoramento em Informática (área de especialização em Ciências da Computação)As a specification carries out relevant information concerning the behaviour of a program, why not explore this fact to slice a program in a semantic sense aiming at optimizing it or easing its verification? It was this idea that Comuzzi, in 1996, introduced with the notion of postcondition-based slicing | slice a program using the information contained in the postcondition (the condition Q that is guaranteed to hold at the exit of a program). After him, several advances were made and different extensions were proposed, bridging the two areas of Program Verification and Program Slicing: specifically precondition-based slicing and specification-based slicing. The work reported in this Ph.D. dissertation explores further relations between these two areas aiming at discovering mutual benefits. A deep study of specification-based slicing has shown that the original algorithm is not efficient and does not produce minimal slices. In this dissertation, traditional specification-based slicing algorithms are revisited and improved (their formalization is proposed under the name of assertion-based slicing), in a new framework that is appropriate for reasoning about imperative programs annotated with contracts and loop invariants. In the same theoretical framework, the semantic slicing algorithms are extended to work at the program level through a new concept called contract based slicing. Contract-based slicing, constituting another contribution of this work, allows for the study of a program at an interprocedural level, enabling optimizations in the context of code reuse. Motivated by the lack of tools to prove that the proposed algorithms work in practice, a tool (GamaSlicer) was also developed. It implements all the existing semantic slicing algorithms, in addition to the ones introduced in this dissertation. This third contribution is based on generic graph visualization and animation algorithms that were adapted to work with verification and slice graphs, two specific cases of labelled control low graphs.Tendo em conta que uma especificação contém informação relevante no que diz respeito ao comportamento de um programa, faz sentido explorar este facto para o cortar em fatias (slice) com o objectivo de o optimizar ou de facilitar a sua verificação. Foi precisamente esta ideia que Comuzzi introduziu, em 1996, apresentando o conceito de postcondition-based slicing que consiste em cortar um programa usando a informação contida na pos-condicão (a condição Q que se assegura ser verdadeira no final da execução do programa). Depois da introdução deste conceito, vários avanços foram feitos e diferentes extensões foram propostas, aproximando desta forma duas áreas que até então pareciam desligadas: Program Verification e Program Slicing. Entre estes conceitos interessa-nos destacar as noções de precondition-based slicing e specification-based slicing, que serão revisitadas neste trabalho. Um estudo aprofundado do conceito de specification-based slicing relevou que o algoritmo original não é eficiente e não produz slices mínimos. O trabalho reportado nesta dissertação de doutoramento explora a ideia de tornar mais próximas essas duas áreas visando obter benefícios mútuos. Assim, estabelecendo uma nova base teórica matemática, os algoritmos originais de specification-based slicing são revistos e aperfeiçoados | a sua formalizacão é proposta com o nome de assertion-based slicing. Ainda sobre a mesma base teórica, os algoritmos de slicing são extendidos, de forma a funcionarem ao nível do programa; alem disso introduz-se um novo conceito: contract-based slicing. Este conceito, contract-based slicing, sendo mais um dos contributos do trabalho aqui descrito, possibilita o estudo de um programa ao nível externo de um procedimento, permitindo, por um lado, otimizações no contexto do seu uso, e por outro, a sua reutilização segura. Devido à falta de ferramentas que provem que os algoritmos propostos de facto funcionam na prática, foi desenvolvida uma, com o nome GamaSlicer, que implementa todos os algoritmos existentes de slicing semântico e os novos propostos. Uma terceira contribuição é baseada nos algoritmos genéricos de visualização e animação de grafos que foram adaptados para funcionar com os grafos de controlo de fluxo etiquetados e os grafos de verificação e slicing.Fundação para a Ciência e a Tecnologia (FCT) através da Bolsa de Doutoramento SFRH/BD/33231/2007Projecto RESCUE (contrato FCT sob a referência PTDC / EIA / 65862 /2006)Projecto CROSS (contrato FCT sob a referência PTDC / EIACCO / 108995 / 2008

Diagrammatic Languages and Formal Verification : A Tool-Based Approach

The importance of software correctness has been accentuated as a growing number of safety-critical systems have been developed relying on software operating these systems. One of the more prominent methods targeting the construction of a correct program is formal verification. Formal verification identifies a correct program as a program that satisfies its specification and is free of defects. While in theory formal verification guarantees a correct implementation with respect to the specification, applying formal verification techniques in practice has shown to be difficult and expensive. In response to these challenges, various support methods and tools have been suggested for all phases from program specification to proving the derived verification conditions. This thesis concerns practical verification methods applied to diagrammatic modeling languages. While diagrammatic languages are widely used in communicating system design (e.g., UML) and behavior (e.g., state charts), most formal verification platforms require the specification to be written in a textual specification language or in the mathematical language of an underlying logical framework. One exception is invariant-based programming, in which programs together with their specifications are drawn as invariant diagrams, a type of state transition diagram annotated with intermediate assertions (preconditions, postconditions, invariants). Even though the allowed program states—called situations—are described diagrammatically, the intermediate assertions defining a situation’s meaning in the domain of the program are still written in conventional textual form. To explore the use of diagrams in expressing the intermediate assertions of invariant diagrams, we designed a pictorial language for expressing array properties. We further developed this notation into a diagrammatic domain-specific language (DSL) and implemented it as an extension to the Why3 platform. The DSL supports expression of array properties. The language is based on Reynolds’s interval and partition diagrams and includes a construct for mapping array intervals to logic predicates. Automated verification of a program is attained by generating the verification conditions and proving that they are true. In practice, full proof automation is not possible except for trivial programs and verifying even simple properties can require significant effort both in specification and proof stages. An animation tool which supports run-time evaluation of the program statements and intermediate assertions given any user-defined input can support this process. In particular, an execution trace leading up to a failed assertion constitutes a refutation of a verification condition that requires immediate attention. As an extension to Socos, a verificion tool for invariant diagrams built on top of the PVS proof system, we have developed an execution model where program statements and assertions can be evaluated in a given program state. A program is represented by an abstract datatype encoding the program state, together with a small-step state transition function encoding the evaluation of a single statement. This allows the program’s runtime behavior to be formally inspected during verification. We also implement animation and interactive debugging support for Socos. The thesis also explores visualization of system development in the context of model decomposition in Event-B. Decomposing a software system becomes increasingly critical as the system grows larger, since the workload on the theorem provers must be distributed effectively. Decomposition techniques have been suggested in several verification platforms to split the models into smaller units, each having fewer verification conditions and therefore imposing a lighter load on automatic theorem provers. In this work, we have investigated a refinement-based decomposition technique that makes the development process more resilient to change in specification and allows parallel development of sub-models by a team. As part of the research, we evaluated the technique on a small case study, a simplified version of a landing gear system verification presented by Boniol and Wiels, within the Event-B specification language.Vikten av programvaras korrekthet har accentuerats då ett växande antal säkerhetskritiska system, vilka är beroende av programvaran som styr dessa, har utvecklas. En av de mer framträdande metoderna som riktar in sig på utveckling av korrekt programvara är formell verifiering. Inom formell verifiering avses med ett korrekt program ett program som uppfyller sina specifikationer och som är fritt från defekter. Medan formell verifiering teoretiskt sett kan garantera ett korrekt program med avseende på specifikationerna, har tillämpligheten av formella verifieringsmetod visat sig i praktiken vara svår och dyr. Till svar på dessa utmaningar har ett stort antal olika stödmetoder och automatiseringsverktyg föreslagits för samtliga faser från specifikationen till bevisningen av de härledda korrekthetsvillkoren. Denna avhandling behandlar praktiska verifieringsmetoder applicerade på diagrambaserade modelleringsspråk. Medan diagrambaserade språk ofta används för kommunikation av programvarudesign (t.ex. UML) samt beteende (t.ex. tillståndsdiagram), kräver de flesta verifieringsplattformar att specifikationen kodas medelst ett textuellt specifikationsspåk eller i språket hos det underliggande logiska ramverket. Ett undantag är invariantbaserad programmering, inom vilken ett program tillsammans med dess specifikation ritas upp som sk. invariantdiagram, en typ av tillståndstransitionsdiagram annoterade med mellanliggande logiska villkor (förvillkor, eftervillkor, invarianter). Även om de tillåtna programtillstånden—sk. situationer—beskrivs diagrammatiskt är de logiska predikaten som beskriver en situations betydelse i programmets domän fortfarande skriven på konventionell textuell form. För att vidare undersöka användningen av diagram vid beskrivningen av mellanliggande villkor inom invariantbaserad programming, har vi konstruerat ett bildbaserat språk för villkor över arrayer. Vi har därefter vidareutvecklat detta språk till ett diagrambaserat domän-specifikt språk (domain-specific language, DSL) och implementerat stöd för det i verifieringsplattformen Why3. Språket låter användaren uttrycka egenskaper hos arrayer, och är baserat på Reynolds intevall- och partitionsdiagram samt inbegriper en konstruktion för mappning av array-intervall till logiska predikat. Automatisk verifiering av ett program uppnås genom generering av korrekthetsvillkor och åtföljande bevisning av dessa. I praktiken kan full automatisering av bevis inte uppnås utom för trivial program, och även bevisning av enkla egenskaper kan kräva betydande ansträngningar både vid specifikations- och bevisfaserna. Ett animeringsverktyg som stöder exekvering av såväl programmets satser som mellanliggande villkor för godtycklig användarinput kan vara till hjälp i denna process. Särskilt ett exekveringspår som leder upp till ett falskt mellanliggande villkor utgör ett direkt vederläggande (refutation) av ett bevisvillkor, vilket kräver omedelbar uppmärksamhet från programmeraren. Som ett tillägg till Socos, ett verifieringsverktyg för invariantdiagram baserat på bevissystemet PVS, har vi utvecklat en exekveringsmodell där programmets satser och villkor kan evalueras i ett givet programtillstånd. Ett program representeras av en abstrakt datatyp för programmets tillstånd tillsammans med en small-step transitionsfunktion för evalueringen av en enskild programsats. Detta möjliggör att ett programs exekvering formellt kan analyseras under verifieringen. Vi har också implementerat animation och interaktiv felsökning i Socos. Avhandlingen undersöker också visualisering av systemutveckling i samband med modelluppdelning inom Event-B. Uppdelning av en systemmodell blir allt mer kritisk då ett systemet växer sig större, emedan belastningen på underliggande teorembe visare måste fördelas effektivt. Uppdelningstekniker har föreslagits inom många olika verifieringsplattformar för att dela in modellerna i mindre enheter, så att varje enhet har färre verifieringsvillkor och därmed innebär en mindre belastning på de automatiska teorembevisarna. I detta arbete har vi undersökt en refinement-baserad uppdelningsteknik som gör utvecklingsprocessen mer kapabel att hantera förändringar hos specifikationen och som tillåter parallell utveckling av delmodellerna inom ett team. Som en del av forskningen har vi utvärderat tekniken på en liten fallstudie: en förenklad modell av automationen hos ett landningsställ av Boniol and Wiels, uttryckt i Event-B-specifikationspråket