Survey of source code metrics for evaluating testability of object oriented systems

Software testing is costly in terms of time and funds. Testability is a software characteristic that aims at producing systems easy to test. Several metrics have been proposed to identify the testability weaknesses. But it is sometimes difficult to be convinced that those metrics are really related with testability. This article is a critical survey of the source-code based metrics proposed in the literature for object-oriented software testability. It underlines the necessity to provide testability metrics that are proved to be intuitive and adequate for the testing cost prediction

Diagrammatic Languages and Formal Verification : A Tool-Based Approach

The importance of software correctness has been accentuated as a growing number of safety-critical systems have been developed relying on software operating these systems. One of the more prominent methods targeting the construction of a correct program is formal verification. Formal verification identifies a correct program as a program that satisfies its specification and is free of defects. While in theory formal verification guarantees a correct implementation with respect to the specification, applying formal verification techniques in practice has shown to be difficult and expensive. In response to these challenges, various support methods and tools have been suggested for all phases from program specification to proving the derived verification conditions. This thesis concerns practical verification methods applied to diagrammatic modeling languages. While diagrammatic languages are widely used in communicating system design (e.g., UML) and behavior (e.g., state charts), most formal verification platforms require the specification to be written in a textual specification language or in the mathematical language of an underlying logical framework. One exception is invariant-based programming, in which programs together with their specifications are drawn as invariant diagrams, a type of state transition diagram annotated with intermediate assertions (preconditions, postconditions, invariants). Even though the allowed program states—called situations—are described diagrammatically, the intermediate assertions defining a situation’s meaning in the domain of the program are still written in conventional textual form. To explore the use of diagrams in expressing the intermediate assertions of invariant diagrams, we designed a pictorial language for expressing array properties. We further developed this notation into a diagrammatic domain-specific language (DSL) and implemented it as an extension to the Why3 platform. The DSL supports expression of array properties. The language is based on Reynolds’s interval and partition diagrams and includes a construct for mapping array intervals to logic predicates. Automated verification of a program is attained by generating the verification conditions and proving that they are true. In practice, full proof automation is not possible except for trivial programs and verifying even simple properties can require significant effort both in specification and proof stages. An animation tool which supports run-time evaluation of the program statements and intermediate assertions given any user-defined input can support this process. In particular, an execution trace leading up to a failed assertion constitutes a refutation of a verification condition that requires immediate attention. As an extension to Socos, a verificion tool for invariant diagrams built on top of the PVS proof system, we have developed an execution model where program statements and assertions can be evaluated in a given program state. A program is represented by an abstract datatype encoding the program state, together with a small-step state transition function encoding the evaluation of a single statement. This allows the program’s runtime behavior to be formally inspected during verification. We also implement animation and interactive debugging support for Socos. The thesis also explores visualization of system development in the context of model decomposition in Event-B. Decomposing a software system becomes increasingly critical as the system grows larger, since the workload on the theorem provers must be distributed effectively. Decomposition techniques have been suggested in several verification platforms to split the models into smaller units, each having fewer verification conditions and therefore imposing a lighter load on automatic theorem provers. In this work, we have investigated a refinement-based decomposition technique that makes the development process more resilient to change in specification and allows parallel development of sub-models by a team. As part of the research, we evaluated the technique on a small case study, a simplified version of a landing gear system verification presented by Boniol and Wiels, within the Event-B specification language.Vikten av programvaras korrekthet har accentuerats då ett växande antal säkerhetskritiska system, vilka är beroende av programvaran som styr dessa, har utvecklas. En av de mer framträdande metoderna som riktar in sig på utveckling av korrekt programvara är formell verifiering. Inom formell verifiering avses med ett korrekt program ett program som uppfyller sina specifikationer och som är fritt från defekter. Medan formell verifiering teoretiskt sett kan garantera ett korrekt program med avseende på specifikationerna, har tillämpligheten av formella verifieringsmetod visat sig i praktiken vara svår och dyr. Till svar på dessa utmaningar har ett stort antal olika stödmetoder och automatiseringsverktyg föreslagits för samtliga faser från specifikationen till bevisningen av de härledda korrekthetsvillkoren. Denna avhandling behandlar praktiska verifieringsmetoder applicerade på diagrambaserade modelleringsspråk. Medan diagrambaserade språk ofta används för kommunikation av programvarudesign (t.ex. UML) samt beteende (t.ex. tillståndsdiagram), kräver de flesta verifieringsplattformar att specifikationen kodas medelst ett textuellt specifikationsspåk eller i språket hos det underliggande logiska ramverket. Ett undantag är invariantbaserad programmering, inom vilken ett program tillsammans med dess specifikation ritas upp som sk. invariantdiagram, en typ av tillståndstransitionsdiagram annoterade med mellanliggande logiska villkor (förvillkor, eftervillkor, invarianter). Även om de tillåtna programtillstånden—sk. situationer—beskrivs diagrammatiskt är de logiska predikaten som beskriver en situations betydelse i programmets domän fortfarande skriven på konventionell textuell form. För att vidare undersöka användningen av diagram vid beskrivningen av mellanliggande villkor inom invariantbaserad programming, har vi konstruerat ett bildbaserat språk för villkor över arrayer. Vi har därefter vidareutvecklat detta språk till ett diagrambaserat domän-specifikt språk (domain-specific language, DSL) och implementerat stöd för det i verifieringsplattformen Why3. Språket låter användaren uttrycka egenskaper hos arrayer, och är baserat på Reynolds intevall- och partitionsdiagram samt inbegriper en konstruktion för mappning av array-intervall till logiska predikat. Automatisk verifiering av ett program uppnås genom generering av korrekthetsvillkor och åtföljande bevisning av dessa. I praktiken kan full automatisering av bevis inte uppnås utom för trivial program, och även bevisning av enkla egenskaper kan kräva betydande ansträngningar både vid specifikations- och bevisfaserna. Ett animeringsverktyg som stöder exekvering av såväl programmets satser som mellanliggande villkor för godtycklig användarinput kan vara till hjälp i denna process. Särskilt ett exekveringspår som leder upp till ett falskt mellanliggande villkor utgör ett direkt vederläggande (refutation) av ett bevisvillkor, vilket kräver omedelbar uppmärksamhet från programmeraren. Som ett tillägg till Socos, ett verifieringsverktyg för invariantdiagram baserat på bevissystemet PVS, har vi utvecklat en exekveringsmodell där programmets satser och villkor kan evalueras i ett givet programtillstånd. Ett program representeras av en abstrakt datatyp för programmets tillstånd tillsammans med en small-step transitionsfunktion för evalueringen av en enskild programsats. Detta möjliggör att ett programs exekvering formellt kan analyseras under verifieringen. Vi har också implementerat animation och interaktiv felsökning i Socos. Avhandlingen undersöker också visualisering av systemutveckling i samband med modelluppdelning inom Event-B. Uppdelning av en systemmodell blir allt mer kritisk då ett systemet växer sig större, emedan belastningen på underliggande teorembe visare måste fördelas effektivt. Uppdelningstekniker har föreslagits inom många olika verifieringsplattformar för att dela in modellerna i mindre enheter, så att varje enhet har färre verifieringsvillkor och därmed innebär en mindre belastning på de automatiska teorembevisarna. I detta arbete har vi undersökt en refinement-baserad uppdelningsteknik som gör utvecklingsprocessen mer kapabel att hantera förändringar hos specifikationen och som tillåter parallell utveckling av delmodellerna inom ett team. Som en del av forskningen har vi utvärderat tekniken på en liten fallstudie: en förenklad modell av automationen hos ett landningsställ av Boniol and Wiels, uttryckt i Event-B-specifikationspråket

Codeklonerkennung mit Dominatorinformationen

If an existing function in a software project is copied and reused (in a slightly modified version), the result is a code clone. If there was an error or vulnerability in the original function, this error or vulnerability is now contained in several places in the software project. This is one of the reasons why research is being done to develop powerful and scalable clone detection techniques. In this thesis, a new clone detection method is presented that uses paths and path sets derived from the dominator trees of the functions to detect the code clones. A dominator tree is a special form of the control flow graph, which does not contain cycles. The dominator tree based method has been implemented in the StoneDetector tool and can detect code clones in Java source code as well as in Java bytecode. It has equally good or better recall and precision results than previously published code clone detection methods. The evaluation was performed using the BigCloneBench. Scalability measurements showed that even source code with several 100 million lines of code can be searched in a reasonable time. In order to evaluate the bytecode based StoneDetector variant, the BigCloneBench files had to be compiled. For this purpose, the Stubber tool was developed, which can compile Java source code files without the required libraries. Finally, it could be shown that using the register code generated from the Java bytecode, similar recall and precision values could be achieved compared to the source code based variant. Since some machine learning studies specify that very good recall and precision values can be achieved for all clone types, a machine learning method was trained with dominator trees. It could be shown that the results published by the studies are not reproducible on unseen data.Wird eine bestehende Funktion in einem Softwareprojekt kopiert und (in leicht angepasster Form) erneut genutzt, entsteht ein Codeklon. War in der ursprünglichen Funktion jedoch ein Fehler oder eine Schwachstelle, so ist dieser Fehler beziehungsweise diese Schwachstelle jetzt an mehreren Stellen im Softwareprojekt enthalten. Dies ist einer der Gründe, weshalb an der Entwicklung von leistungsstarken und skalierbaren Klonerkennungsverfahren geforscht wird. In der hier vorliegenden Arbeit wird ein neues Klonerkennungsverfahren vorgestellt, das zum Detektieren der Codeklone Pfade und Pfadmengen nutzt, die aus den Dominatorbäumen der Funktionen abgeleitet werden. Ein Dominatorbaum wird aus dem Kontrollflussgraphen abgeleitet und enthält keine Zyklen. Das Dominatorbaum-basierte Verfahren wurde in dem Werkzeug StoneDetector umgesetzt und kann Codeklone sowohl im Java-Quelltext als auch im Java-Bytecode detektieren. Dabei hat es gleich gute oder bessere Recall- und Precision-Werte als bisher veröffentlichte Codeklonerkennungsverfahren. Die Wert-Evaluierungen wurden dabei unter Verwendung des BigClone-Benchs durchgeführt. Skalierbarkeitsmessungen zeigten, dass sogar Quellcodedateien mit mehreren 100-Millionen Codezeilen in angemessener Zeit durchsucht werden können. Damit die Bytecode-basierte StoneDetector-Variante auch evaluiert werden konnte, mussten die Dateien des BigCloneBench kompiliert werden. Dazu wurde das Stubber-Tool entwickelt, welches Java-Quelltextdateien ohne die benötigten Abhängigkeiten kompilieren kann. Schlussendlich konnte somit gezeigt werden, dass mithilfe des aus dem Java-Bytecode generierten Registercodes ähnliche Recall- und Precision-Werte im Vergleich zu der Quelltext-basierten Variante erreicht werden können. Da einige Arbeiten mit maschinellen Lernverfahren angeben, bei allen Klontypen sehr gute Recall- und Precision-Werte zu erreichen, wurde ein maschinelles Lernverfahren mit Dominatoräumen trainiert. Es konnte gezeigt werden, dass die von den Arbeiten veröffentlichten Ergebnisse nicht auf ungesehenen Daten reproduzierbar sind

Using a Dynamic Domain-Specific Modeling Language for the Model-Driven Development of Cross-Platform Mobile Applications

There has been a gradual but steady convergence of dynamic programming languages with modeling languages. One area that can benefit from this convergence is modeldriven development (MDD) especially in the domain of mobile application development. By using a dynamic language to construct a domain-specific modeling language (DSML), it is possible to create models that are executable, exhibit flexible type checking, and provide a smaller cognitive gap between business users, modelers and developers than more traditional model-driven approaches. Dynamic languages have found strong adoption by practitioners of Agile development processes. These processes often rely on developers to rapidly produce working code that meets business needs and to do so in an iterative and incremental way. Such methodologies tend to eschew “throwaway” artifacts and models as being wasteful except as a communication vehicle to produce executable code. These approaches are not readily supported with traditional heavyweight approaches to model-driven development such as the Object Management Group’s Model-Driven Architecture approach. This research asks whether it is possible for a domain-specific modeling language written in a dynamic programming language to define a cross-platform model that can produce native code and do so in a way that developer productivity and code quality are at least as effective as hand-written code produced using native tools. Using a prototype modeling tool, AXIOM (Agile eXecutable and Incremental Objectoriented Modeling), we examine this question through small- and mid-scale experiments and find that the AXIOM approach improved developer productivity by almost 400%, albeit only after some up-front investment. We also find that the generated code can be of equal if not better quality than the equivalent hand-written code. Finally, we find that there are significant challenges in the synthesis of a DSML that can be used to model applications across platforms as diverse as today’s mobile operating systems, which point to intriguing avenues of subsequent research

Summary of Research 1994

The views expressed in this report are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.This report contains 359 summaries of research projects which were carried out under funding of the Naval Postgraduate School Research Program. A list of recent publications is also included which consists of conference presentations and publications, books, contributions to books, published journal papers, and technical reports. The research was conducted in the areas of Aeronautics and Astronautics, Computer Science, Electrical and Computer Engineering, Mathematics, Mechanical Engineering, Meteorology, National Security Affairs, Oceanography, Operations Research, Physics, and Systems Management. This also includes research by the Command, Control and Communications (C3) Academic Group, Electronic Warfare Academic Group, Space Systems Academic Group, and the Undersea Warfare Academic Group

Automated Failure Explanation Through Execution Comparison

When fixing a bug in software, developers must build an understanding or explanation of the bug and how the bug flows through a program. The effort that developers must put into building this explanation is costly and laborious. Thus, developers need tools that can assist them in explaining the behavior of bugs. Dynamic slicing is one technique that can effectively show how a bug propagates through an execution up to the point where a program fails. However, dynamic slices are large because they do not just explain the bug itself; they include extra information that explains any observed behavior that might be connected to the bug. Thus, the explanation of the bug is hidden within this other tangentially related information. This dissertation addresses the problem and shows how a failing execution and a correct execution may be compared in order to construct explanations that include only information about what caused the bug. As a result, these automated explanations are significantly more concise than those explanations produced by existing dynamic slicing techniques. To enable the comparison of executions, we develop new techniques for dynamic analyses that identify the commonalities and differences between executions. First, we devise and implement the notion of a point within an execution that may exist across multiple executions. We also note that comparing executions involves comparing the state or variables and their values that exist within the executions at different execution points. Thus, we design an approach for identifying the locations of variables in different executions so that their values may be compared. Leveraging these tools, we design a system for identifying the behaviors within an execution that can be blamed for a bug and that together compose an explanation for the bug. These explanations are up to two orders of magnitude smaller than those produced by existing state of the art techniques. We also examine how different choices of a correct execution for comparison can impact the practicality or potential quality of the explanations produced via our system

Combining SOA and BPM Technologies for Cross-System Process Automation

This paper summarizes the results of an industry case study that introduced a cross-system business process automation solution based on a combination of SOA and BPM standard technologies (i.e., BPMN, BPEL, WSDL). Besides discussing major weaknesses of the existing, custom-built, solution and comparing them against experiences with the developed prototype, the paper presents a course of action for transforming the current solution into the proposed solution. This includes a general approach, consisting of four distinct steps, as well as specific action items that are to be performed for every step. The discussion also covers language and tool support and challenges arising from the transformation