Automated Fixing of Programs with Contracts

This paper describes AutoFix, an automatic debugging technique that can fix faults in general-purpose software. To provide high-quality fix suggestions and to enable automation of the whole debugging process, AutoFix relies on the presence of simple specification elements in the form of contracts (such as pre- and postconditions). Using contracts enhances the precision of dynamic analysis techniques for fault detection and localization, and for validating fixes. The only required user input to the AutoFix supporting tool is then a faulty program annotated with contracts; the tool produces a collection of validated fixes for the fault ranked according to an estimate of their suitability. In an extensive experimental evaluation, we applied AutoFix to over 200 faults in four code bases of different maturity and quality (of implementation and of contracts). AutoFix successfully fixed 42% of the faults, producing, in the majority of cases, corrections of quality comparable to those competent programmers would write; the used computational resources were modest, with an average time per fix below 20 minutes on commodity hardware. These figures compare favorably to the state of the art in automated program fixing, and demonstrate that the AutoFix approach is successfully applicable to reduce the debugging burden in real-world scenarios.Comment: Minor changes after proofreadin

Fostering design by contract by exploiting the relationship between code commentary and contracts.

Embora contratos no estilo de programação por contratos (DBC) tragam precisão para expressar o comportamento do código, desenvolvedores são resistentes ao seu uso. Há várias razões para isto, tais como a dificuldade na produção de contratos ou o trabalho de manter os contratos consistentes com o código em evolução. Por outro lado, Javadoc é uma abordagem comumente usada para documentar programas Java. Mesmo assim, comentários Javadoc não servem para a checagem automática de conformidade devido à ambiguidade inerente à linguagem natural. Neste trabalho, procuramos minimizar a distância entre contratos e Javadoc, estimulando a adoção de DBC a partir de duas contribuições principais; primeiro, propomos uma extensão ao sistema de tags do Javadoc (CONTRACTJ DOC) para possibilitar a integração de contratos na notação de comentários; então, propomos uma abordagem para geração de contratos a partir de comentários em linguagem natural (CONTRACT SUGGESTOR). Nós realizamos três avaliações: primeiro, avaliamos a aplicabilidade e a compreensibilidade de CONTRACTJ DOC. Como resultados, detectamos inconsistências entre a documentação Javadoc e o código fonte. A maioria dos contratos que escrevemos foram checagens de valores limítrofes para parâmetros e repetições de expressões de retorno de métodos. Além disso, a legibilidade dos comentários percebida pelos desenvolvedores não diferiu significativamente entre as abordagens, o que é promissor, dado que contratos são usualmente classificados como difíceis de ler. Segundo, avaliamos a qualidade dos contratos gerados por CONTRACT SUGGESTOR verificando a taxa de falsos positivos gerados. Como resultado, são gerados mais contratos corretos para non-null do que para relational, devido a quantidade de instâncias de comentários para cada propriedade. Por fim, realizamos estudos de caso com JMLOK2 e C ONTRACTOK – CONTRACTOK é uma extensão da abordagem de J MLOK2 para o contexto C#/Code Contracts. Primeiro, usamos JMLOK2 para verificar os contratos gerados automaticamente por CONTRACT SUGGESTOR; depois usamos as ferramentas para verificar 24 sistemas de código aberto (12 para cada ferramenta). As ferramentas detectaram 188 não-conformidades, sendo 72 problemas de pós-condição e 61 de invariante; as causas prováveis mais comuns foram Pré-condição fraca (91) e Erro de código (56). Com isso, objetivamos motivar a adoção de DBC como forma de aprimorar o projeto dos programas, e por consequência, sua qualidade geral.Contracts in Design by Contract style bring about preciseness for expressing the code behavior; however, developers are resistant to their use. There are several likely reasons for this, such as the trouble to conceive good, useful contracts, or the burden of maintaining contracts consistent with the evolving code. On the other hand, Javadoc is a common way of documenting Java programs. Nevertheless, Javadoc comments do not serve to an automated conformance checking due to ambiguity issues inherent to the natural languages. In this work, we try to minimize the distance between contracts and Javadoc, fostering DBC adoption by means of two main contributions; first, we propose an extension to the Javadoc tagging system (C ONTRACTJD OC) for allowing the integration of contracts into the comments notation; then, we propose an approach for automatically generating contracts based on natural language code commentary (CONTRACTSUGGESTOR). We perform three evaluations: first, we evaluate the applicability and comprehensibility of C ONTRACTJD OC. As results, we detected inconsistencies between the documentation available by means of Javadoc comments and the source code. The majority of the contracts we could write from the comments remains between common-case and repetitive with the code. Moreover, developers’ impression about the readability of comments did not differ significantly, which is promising, as contracts are usually regarded as hard to read – one reason for its non-adoption. Then, we evaluate the quality of contracts generated by CONTRACTSUGGESTOR by analyzing the false positives rate. As result, the approach generates more correct contracts for non-null than for relational, due to the number of comment instances for each property. Finally, we perform case studies with J MLOK2 and C ONTRACTOK – C ONTRACTOK is an extension of JMLOK2 for C#/Code Contracts context. First, we used J MLOK2 for conformance checking the contracts automatically generated; then, we run the tools over 24 open-source systems (12 with each tool). The tools detected 188 nonconformances. From those, 72 are postcondition and 61 are invariant problems; with respect to likely causes manually established, Weak precondition (91) and Code error (56) are the most commons. With this, we aim to promote DBC adoption as a way for improving the design of the projects, and consequently, their quality in general.Cape

Automating test oracles generation

Software systems play a more and more important role in our everyday life. Many relevant human activities nowadays involve the execution of a piece of software. Software has to be reliable to deliver the expected behavior, and assessing the quality of software is of primary importance to reduce the risk of runtime errors. Software testing is the most common quality assessing technique for software. Testing consists in running the system under test on a finite set of inputs, and checking the correctness of the results. Thoroughly testing a software system is expensive and requires a lot of manual work to define test inputs (stimuli used to trigger different software behaviors) and test oracles (the decision procedures checking the correctness of the results). Researchers have addressed the cost of testing by proposing techniques to automatically generate test inputs. While the generation of test inputs is well supported, there is no way to generate cost-effective test oracles: Existing techniques to produce test oracles are either too expensive to be applied in practice, or produce oracles with limited effectiveness that can only identify blatant failures like system crashes. Our intuition is that cost-effective test oracles can be generated using information produced as a byproduct of the normal development activities. The goal of this thesis is to create test oracles that can detect faults leading to semantic and non-trivial errors, and that are characterized by a reasonable generation cost. We propose two ways to generate test oracles, one derives oracles from the software redundancy and the other from the natural language comments that document the source code of software systems. We present a technique that exploits redundant sequences of method calls encoding the software redundancy to automatically generate test oracles named CCOracles. We describe how CCOracles are automatically generated, deployed, and executed. We prove the effectiveness of CCOracles by measuring their fault-finding effectiveness when combined with both automatically generated and hand-written test inputs. We also present Toradocu, a technique that derives executable specifications from Javadoc comments of Java constructors and methods. From such specifications, Toradocu generates test oracles that are then deployed into existing test suites to assess the outputs of given test inputs. We empirically evaluate Toradocu, showing that Toradocu accurately translates Javadoc comments into procedure specifications. We also show that Toradocu oracles effectively identify semantic faults in the SUT. CCOracles and Toradocu oracles stem from independent information sources and are complementary in the sense that they check different aspects of the system undertest

Tools and Language Elements for Testing, Encapsulation and Controlling Abstraction in Large Scale C++ Projects

A disszertáció új kutatási eredményeket mutat be három alapvető szoftver fejlesztési területen: tesztelés, egységbezárás és absztrakció. Az első három tézis az ún. nem-tolakodó teszteléssel foglalkozik, amely egy olyan tesztelési technika amely során nem szükséges semmilyen strukturális módosítást végrehajtanunk a termék forráskódján. Megvitatjuk a már létező nem-tolakodó tesztelési módszereket és felsoroljuk ezek előnyeit és hátrányait. Bevezetünk egy új, nem-tolakodó tesztelési módszert amely függvény hívás közbeavatkozáson alapszik és számos egyértelmű előnnyel rendelkezik a korábbi megoldásokhoz képest. Ezzel az új technikával képesek vagyunk függvényeket teszt dublőrökkel helyettesíteni még akkor is ha azok inline függvények. Továbbá bemutatunk két új kísérleti eljárást amelyek lehetővé teszik, hogy akár típusokat is helyettesítsünk teszt dublőrökkel: az egyik metódus szintaxis fa transzformációkon alapszik, a másik pedig fordítási idejű reflectionön. Demonstráljuk, hogy gyakran előfordul, hogy szükséges privát tagokhoz hozzáférni a nem-tolakodó tesztek esetében. Bemutatunk két új módszert a privát tagok eléréséhez (és ily módon támogatjuk a nem-tolakodó és fehér doboz tesztek létrehozását): egy program könyvtárat amely explicit sablon példányosításon alapszik, illetve az osztályon kívüli barát (friend) nyelvi elemet. Az egységbezárással kapcsolatosan szemléltetjük, hogy bizonyos nyelvi konstrukciók minta C++ barát (friend) túlzottan erős hozzáférést nyújthat egy osztály belső elemeihez. Ez a túlzott hozzáférés hibák forrása lehet az adott szoftverben. Javaslatot teszünk egy új nyelvi elem létrehozására amely lehetővé teszi, hogy megszorítsuk ezt a hozzáférést csupán néhány jól specifikált taghoz, ily módon erősítendő az egységbezárást és adatrejtést. Az egységbezárás mellett az absztrakció a másik alapvető szereplő ha nagy méretű szoftverek fejlesztéséről van szó. Különösen,ha többszálú programokról beszélünk. Bemutatunk egy új magas szintű C++ absztrakciót mely a read-copy-update konkurrens programozási mintán alapszik és elfogadható teljesítményt nyújt amellett, hogy kellően generikus és biztonságos használni. Az itt bemutatott új módszerek mindegyikéhez tartozik prototípus implementáció (ez alól kivételt képez a reflection alapú nem-tolakodó tesztelés ötlete)

Tagungsband zum 21. Kolloquium Programmiersprachen und Grundlagen der Programmierung

Das 21. Kolloquium Programmiersprachen und Grundlagen der Programmierung (KPS 2021) setzt eine traditionelle Reihe von Arbeitstagungen fort, die 1980 von den Forschungsgruppen der Professoren Friedrich L. Bauer (TU München), Klaus Indermark (RWTH Aachen) und Hans Langmaack(CAU Kiel) ins Leben gerufen wurde.Die Veranstaltung ist ein offenes Forum für alle interessierten deutschsprachigen Wissenschaftlerinnen und Wissenschaftler zum zwanglosen Austausch neuer Ideen und Ergebnisse aus den Forschungsbereichen Entwurf und Implementierung von Programmiersprachen sowie Grundlagen und Methodik des Programmierens. Dieser Tagungsband enthält die wissenschaftlichen Beiträge,die bei dem 21. Kolloquium dieser Tagungsreihe präsentiert wurden, welches vom 27. bis 29. September 2021 in Kiel stattfand und von der Arbeitsgruppe Programmiersprachen und Übersetzerkonstruktion der Christian-Albrechts-Universität zu Kiel organisiert wurde

Proceedings of Monterey Workshop 2001 Engineering Automation for Sofware Intensive System Integration

The 2001 Monterey Workshop on Engineering Automation for Software Intensive System Integration was sponsored by the Office of Naval Research, Air Force Office of Scientific Research, Army Research Office and the Defense Advance Research Projects Agency. It is our pleasure to thank the workshop advisory and sponsors for their vision of a principled engineering solution for software and for their many-year tireless effort in supporting a series of workshops to bring everyone together.This workshop is the 8 in a series of International workshops. The workshop was held in Monterey Beach Hotel, Monterey, California during June 18-22, 2001. The general theme of the workshop has been to present and discuss research works that aims at increasing the practical impact of formal methods for software and systems engineering. The particular focus of this workshop was "Engineering Automation for Software Intensive System Integration". Previous workshops have been focused on issues including, "Real-time & Concurrent Systems", "Software Merging and Slicing", "Software Evolution", "Software Architecture", "Requirements Targeting Software" and "Modeling Software System Structures in a fastly moving scenario".Office of Naval ResearchAir Force Office of Scientific Research Army Research OfficeDefense Advanced Research Projects AgencyApproved for public release, distribution unlimite