JWalk: a tool for lazy, systematic testing of java classes by design introspection and user interaction

Popular software testing tools, such as JUnit, allow frequent retesting of modified code; yet the manually created test scripts are often seriously incomplete. A unit-testing tool called JWalk has therefore been developed to address the need for systematic unit testing within the context of agile methods. The tool operates directly on the compiled code for Java classes and uses a new lazy method for inducing the changing design of a class on the fly. This is achieved partly through introspection, using Java’s reflection capability, and partly through interaction with the user, constructing and saving test oracles on the fly. Predictive rules reduce the number of oracle values that must be confirmed by the tester. Without human intervention, JWalk performs bounded exhaustive exploration of the class’s method protocols and may be directed to explore the space of algebraic constructions, or the intended design state-space of the tested class. With some human interaction, JWalk performs up to the equivalent of fully automated state-based testing, from a specification that was acquired incrementally

Fostering design by contract by exploiting the relationship between code commentary and contracts.

Embora contratos no estilo de programação por contratos (DBC) tragam precisão para expressar o comportamento do código, desenvolvedores são resistentes ao seu uso. Há várias razões para isto, tais como a dificuldade na produção de contratos ou o trabalho de manter os contratos consistentes com o código em evolução. Por outro lado, Javadoc é uma abordagem comumente usada para documentar programas Java. Mesmo assim, comentários Javadoc não servem para a checagem automática de conformidade devido à ambiguidade inerente à linguagem natural. Neste trabalho, procuramos minimizar a distância entre contratos e Javadoc, estimulando a adoção de DBC a partir de duas contribuições principais; primeiro, propomos uma extensão ao sistema de tags do Javadoc (CONTRACTJ DOC) para possibilitar a integração de contratos na notação de comentários; então, propomos uma abordagem para geração de contratos a partir de comentários em linguagem natural (CONTRACT SUGGESTOR). Nós realizamos três avaliações: primeiro, avaliamos a aplicabilidade e a compreensibilidade de CONTRACTJ DOC. Como resultados, detectamos inconsistências entre a documentação Javadoc e o código fonte. A maioria dos contratos que escrevemos foram checagens de valores limítrofes para parâmetros e repetições de expressões de retorno de métodos. Além disso, a legibilidade dos comentários percebida pelos desenvolvedores não diferiu significativamente entre as abordagens, o que é promissor, dado que contratos são usualmente classificados como difíceis de ler. Segundo, avaliamos a qualidade dos contratos gerados por CONTRACT SUGGESTOR verificando a taxa de falsos positivos gerados. Como resultado, são gerados mais contratos corretos para non-null do que para relational, devido a quantidade de instâncias de comentários para cada propriedade. Por fim, realizamos estudos de caso com JMLOK2 e C ONTRACTOK – CONTRACTOK é uma extensão da abordagem de J MLOK2 para o contexto C#/Code Contracts. Primeiro, usamos JMLOK2 para verificar os contratos gerados automaticamente por CONTRACT SUGGESTOR; depois usamos as ferramentas para verificar 24 sistemas de código aberto (12 para cada ferramenta). As ferramentas detectaram 188 não-conformidades, sendo 72 problemas de pós-condição e 61 de invariante; as causas prováveis mais comuns foram Pré-condição fraca (91) e Erro de código (56). Com isso, objetivamos motivar a adoção de DBC como forma de aprimorar o projeto dos programas, e por consequência, sua qualidade geral.Contracts in Design by Contract style bring about preciseness for expressing the code behavior; however, developers are resistant to their use. There are several likely reasons for this, such as the trouble to conceive good, useful contracts, or the burden of maintaining contracts consistent with the evolving code. On the other hand, Javadoc is a common way of documenting Java programs. Nevertheless, Javadoc comments do not serve to an automated conformance checking due to ambiguity issues inherent to the natural languages. In this work, we try to minimize the distance between contracts and Javadoc, fostering DBC adoption by means of two main contributions; first, we propose an extension to the Javadoc tagging system (C ONTRACTJD OC) for allowing the integration of contracts into the comments notation; then, we propose an approach for automatically generating contracts based on natural language code commentary (CONTRACTSUGGESTOR). We perform three evaluations: first, we evaluate the applicability and comprehensibility of C ONTRACTJD OC. As results, we detected inconsistencies between the documentation available by means of Javadoc comments and the source code. The majority of the contracts we could write from the comments remains between common-case and repetitive with the code. Moreover, developers’ impression about the readability of comments did not differ significantly, which is promising, as contracts are usually regarded as hard to read – one reason for its non-adoption. Then, we evaluate the quality of contracts generated by CONTRACTSUGGESTOR by analyzing the false positives rate. As result, the approach generates more correct contracts for non-null than for relational, due to the number of comment instances for each property. Finally, we perform case studies with J MLOK2 and C ONTRACTOK – C ONTRACTOK is an extension of JMLOK2 for C#/Code Contracts context. First, we used J MLOK2 for conformance checking the contracts automatically generated; then, we run the tools over 24 open-source systems (12 with each tool). The tools detected 188 nonconformances. From those, 72 are postcondition and 61 are invariant problems; with respect to likely causes manually established, Weak precondition (91) and Code error (56) are the most commons. With this, we aim to promote DBC adoption as a way for improving the design of the projects, and consequently, their quality in general.Cape

Systematic techniques for more effective fault localization and program repair

Debugging faulty code is a tedious process that is often quite expensive and can require much manual effort. Developers typically perform debugging in two key steps: (1) fault localization, i.e., identifying the location of faulty line(s) of code; and (2) program repair, i.e., modifying the code to remove the fault(s). Automating debugging to reduce its cost has been the focus of a number of research projects during the last decade, which have introduced a variety of techniques. However, existing techniques suffer from two basic limitations. One, they lack accuracy to handle real programs. Two, they focus on automating only one of the two key steps, thereby leaving the other key step to the developer. Our thesis is that an approach that integrates systematic search based on state-of-the-art constraint solvers with techniques to analyze artifacts that describe application specific properties and behaviors, provides the basis for developing more effective debugging techniques. We focus on faults in programs that operate on structurally complex inputs, such as heap-allocated data or relational databases. Our approach lays the foundation for a unified framework for localization and repair of faults in programs. We embody our thesis in a suite of integrated techniques based on propositional satisfiability solving, correctness specifications analysis, test-spectra analysis, and rule-learning algorithms from machine learning, implement them as a prototype tool-set, and evaluate them using several subject programs.Electrical and Computer Engineerin

Proceedings of the 22nd Conference on Formal Methods in Computer-Aided Design – FMCAD 2022

The Conference on Formal Methods in Computer-Aided Design (FMCAD) is an annual conference on the theory and applications of formal methods in hardware and system verification. FMCAD provides a leading forum to researchers in academia and industry for presenting and discussing groundbreaking methods, technologies, theoretical results, and tools for reasoning formally about computing systems. FMCAD covers formal aspects of computer-aided system design including verification, specification, synthesis, and testing

Proceedings of the 22nd Conference on Formal Methods in Computer-Aided Design – FMCAD 2022

The Conference on Formal Methods in Computer-Aided Design (FMCAD) is an annual conference on the theory and applications of formal methods in hardware and system verification. FMCAD provides a leading forum to researchers in academia and industry for presenting and discussing groundbreaking methods, technologies, theoretical results, and tools for reasoning formally about computing systems. FMCAD covers formal aspects of computer-aided system design including verification, specification, synthesis, and testing

Combining data structure repair and program repair

textBugs in code continue to pose a fundamental problem for software reliability and cause expensive failures. The process of removing known bugs is termed debugging, which is a classic methodology commonly performed before code is deployed. Traditionally, debugging is tedious, often requiring much manual effort. A more recent technique that complements debugging is data structure repair, which handles bugs that make it to deployed systems and lead to erroneous behavior at runtime by modifying erroneous program states to recover from errors. While data structure repair presents a promising basis for dealing with bugs at runtime, it remains computationally expensive. Our thesis is that debugging and data structure repair can be integrated to provide the basis of an effective approach for removing bugs before code is deployed and handling them after it is deployed. We present a bi-directional integration where ideas at the basis of data structure repair assist in automating debugging and vice versa. Our key insight is two-fold: (1)a repair action performed to mutate an erroneous object field value to repair it can be abstracted into a program statement that performs that update correctly; and (2)repair actions that are performed repeatedly to fix the same error can be memoized and re-used. We design, develop, and evaluate two techniques that embody our insight. One, we present an automated debugging technique that leverages a systematic constraint-based data structure repair technique developed in previous work and provides suggestions on how to fix a faulty program. Two, we present repair abstractions that are based on the same central ideas as in our automated debugging technique and memoize how an erroneous state was repaired, which enables prioritizing and re-using repair actions when the same error occurs again. The focus of our work is programs that operate on structurally complex data, e.g., heap-allocated data structures that have complex structural integrity constraints, such as acyclicity. Checking such constraints plays a central role in the techniques that lay at the foundation of our work. These techniques require the user to provide the constraints, which poses a burden on the user. To facilitate the use of constraint-based techniques, we present a third technique to check constraint violations at runtime using graph spectra, which have been studied extensively by mathematicians to capture properties of graphs. We view the heap of an object-oriented program as an edge-labeled graph, which allows us to apply results from graph spectra theory. Experimental results show the effectiveness of using graph spectra as a basis of capturing structural properties of a class of commonly used data structures.Electrical and Computer Engineerin

Computer Aided Verification

The open access two-volume set LNCS 11561 and 11562 constitutes the refereed proceedings of the 31st International Conference on Computer Aided Verification, CAV 2019, held in New York City, USA, in July 2019. The 52 full papers presented together with 13 tool papers and 2 case studies, were carefully reviewed and selected from 258 submissions. The papers were organized in the following topical sections: Part I: automata and timed systems; security and hyperproperties; synthesis; model checking; cyber-physical systems and machine learning; probabilistic systems, runtime techniques; dynamical, hybrid, and reactive systems; Part II: logics, decision procedures; and solvers; numerical programs; verification; distributed systems and networks; verification and invariants; and concurrency

Learning Program Specifications from Sample Runs

With science fiction of yore being reality recently with self-driving cars, wearable computers and autonomous robots, software reliability is growing increasingly important. A critical pre-requisite to ensure the software that controls such systems is correct is the availability of precise specifications that describe a program\u27s intended behaviors. Generating these specifications manually is a challenging, often unsuccessful, exercise; unfortunately, existing static analysis techniques often produce poor quality specifications that are ineffective in aiding program verification tasks. In this dissertation, we present a recent line of work on automated synthesis of specifications that overcome many of the deficiencies that plague existing specification inference methods. Our main contribution is a formulation of the problem as a sample driven one, in which specifications, represented as terms in a decidable refinement type representation, are discovered from observing a program\u27s sample runs in terms of either program execution paths or input-output values, and automatically verified through the use of expressive refinement type systems. Our approach is realized as a series of inductive synthesis frameworks, which use various logic-based or classification-based learning algorithms to provide sound and precise machine-checked specifications. Experimental results indicate that the learning algorithms are both efficient and effective, capable of automatically producing sophisticated specifications in nontrivial hypothesis domains over a range of complex real-world programs, going well beyond the capabilities of existing solutions

Computer Aided Verification

This open access two-volume set LNCS 11561 and 11562 constitutes the refereed proceedings of the 31st International Conference on Computer Aided Verification, CAV 2019, held in New York City, USA, in July 2019. The 52 full papers presented together with 13 tool papers and 2 case studies, were carefully reviewed and selected from 258 submissions. The papers were organized in the following topical sections: Part I: automata and timed systems; security and hyperproperties; synthesis; model checking; cyber-physical systems and machine learning; probabilistic systems, runtime techniques; dynamical, hybrid, and reactive systems; Part II: logics, decision procedures; and solvers; numerical programs; verification; distributed systems and networks; verification and invariants; and concurrency