Structuring Documentation to Support State Search: A Laboratory Experiment about Protocol Programming

Abstract. Application Programming Interfaces (APIs) often define object protocols. Objects with protocols have a finite number of states and in each state a different set of method calls is valid. Many researchers have developed protocol verification tools because protocols are notoriously difficult to follow correctly. However, recent research suggests that a major challenge for API protocol programmers is effectively searching the state space. Verification is an ineffective guide for this kind of search. In this paper we instead propose Plaiddoc, which is like Javadoc except it organizes methods by state instead of by class and it includes explicit state transitions, state-based type specifications, and rich state relationships. We compare Plaiddoc to a Javadoc control in a betweensubjects laboratory experiment. We find that Plaiddoc participants complete state search tasks in significantly less time and with significantly fewer errors than Javadoc participants

A self-healing framework for general software systems

Modern systems must guarantee high reliability, availability, and efficiency. Their complexity, exacerbated by the dynamic integration with other systems, the use of third- party services and the various different environments where they run, challenges development practices, tools and testing techniques. Testing cannot identify and remove all possible faults, thus faulty conditions may escape verification and validation activities and manifest themselves only after the system deployment. To cope with those failures, researchers have proposed the concept of self-healing systems. Such systems have the ability to examine their failures and to automatically take corrective actions. The idea is to create software systems that can integrate the knowledge that is needed to compensate for the effects of their imperfections. This knowledge is usually codified into the systems in the form of redundancy. Redundancy can be deliberately added into the systems as part of the design and the development process, as it occurs for many fault tolerance techniques. Although this kind of redundancy is widely applied, especially for safety- critical systems, it is however generally expensive to be used for common use software systems. We have some evidence that modern software systems are characterized by a different type of redundancy, which is not deliberately introduced but is naturally present due to the modern modular software design. We call it intrinsic redundancy. This thesis proposes a way to use the intrinsic redundancy of software systems to increase their reliability at a low cost. We first study the nature of the intrinsic redundancy to demonstrate that it actually exists. We then propose a way to express and encode such redundancy and an approach, Java Automatic Workaround, to exploit it automatically and at runtime to avoid system failures. Fundamentally, the Java Automatic Workaround approach replaces some failing operations with other alternative operations that are semantically equivalent in terms of the expected results and in the developer’s intent, but that they might have some syntactic difference that can ultimately overcome the failure. We qualitatively discuss the reasons of the presence of the intrinsic redundancy and we quantitatively study four large libraries to show that such redundancy is indeed a characteristic of modern software systems. We then develop the approach into a prototype and we evaluate it with four open source applications. Our studies show that the approach effectively exploits the intrinsic redundancy in avoiding failures automatically and at runtime

Coping with the reality: adding crucial features to a typestate-oriented language

Detecting programming errors and vulnerabilities in software is increasingly important, and building tools that help with this task is an area of investigation, crucial for the industry these days. When programming in an object-oriented language, one naturally defines stateful objects that are non-uniform, i.e., their methods’ availability depends on their internal state. One might represent their intended usage protocol with an automaton or a state machine. Behavioral types allow to statically check if all the code of a program respects the usage protocol of each object. In this thesis we present a tool that extends Java with typestate definitions. These typestates are associated with Java classes and define the behavior of instances of those classes, specifying the sequences of method calls allowed. This tool checks statically that method calls happen in order, following the specified behavior. The tool was implemented in Kotlin as a plugin for the Checker Framework. It is a new implementation of the Mungo tool and supports prevention of null pointer errors, state transitions depending on return values, assurance of protocol completion, droppable states, and association of protocols with classes from the standard Java library or from third-party libraries. Additionally, the tool integrates behavioral types with access permissions, allowing objects to be shared in a controlled way using a language of assertions. This language of assertions supports concepts like packing and unpacking, including unpacking of aliases objects, and transferring of permissions between aliases. To relieve the programmer from manually writing all the necessary assertions, the tool implements an inference algorithm which analyzes the code statically and, given the uses of objects, constructs all the required assertions.A deteção de erros de programação e vulnerabilidades no software é cada vez mais importante, e a criação de ferramentas que ajudem nesta tarefa é uma área de investigação crucial para a indústria atualmente. Ao programar numa linguagem orientada a objetos, definem-se naturalmente objetos com estado que não são uniformes, ou seja, a disponibilidade dos seus métodos depende do seu estado interno. Pode-se representar o protocolo de uso pretendido com um autómato ou uma máquina de estados. Os tipos comportamentais permitem verificar estaticamente se todo o código de um programa respeita o protocolo de uso de cada objeto. Nesta tese apresentamos uma ferramenta que estende o Java com definições de typestates. Esses estão associados às classes Java e definem o comportamento das instâncias dessas classes, especificando as sequências de chamadas de métodos permitidas. Esta ferramenta verifica estaticamente se as chamadas de métodos ocorrem pela ordem correta, seguindo o comportamento especificado. A ferramenta foi implementada em Kotlin como um plugin para o Checker Framework. É uma implementação nova da ferramenta Mungo e suporta a prevenção de erros de ponteiro nulo, transições de estado dependendo de valores de retorno, asseguração da conclusão dos protocolos, objetos que podem ser «largados», e a associação de protocolos com classes da biblioteca padrão do Java ou de terceiros. Além disso, esta integra tipos comportamentais com permissões de acesso, permitindo que objetos possam ser partilhados por meio de uma linguagem de asserções. Esta linguagem de asserções oferece suporte para conceitos como packing e unpacking, incluindo unpacking de objetos partilhados, e transferência de permissões entre variáveis que apontam para o mesmo objeto. Para aliviar o programador de escrever manualmente todas as asserções necessárias, a ferramenta implementa um algoritmo de inferência que analisa o código estaticamente e, consoante os usos dos objetos, constrói todas as asserções necessárias

Generating Effective Test Suites for Reactive Systems using Specification Mining

Failures in reactive embedded systems are often unacceptable. Effective test-ing of embedded systems to detect such unacceptable failures is a difficult task. We present an automated black box test suite generation technique for embedded systems. The technique is based on dynamic mining of specifications, in the form of a finite state machine (FSM), from initial runs. The set of test cases thus produced may contain several redundant test cases. Many of the redundant test cases are then eliminated by an aggressive greedy test suite reduction algorithm to yield the final test suite. The tests generated by our technique were evaluated for their effectiveness on five case studies from the embedded domain. The evaluation of the results indicate that a test suite generated by our technique is promising in terms of effectiveness and scales easily. Further, the test suite reduction algorithm may sometimes remove non-redundant test cases too. Therefore, in our experimentation, we have also evaluated the change in the effectiveness of test suites due to this reduction. In this thesis, we describe the test suite generation and reduction technique in detail and present the results of the case studies