Towards sound refactoring in erlang

Erlang is an actor-based programming language used extensively for building concurrent, reactive systems that are highly available and suff er minimum downtime. Such systems are often mission critical, making system correctness vital. Refactoring is code restructuring that improves the code but does not change behaviour. While using automated refactoring tools is less error-prone than performing refactorings manually, automated refactoring tools still cannot guarantee that the refactoring is correct, i.e., program behaviour is preserved. This leads to lack of trust in automated refactoring tools. We rst survey solutions to this problem proposed in the literature. Erlang refactoring tools as commonly use approximation techniques which do not guarantee behaviour while some other works propose the use of formal methodologies. In this work we aim to develop a formal methodology for refactoring Erlang code. We study behavioural preorders, with a special focus on the testing preorder as it seems most suited to our purpose.peer-reviewe

Combining testing and runtime verification

Testing and runtime verification are intimately related: runtime verification enables testing of systems beyond their deployment by monitoring them under normal use while testing is not only concerned with monitoring the behaviour of systems but also generat- ing test cases which are able sufficiently cover their behaviour. Given this link between testing and runtime verification, one is surprised to find that in the literature the two have not been well studied in each other’s context. Below we outline three ways in which this can be done: one where testing can be used to support runtime verification, another where the two techniques can be used together in a single tool, and a third approach where runtime verification can be used to support testing.peer-reviewe

Equivalence proofs for Erlang refactoring

Erlang is an actor-based programming language used extensively for building concurrent, reactive systems that are highlighly available and suffer minimum downtime. Such systems are often mission critical, making system correctness vital. In industrial-scale systems, correctness is usually ascertained through testing, a lightweight verification technique trading analysis completeness for scalability. In such cases, a system is deemed correct whenever it “passes” a suite of tests, each checking for the correct functionality of a particular aspect of a system. This is also true for large Erlang systems: even when correctness specifications are provided, it is commonplace for Erlang developers to use testing tools, automating test-case generation from these specifications.peer-reviewe

Linking Unit Tests and Properties

QuickCheck allows us to verify software against particular proper- ties. A property can be regarded as an abstraction over many unit tests. QuickCheck uses generated random input data to test such properties. If a counterexample is found, it becomes immediately clear what we have tested. This is not the case when all tests pass, since we do not (and shall not) see the actual generated test cases. How can we be sure about what is tested? QuickCheck has the ability to gather statistics about the test cases, which is insightful. But still it does not tell us whether the particular unit test scenarios we have in mind are included. For this reason, we have developed a tool that can answer this question. It checks if a given unit test can be generated by a property, making it easier to judge the property’s quality. We have applied our tool to an industrial use case of testing the AUTOSAR basic software modules and shows that it can handle complex models and large unit tests

Towards property-based testing of RESTful web services

Developing APIs as Web Services over HTTP implies adding an extra layer to software, compared to the ones that we would need to develop an API distributed as, for example, a library. This additional layer must be included in testing too, but this implies that the software under test has an additional complexity due both to the need to use an intermediate protocol in tests and to the need to test compliance with the constraints imposed by that protocol: in this case the constraints defined by the REST architectural style. On the other hand, these requirements are common to all the Web Services, and because of that, we should be able to abstract this aspect of the testing model so that we can reuse it in testing any Web Service. In this paper, as a first step towards automating the testing of Web Services over HTTP, we describe a practical mechanism and model for testing RESTful Web Services without side effects and give an example of how we successfully adapted that mechanism to test two different existing Web Services: Storage Room by Thriventures and Google Tasks by Google. For this task we have used Erlang together with state machine models in the property-based testing tool Quviq QuickCheck, implemented using the statem module. 1

Validation of SDN policies: a property-based testing perspective

[Abstract] Software-defined networks are being widely adopted and used in large and complex networks supporting critical operations. Their increasing importance highlights the need for effective validation of SDN topologies and routing policies both prior and during operation. The policies that configure an SDN deployment come from several, possibly conflicting sources. This may lead to undesired effects such as node isolation, network partitions, performance drops and routing loops. Such effects can be formulated as automatically testable reusable conditions using property-based testing (PBT). This approach allows to automatically determine and formulate as a counterexample the minimum set of conflicting rules. The approach is especially useful when policies are configured in an incremental manner. PBT techniques are particularly good at automatic counterexample shrinking and have the potential of being extremely effective in this area.Swedish Foundationfor Strategic Research; RIT17-0035

Lifelong verification of software systems

Computers systems are increasingly interacting with our day-to-day life, but for this interaction to be facilitating and supporting, rather than interfering with our actions, these systems have to be dependable and trustworthy. The area of system verification and validation has a long history in computer science, but scaling up existing approaches to complex and large real-life systems is still an open-ended research question. In this paper we summarise and relate several ongoing research projects and tool development efforts in this field taking place within the Department of Computer Science.peer-reviewe

Automating property-based testing of evolving web services

Web services are the most widely used service technology that drives the Service-Oriented Computing~(SOC) paradigm. As a result, effective testing of web services is getting increasingly important. In this paper, we present a framework and toolset for testing web services and for evolving test code in sync with the evolution of web services. Our approach to testing web services is based on the Erlang programming language and QuviQ QuickCheck, a property-based testing tool written in Erlang, and our support for test code evolution is added to Wrangler, the Erlang refactoring tool. The key components of our system include the automatic generation of initial test code, the inference of web service interface changes between versions, the provision of a number of domain specific refactorings and the automatic generation of refactoring scripts for evolving the test code. Our framework provides users with a powerful and expressive web service testing framework, while minimising users' effort in creating, maintaining and evolving the test model. The framework presented in this paper can be used by both web service providers and consumers, and can be used to test web services written in whatever language; the approach advocated here could also be adopted in other property-based testing frameworks and refactoring tools

Automated Derivation of Random Generators for Algebraic Data Types

Many testing techniques such as generational fuzzing or random property-based testing require the existence of some sort of random generation process for the values used as test inputs. Implementing such generators is usually a task left to end-users, who do their best to come up with somewhat sensible implementations after several iterations of trial and error. This necessary effort is of no surprise, implementing good random data generators is a hard task. It requires deep knowledge about both the domain of the data being generated, as well as the behavior of the stochastic process generating such data. In addition, when the data we want to generate has a large number of possible variations, this process is not only intricate, but also very cumbersome. To mitigate this issues, this thesis explores different ideas for automatically deriving random generators based on existing static information. In this light, we design and implement different derivation algorithms in Haskell for obtaining random generators of values encoded using Algebraic Data Types (ADTs). Although there exists other tools designed directly or indirectly for this very purpose, they are not without disadvantages. In particular, we aim to tackle the lack of flexibility and static guarantees in the distribution induced by derived generators. We show how automatically derived generators for ADTs can be framed using a simple yet powerful stochastic model. This models can be used to obtain analytical guarantees about the distribution of values produced by the derived generators. This, in consequence, can be used to optimize the stochastic generation parameters of the derived generators towards target distributions set by the user, providing more flexible derivation mechanisms