Finding Race Conditions in Erlang with Quick Check and PULSE

We address the problem of testing and debugging concurrent, distributed Erlang applications. In concurrent programs, race conditions are a common class of bugs and are very hard to find in practice. Traditional unit testing is normally unable to help finding all race conditions, because their occurrence depends so much on timing. Therefore, race conditions are often found during system testing, where due to the vast amount of code under test, it is often hard to diagnose the error resulting from race conditions. We present three tools (Quick Check, PULSE, and a visualizer) that in combination can be used to test and debug concurrent programs in unit testing with a much better possibility of detecting race conditions. We evaluate our method on an industrial concurrent case study and illustrate how we find and analyze the race conditions

Property-based testing for functional programs

This thesis advances the view that property-based testing is a powerful way of testing functional programs, that has advantages not shared by traditional unit testing. It does this by showing two new applications of property-based testing to functional programming as well as a study of the effectiveness of property-based testing.First, we present a tool, QuickSpec, which attempts to infer an equational specification from a functional program with the help of testing. The resulting specifications can be used to improve your understanding of the code or as properties in a test suite. The tool is applicable to quite a wide variety of situations.Second, we describe a system that helps to find race conditions in Erlang programs. It consists of two parts: a randomised scheduler to provoke unusual behaviour in the program under test and allow replayability of test cases, and a module that tests that all of the functions of an API behave atomically with respect to each other.Finally, we present an experiment we carried out to compare property-based testing against test-driven development. The results were inconclusive, but in the process we developed a black-box algorithm for automatically grading student programs by testing, by inferring for each program a set of bugs that the program contains

Property-based testing for functional programs

This thesis advances the view that property-based testing is a powerful way of testing functional programs, that has advantages not shared by traditional unit testing. It does this by showing two new applications of property-based testing to functional programming as well as a study of the effectiveness of property-based testing.First, we present a tool, QuickSpec, which attempts to infer an equational specification from a functional program with the help of testing. The resulting specifications can be used to improve your understanding of the code or as properties in a test suite. The tool is applicable to quite a wide variety of situations.Second, we describe a system that helps to find race conditions in Erlang programs. It consists of two parts: a randomised scheduler to provoke unusual behaviour in the program under test and allow replayability of test cases, and a module that tests that all of the functions of an API behave atomically with respect to each other.Finally, we present an experiment we carried out to compare property-based testing against test-driven development. The results were inconclusive, but in the process we developed a black-box algorithm for automatically grading student programs by testing, by inferring for each program a set of bugs that the program contains

Lightweight verification of functional programs

We have built several tools to help with testing and verifying functional programs. All three tools are based on QuickCheck properties. Our goal is to allow programmers to do more with QuickCheck properties than just test them.The first tool is QuickSpec, which finds equational specifications, and can be used to help with writing a specification or for program understanding. On top of QuickSpec, we have built HipSpec, which proves properties about Haskell programs, and uses QuickSpec to prove the necessary lemmas. We also describe PULSE and eqc_par_statem, which together can be used to find race conditions in Erlang programs.We believe that testable properties are a good basis for reasoning and verification, and that they give many of the benefits of formal verification without the cost of proof. The chief reason is that they are formal specifications for which the programmer can always get a counterexample when they are false. Furthermore, using testable properties allows us to write better tools. None of our tools would be possible if our properties were not testable.We also present work on encoding types in first-order logic, an essential component when using first-order provers to reason about programs. Our encodings are simple but extremely efficient, as evidenced by benchmarks. We develop the theory behind sound type encodings, and have written tools that implement our ideas

Lightweight verification of functional programs

We have built several tools to help with testing and verifying functional programs. All three tools are based on QuickCheck properties. Our goal is to allow programmers to do more with QuickCheck properties than just test them.The first tool is QuickSpec, which finds equational specifications, and can be used to help with writing a specification or for program understanding. On top of QuickSpec, we have built HipSpec, which proves properties about Haskell programs, and uses QuickSpec to prove the necessary lemmas. We also describe PULSE and eqc_par_statem, which together can be used to find race conditions in Erlang programs.We believe that testable properties are a good basis for reasoning and verification, and that they give many of the benefits of formal verification without the cost of proof. The chief reason is that they are formal specifications for which the programmer can always get a counterexample when they are false. Furthermore, using testable properties allows us to write better tools. None of our tools would be possible if our properties were not testable.We also present work on encoding types in first-order logic, an essential component when using first-order provers to reason about programs. Our encodings are simple but extremely efficient, as evidenced by benchmarks. We develop the theory behind sound type encodings, and have written tools that implement our ideas

Lightweight verification of functional programs

We have built several tools to help with testing and verifying functional programs. All three tools are based on QuickCheck properties. Our goal is to allow programmers to do more with QuickCheck properties than just test them.The first tool is QuickSpec, which finds equational specifications, and can be used to help with writing a specification or for program understanding. On top of QuickSpec, we have built HipSpec, which proves properties about Haskell programs, and uses QuickSpec to prove the necessary lemmas. We also describe PULSE and eqc_par_statem, which together can be used to find race conditions in Erlang programs.We believe that testable properties are a good basis for reasoning and verification, and that they give many of the benefits of formal verification without the cost of proof. The chief reason is that they are formal specifications for which the programmer can always get a counterexample when they are false. Furthermore, using testable properties allows us to write better tools. None of our tools would be possible if our properties were not testable.We also present work on encoding types in first-order logic, an essential component when using first-order provers to reason about programs. Our encodings are simple but extremely efficient, as evidenced by benchmarks. We develop the theory behind sound type encodings, and have written tools that implement our ideas