A Study of Concurrency Bugs and Advanced Development Support for Actor-based Programs

The actor model is an attractive foundation for developing concurrent applications because actors are isolated concurrent entities that communicate through asynchronous messages and do not share state. Thereby, they avoid concurrency bugs such as data races, but are not immune to concurrency bugs in general. This study taxonomizes concurrency bugs in actor-based programs reported in literature. Furthermore, it analyzes the bugs to identify the patterns causing them as well as their observable behavior. Based on this taxonomy, we further analyze the literature and find that current approaches to static analysis and testing focus on communication deadlocks and message protocol violations. However, they do not provide solutions to identify livelocks and behavioral deadlocks. The insights obtained in this study can be used to improve debugging support for actor-based programs with new debugging techniques to identify the root cause of complex concurrency bugs.Comment: - Submitted for review - Removed section 6 "Research Roadmap for Debuggers", its content was summarized in the Future Work section - Added references for section 1, section 3, section 4.3 and section 5.1 - Updated citation

Testing of Concurrent Programs

Testing concurrent systems requires exploring all possible non-deterministic interleavings that the concurrent execution may have, as any of the interleavings may reveal erroneous behaviour. This introduces a new problem: the well-known state space problem, which is often computationally intractable. In the present thesis, this issue will be addressed through: (1) the development of new Partial-Order Reduction Techniques and (2) the combination of static analysis and testing (property-based testing) in order to reduce the combinatorial explosion. As a preliminary result, we have performed an experimental evaluation on the SYCO tool, a CLP-based testing framework for actor-based concurrency, where these techniques have been implemented. Finally, our experiments prove the effectiveness and applicability of the proposed techniques

Systematic Testing for Detecting Concurrency Errors in Erlang Programs

We present the techniques used in Concuerror, a systematic testing tool able to find and reproduce a wide class of concurrency errors in Erlang programs. We describe how we take advantage of the characteristics of Erlang's actor model of concurrency to selectively instrument the program under test and how we subsequently employ a stateless search strategy to systematically explore the state space of process interleaving sequences triggered by unit tests. To ameliorate the problem of combinatorial explosion, we propose a novel technique for avoiding process blocks and describe how we can effectively combine it with preemption bounding, a heuristic algorithm for reducing the number of explored interleaving sequences. We also briefly discuss issues related to soundness, completeness and effectiveness of techniques used by Concuerror

Towards Automated Test Sequence Generation

The article presents a novel control-flow based test sequence generation technique using UML 2.0 activity diagram, which is a behavioral type of UML diagram. Like other model-based techniques, this technique can be used in the earlier phases of the development process owing to the availability of the design models of the system. The activity diagram model is seamlessly converted into a colored Petri net. We proposed a technique that enables the automatic generation of test sequences according to a given coverage criteria from the execution of the colored Petri nets model. Two types of structural coverage criteria for AD based models, namely sequential and concurrent coverage are described. The proposed technique was applied to an example to demonstrate its feasibility and the generated test sequences were evaluated against selected coverage criteria. This technique can potentially be adapted to service oriented applications, workflows, and concurrent applications

결정론적 스케쥴링을 이용하여 동시성 버그를 재현하는 커널 퍼져 제작

학위논문(석사) -- 서울대학교대학원 : 공과대학 전기·정보공학부, 2022. 8. 이병영.커널 퍼져들이 최근 몇 년간 발전해오면서 퍼져에 의해 보고되는 버그들의 수가 늘어났다. 커널 개발자들이 이런 버그들을 모두 분석할 수 없기 때문에 이런 상황은 디버깅 정보를 제공해줄 수 있는 버그 재현의 중요성을 더욱 중요하게 만든다. 불행 히도, 퍼져는 종종 버그 재현에 실패하는데 재현이 힘든 버그 중 하나가 바로 동시성 버그다. 동시성 버그는 여러 스레드 사이의 특정한 조건을 만족할때 발생하는데 커 널 스케쥴링의 비결정적인 스레드 간섭은 동시성 버그의 재현을 막는다. 그 결과, 일부 동시성 버그들은 버그의 재현에 실패한 뒤 고쳐지지 않은채 버려진다. 이 논문에서는, 퍼져에 의해 발견된 버그를 결정적으로 재현할 수 있도록 도와주 는 REPFUZZER 를 소개한다. 이는 선택적 스레드 추적과 결정적 스케쥴러를 통해 문 제를 해결한다. 선택적 스레드 추적을 통해 REPFUZZER 는 퍼져에 있어서 흥미로운 스레드에만 집중할 수 있도록 한다. 그리고 REPFUZZER 는 결정적 스케쥴러를 통해 선택된 스레드를 스케쥴 해주며 결정적인 스레드 간섭을 만들어 낸다. 퍼징 단계와 재현 단계 모두에서 REPFUZZER 를 사용함으로써 퍼져는 찾아낸 버그를 재현해낼 수 있게 된다. 결과적으로 REPFUZZER 는 15개의 실제 동시성 버그를 재현하면서 효율성을 보여줬으며, 버그 중 일부는 비결정적인 커널 스케쥴링으로 재현해내기 위해 엄청난 시간을 필요로 했다.As kernel fuzzer has been studied and becomes better for years, the number of reported bugs from the fuzzer increased. Since kernel developers could not analyze all of the bugs, the situation emphasizes the importance of bug reproduce which can provide debug information. Unfortunately, the fuzzer often fails to reproduce bug, and one of the most difficult bug type is concurrency bug. The concurrency bug requires certain conditions between several threads, and non-deterministic thread interleavings from kernel scheduling prevent reproducing. As a result, some concurrency bugs are left unpatched after failures of reproduce. In this thesis, we presents REPFUZZER which enables deterministic reproduce for bugs found by fuzzer. It solves the problem with selective thread tracing and deterministic scheduler. With selective thread tracing, REPFUZZER can focus on only interesting thread in fuzzing context. Then REPFUZZER schedules the selected threads with deterministic scheduler and produces deterministic thread interleavings. Using REPFUZZER’s scheduling at both fuzzing and reproduce phase, the fuzzer can reproduce bugs found at fuzzing phase. As a result, REPFUZZER shows its effectiveness for reproducing concurrency bugs with 15 real-world bugs, and some of the bugs require enormous times to be reproduced with non-deterministic kernel scheduling.Abstract i Contents ii List of Tables iv List of Figures v 1 INTRODUCTION 1 2 BACKGROUND 5 2.1 Fuzzing 5 2.2 Concurrency Bug 6 3 DESIGN 7 3.1 Design 7 3.1.1 Selective Thread Tracing 9 3.1.2 Deterministic Scheduler 12 3.2 Implementation 18 4 EVALUATION 20 4.1 Effectiveness of Deterministic Scheduler 20 4.2 Statistics of Deterministic Scheduler 23 4.3 Overhead of Deterministic Scheduler 25 5 DISCUSSION 27 6 RELATED WORKS 29 6.1 Kernel Concurrency Bugs 29 6.2 Reproducing Bugs 30 7 CONCLUSION 32 Abstract (In Korean) 37석

Software testing or the bugs’ nightmare

Software development is not error-free. For decades, bugs –including physical ones– have become a significant development problem requiring major maintenance efforts. Even in some cases, solving bugs led to increment them. One of the main reasons for bug’s prominence is their ability to hide. Finding them is difficult and costly in terms of time and resources. However, software testing made significant progress identifying them by using different strategies that combine knowledge from every single part of the program. This paper humbly reviews some different approaches from software testing that discover bugs automatically and presents some different state-of-the-art methods and tools currently used in this area. It covers three testing strategies: search-based methods, symbolic execution, and fuzzers. It also provides some income about the application of diversity in these areas, and common and future challenges on automatic test generation that still need to be addressed