A Context-Sensitive Coverage Criterion for Test Suite Reduction

Modern software is increasingly developed using multi-language implementations, large supporting libraries and frameworks, callbacks, virtual function calls, reflection, multithreading, and object- and aspect-oriented programming. The predominant example of such software is the graphical user interface (GUI), which is used as a front-end to most of today's software applications. The characteristics of GUIs and other modern software present new challenges to software testing. Because recently developed techniques for automated test case generation can generate more tests than are practical to regularly execute, one important challenge is test suite reduction. Test suite reduction seeks to decrease the size of a test suite without overly compromising its original fault detection ability. This research advances the state-of-the-art in test suite reduction by empirically studying a coverage criterion which considers the context in which program concepts are covered. Conventional approaches to test suite reduction were developed and evaluated on batch-style applications and, due to the aforementioned considerations, are not always easily applicable to modern software. Furthermore, many existing techniques fail to consider the context in which code executes inside an event-driven paradigm, where programs wait for and interactively respond to user- and system-generated events. Consequently, they yield reduced test suites with severely impaired fault detection ability. The novel feature of this research is a test suite reduction technique based on the call stack coverage criterion which addresses many of the challenges associated with coverage-based test suite reduction in modern applications. Results show that reducing test suites while maintaining call stack coverage yields good tradeoffs between size reduction and fault detection effectiveness compared to traditional techniques. The output of this research includes models, metrics, algorithms, and techniques based upon this approach

Migration of Applications across Object-Oriented APIs

Software developers often encapsulate reusable code as Application Programming Interfaces (APIs). The co-evolution of applications and APIs may motivate an API migration: the replacement of application dependencies to an original API by dependencies to an alternative API that provides similar functionality and abstractions. In this dissertation, we investigate issues associated with API migration in object-oriented systems, with special focus on wrapping approaches. We present two studies and a set of developer interviews that elicit issues in the process and techniques used in API migration in practice. The results suggest that the most pressing issues relate to discovery and specification of differences between APIs, and to assessment of migration correctness. This dissertation introduces techniques and a method to address these issues. We propose the use of design patterns to support the specification of API wrappers. API wrapping design patterns encode solutions to common wrapping design problems. We present an initial catalog of such patterns that were abstracted from programming idioms found in existing API wrappers. We introduce the concept of compliance testing for API migration, a form of automated testing. Compliance testing supports the discovery of behavioral differences between a wrapper and its corresponding original API, as well as assessment of wrapper correctness. Compliance testing uses API contracts and assertion tunings to explicitly capture and enforce the notion of a “good enough” wrapper that is informal in practice. We present the Koloo method for wrapper-based API migration. The method prescribes practical steps to use compliance testing as a means to elicit the requirements for the API migration, and to assess its correctness. Koloo fits within the iterative, sample-driven general API migration process usually followed by developers in practice. We evaluate the Koloo method in an empirical study. The subjects cover the domains of XML processing, GUI programming and bytecode engineering. The results provide evidence that Koloo is superior to alternative methods in driving the development of a wrapper that is tailored for the application under migration. The results also show that API contracts help driving the evolution of the wrapper, and assertion tuning is necessary to relax the semantics of strict equality contracts, and useful to compromise on features that are difficult to emulate perfectly. Finally, we validate that the proposed design patterns are used in practical wrappers