Automated refactoring for Java concurrency

In multicore era, programmers exploit concurrent programming to gain performance and responsiveness benefits. However, concurrent programs are difficult to write: the programmer has to balance two conflicting forces, thread safety and performance. To make concurrent programming easier, modern programming languages provide many kinds of concurrent constructs, such as threads, asynchronous tasks, concurrent collections, etc. However, despite the existence of these concurrent constructs, we know little about how developers use them. On the other hand, although existing API documentation teach developers how to use concurrent constructs, developers can still misuse and underuse them. In this dissertation, we study the use, misuse, and underuse of two types of commonly used Java concurrent constructs: Java concurrent collections and Android async constructs. Our studies show that even though concurrent constructs are widely used in practice, developers still misuse and underuse them, causing semantic and performance bugs. We propose and develop a refactoring toolset to help developers correctly use concurrent constructs. The toolset is composed of three automated refactorings: (i) detecting and fixing the misuses of Java concurrent collections, (ii) retro fitting concurrency for existing sequential Android code via a basic Android async construct, and (iii) converting inappropriately used basic Android async constructs to appropriately enhanced constructs for Android apps. Refactorings (i) and (iii) aim to fix misused constructs while refactoring (ii) aims to eliminate underuses. First, we cataloged nine commonly misused check-then-act idioms of Java concurrent collections, and show the correct usage of each idiom. We implemented the detection strategies in a tool, CTADetector, that finds and fi xes misused check-then-act idioms. We applied CTADetector to 28 widely used open source Java projects (comprising 6.4 million lines of code) that use Java concurrent collections. CTADetector discovered and fixed 60 bugs. These bugs were con firmed by developers and the fixes were accepted. Second, we conducted a formative study on how a basic Android async construct, AsyncTask, is used, misused, and underused in Android apps. Based on the study, we designed, developed, and evaluated Asynchronizer, an automated refactoring tool that enables developers to retrofit concurrency into Android apps. The refactoring uses a points-to static analysis to determine the safety of the refactoring. We applied Asynchronizer to perform 123 refactorings in 19 widely used Android apps; their developers accepted 40 refactorings in 7 projects. Third, we conducted a formative study on a corpus of 611 widely-used Android apps to map the asynchronous landscape of Android apps, understand how developers retrofi t concurrency in Android apps, and learn about barriers encountered by developers. Based on this study, we designed, implemented, and evaluated AsyncDroid, a refactoring tool which enables Android developers to transform existing improperly-used async constructs into correct constructs. We submitted 45 refactoring patches generated by AsyncDroid in 7 widely used Android projects, and developers accepted 15 of them. Finally, we released all tools as open-source plugins for the widely used Eclipse IDE which has millions of Java users. Moreover, we also integrated CTADetector and AsyncDroid with a static analysis platform, ShipShape, that is developed by Google. Google envisions ShipShape to become a widely-used platform. Any app developer that wants to check code quality, for example before submitting an app to the app store, would run ShipShape on her code base. We expect that by contributing new async analyzers to ShipShape, millions of app developers would bene t by being able to execute our analysis and transformations on their code

Enabling Additional Parallelism in Asynchronous JavaScript Applications

JavaScript is a single-threaded programming language, so asynchronous programming is practiced out of necessity to ensure that applications remain responsive in the presence of user input or interactions with file systems and networks. However, many JavaScript applications execute in environments that do exhibit concurrency by, e.g., interacting with multiple or concurrent servers, or by using file systems managed by operating systems that support concurrent I/O. In this paper, we demonstrate that JavaScript programmers often schedule asynchronous I/O operations suboptimally, and that reordering such operations may yield significant performance benefits. Concretely, we define a static side-effect analysis that can be used to determine how asynchronous I/O operations can be refactored so that asynchronous I/O-related requests are made as early as possible, and so that the results of these requests are awaited as late as possible. While our static analysis is potentially unsound, we have not encountered any situations where it suggested reorderings that change program behavior. We evaluate the refactoring on 20 applications that perform file- or network-related I/O. For these applications, we observe average speedups ranging between 0.99% and 53.6% for the tests that execute refactored code (8.1% on average)

Systematic Asynchrony Bug Exploration for Android Apps

Abstract. Smartphone and tablet "apps" are particularly susceptible to asynchrony bugs. In order to maintain responsive user interfaces, events are handled asynchronously. Unexpected schedules of event handlers can result in apparently-random bugs which are notoriously difficult to reproduce, even given the user-event sequences that trigger them. We develop the AsyncDroid tool for the systematic discovery and reproduction of asynchrony bugs in Android apps. Given an app and a user-event sequence, AsyncDroid systematically executes alternate schedules of the same asynchronous event handlers, according to a programmable schedule enumerator. The input user-event sequence is given either by user interaction, or can be generated by automated ui "monkeys". By exposing and controlling the factors which influence the scheduling order of asynchronous handlers, our programmable enumerators can explicate reproducible schedules harboring bugs. By enumerating all schedules within a limited threshold of reordering, we maximize the likelihood of encountering asynchrony bugs, according to prevailing hypotheses in the literature, and discover several bugs in Android apps found in the wild

Safe Automated Refactoring for Intelligent Parallelization of Java 8 Streams

Streaming APIs are becoming more pervasive in mainstream Object-Oriented programming languages. For example, the Stream API introduced in Java 8 allows for functional-like, MapReduce-style operations in processing both finite and infinite data structures. However, using this API efficiently involves subtle considerations like determining when it is best for stream operations to run in parallel, when running operations in parallel can be less efficient, and when it is safe to run in parallel due to possible lambda expression side-effects. In this paper, we present an automated refactoring approach that assists developers in writing efficient stream code in a semantics-preserving fashion. The approach, based on a novel data ordering and typestate analysis, consists of preconditions for automatically determining when it is safe and possibly advantageous to convert sequential streams to parallel and unorder or de-parallelize already parallel streams. The approach was implemented as a plug-in to the Eclipse IDE, uses the WALA and SAFE analysis frameworks, and was evaluated on 11 Java projects consisting of ∼642K lines of code. We found that 57 of 157 candidate streams (36.31%) were refactorable, and an average speedup of 3.49 on performance tests was observed. The results indicate that the approach is useful in optimizing stream code to their full potential

Safe Automated Refactoring for Intelligent Parallelization of Java 8 Streams

Streaming APIs are becoming more pervasive in mainstream Object-Oriented programming languages and platforms. For example, the Stream API introduced in Java 8 allows for functional-like, MapReduce-style operations in processing both finite, e.g., collections, and infinite data structures. However, using this API efficiently involves subtle considerations such as determining when it is best for stream operations to run in parallel, when running operations in parallel can be less efficient, and when it is safe to run in parallel due to possible lambda expression side-effects. ics-preserving fashion. The approach, based on a novel data ordering and typestate analysis, consists of preconditions and transformations for automatically determining when it is safe and possibly advantageous to convert sequential streams to parallel and unorder or de-parallelize already parallel streams. The approach was implemented as a plug-in to the popular Eclipse IDE, uses the WALA and SAFE analysis frameworks, and was evaluated on 18 Java projects consisting of ∼1.65M lines of code. We found that 116 of 419 candidate streams (27.68%) were refactorable, and an average speedup of 3.49 on performance tests was observed. The results indicate that the approach is useful in optimizing stream code to their full potential

Multi-Objective Improvement of Android Applications

Non-functional properties, such as runtime or memory use, are important to mobile app users and developers, as they affect user experience. Previous work on automated improvement of non-functional properties in mobile apps failed to address the inherent trade-offs between such properties. We propose a practical approach and the first open-source tool, GIDroid (2023), for multi-objective automated improvement of Android apps. In particular, we use Genetic improvement, a search-based technique that navigates the space of software variants to find improved software. We use a simulation-based testing framework to greatly improve the speed of search. GIDroid contains three state-of-the-art multi-objective algorithms, and two new mutation operators, which cache the results of method calls. Genetic improvement relies on testing to validate patches. Previous work showed that tests in open-source Android applications are scarce. We thus wrote tests for 21 versions of 7 Android apps, creating a new benchmark for performance improvements. We used GIDroid to improve versions of mobile apps where developers had previously found improvements to runtime, memory, and bandwidth use. Our technique automatically re-discovers 64% of existing improvements. We then applied our approach to current versions of software in which there were no known improvements. We were able to improve execution time by up to 35%, and memory use by up to 33% in these apps.Comment: 32 pages, 8 Figure

Enabling modern concurrency through program transformations

Concurrency is becoming the norm in modern software because multicores are now everywhere. Developers use concurrency constructs (i) to make their software responsive and scalable via asynchrony and (ii) to increase the throughput of their software via parallelism. Yet, we know little about how developers use these constructs in practice. Without such knowledge, (i) other developers cannot educate themselves about the state of the practice, (ii) language and library designers are unaware of any misuse, (iii) researchers make wrong assumptions, and (iv) tool providers do not provide the tools that developers really need. We claim that concurrent programming deserves first-class citizenship in empirical research and tool support. In this dissertation, we study the use, misuse, and underuse of concurrency constructs for C#. Based on our large-scale empirical studies, we found two main problems. First, developers still use complex, slow, low-level, and old constructs. Second, when developers use modern concurrency constructs, they highly misuse them, causing severe consequences such as subtle data-races and swallowed exceptions. Some significantly degrade the performance of async programs or can even deadlock the program. Other misuses make developers erroneously think that their code runs sequentially, when in reality the code runs concurrently; this causes unintended behavior and wrong results. This dissertation presents four tools to enable C# developers (i) to migrate their software to modern concurrency constructs and (ii) to fix misused modern constructs. First, we present an automated migration tool, Taskifier, that transforms old style Thread and ThreadPool constructs to higher-level Task constructs. Second, we present a refactoring tool, Simplifier, that extracts and converts Task-related code snippets into higher-level parallel patterns. Third, we provide Asyncifier to developers who want to refactor their callback-based asynchronous constructs to new async/await keywords. Fourth, we developed AsyncFixer, a static analysis tool that detects and corrects 14 common kinds of async/await misuses that cannot be detected by any existing tools. We released all our tools as plugins for the widely used Visual Studio IDE which has millions of users. We conducted both quantitative and qualitative evaluation of our tools. We applied the tools thousands of times over the real-world software and we show that they (i) are highly applicable, (ii) are much safer than manual transformations, and (iii) fast enough to be used interactively in Visual Studio. Our tools are useful in practice for both industry and open-source communities. Developers of the open-source software accepted 408 patches which are generated by the tools. We also received positive feedback from the early adopters. In industry, our tools are embraced by two companies. They started to actively use AsyncFixer in their automated build process. Our empirical findings have (i) educated thousands of developers through our educational web portals and (ii) influenced the design of modern constructs across programming languages. We hope that our dissertation serves as a call to action for the testing, program verification, and program analysis communities to study new constructs from programming languages and standard libraries. Researchers can empirically study these constructs and provide necessary tools to help developers adopt them. Hence, we also present a set of guidelines for researchers that aim to make a practical impact on the usage of new programming constructs