How do programs become more concurrent? A story of program transformations

For several decades, programmers have relied onMooreâ s Law to improve the performance of their softwareapplications. From now on, programmers need to programthe multi-cores if they want to deliver efficient code. Inthe multi-core era, a major maintenance task will be tomake sequential programs more concurrent. What are themost common transformations to retrofit concurrency intosequential programs?We studied the source code of 5 open-source Javaprojects. We analyzed qualitatively and quantitatively thechange patterns that developers have used in order toretrofit concurrency. We found that these transformationsbelong to four categories: transformations that improve thelatency, the throughput, the scalability, or correctness of theapplications. In addition, we report on our experience ofparallelizing one of our own programs. Our findings caneducate software developers on how to parallelize sequentialprograms, and can provide hints for tool vendors aboutwhat transformations are worth automating

Refactoring Sequential Java Code for Concurrency via Concurrent Libraries

Parallelizing existing sequential programs to run efficiently on multicores is hard. The Java 5 packagejava.util.concurrent (j.u.c.) supports writing concurrent programs: much of the complexity of writing threads-safe and scalable programs is hidden in the library. To use this package, programmers still need to reengineer existing code. This is tedious because it requires changing many lines of code, is error-prone because programmers can use the wrong APIs, and is omission-prone because programmers can miss opportunities to use the enhanced APIs. This paper presents our tool, CONCURRENCER, which enables programmers to refactor sequential code into parallel code that uses j.u.c. concurrent utilities. CONCURRENCER does not require any program annotations, although the transformations are very involved: they span multiple program statements and use custom program analysis. A find-and-replace tool can not perform such transformations. Empirical evaluation shows that CONCURRENCER refactors code effectively: CONCURRENCER correctly identifies and applies transformations that some open-source developers overlooked, and the converted code exhibits good speedup

Finding Bugs in Web Applications Using Dynamic Test Generation and Explicit State Model Checking

Web script crashes and malformed dynamically-generated web pages are common errors, and they seriously impact the usability of web applications. Current tools for web-page validation cannot handle the dynamically generated pages that are ubiquitous on today's Internet. We present a dynamic test generation technique for the domain of dynamic web applications. The technique utilizes both combined concrete and symbolic execution and explicit-state model checking. The technique generates tests automatically, runs the tests capturing logical constraints on inputs, and minimizes the conditions on the inputs to failing tests, so that the resulting bug reports are small and useful in finding and fixing the underlying faults. Our tool Apollo implements the technique for the PHP programming language. Apollo generates test inputs for a web application, monitors the application for crashes, and validates that the output conforms to the HTML specification. This paper presents Apollo's algorithms and implementation, and an experimental evaluation that revealed 302 faults in 6 PHP web applications

Finding Bugs In Dynamic Web Applications

Web script crashes and malformed dynamically-generated web pages are common errors, and they seriously impact usability of web applications. Currenttools for web-page validation cannot handle the dynamically-generatedpages that are ubiquitous on today's Internet.In this work, we apply a dynamic test generation technique, based oncombined concrete and symbolic execution, to the domain of dynamic webapplications. The technique generates tests automatically andminimizes the bug-inducing inputs to reduce duplication and to makethe bug reports small and easy to understand and fix.We implemented the technique in Apollo, an automated tool thatfound dozens of bugs in real PHP applications. Apollo generatestest inputs for the web application, monitors the application forcrashes, and validates that the output conforms to the HTMLspecification. This paper presents Apollo's algorithms andimplementation, and an experimental evaluation that revealed a totalof 214 bugs in 4 open-source PHP web applications

Examining User-Developer Feedback Loops in the iOS App Store

Application Stores, such as the iTunes App Store, give developers access to their users’ complaints and requests in the form of app reviews. However, little is known about how developers are responding to app reviews. Without such knowledge developers, users, App Stores, and researchers could build upon wrong foundations. To address this knowledge gap, in this study we focus on feedback loops, which occur when developers address a user concern. To conduct this study we use both supervised and unsupervised methods to automatically analyze a corpus of 1752 different apps from the iTunes App Store consisting of 30,875 release notes and 806,209 app reviews. We found that 18.7% of the apps in our corpus contain instances of feedback loops. In these feedback loops we observed interesting behaviors. For example, (i) feedback loops with feature requests and login issues were twice as likely as general bugs to be fixed by developers, (ii) users who reviewed with an even tone were most likely to have their concerns addressed, and (iii) the star rating of the app reviews did not influence the developers likelihood of completing a feedback loop

Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example

Software developers often repeat code changes, known as "code change patterns" (CPATs), within and across projects. Automating these CPATs accelerates development, but current Transformation by Example (TBE) techniques are limited by the input examples' quality and quantity, missing variations with different syntax or flow yet semantically similar. Large Language Models (LLMs), trained on vast code datasets, can overcome these limitations by generating semantically equivalent, unseen CPAT variants, enhancing TBE effectiveness. We identified best practices for using LLMs to generate code variants meeting criteria of correctness, usefulness, and applicability. Implementing these in PyCraft, combining static and dynamic analysis with LLMs, we achieved an F-measure of 96.6% in identifying correct variants, expanding inputs by 58x on average, and automating changes to increase target codes by up to 39x. Patches from PyCraft were submitted to projects like microsoft/DeepSpeed and IBM/inFairness, with an 83% acceptance rate, validating our approach's usefulness.Comment: This paper is accepted to Proceedings of the 32nd ACM Symposium on the Foundations of Software Engineering (FSE - 2024), This is an author cop

Are Web Applications Ready for Parallelism?

In recent years, web applications have become pervasive. Their backbone is JavaScript, the only programming language supported by all major web browsers. Most browsers run on desktop or mobile devices with parallel hardware. However, JavaScript is by design sequential, and current web applications make little use of hardware parallelism. Are web applications ready to exploit parallel hardware? \ \ To answer this question we take a two-step approach. First, we survey 174 web developers regarding the potential and challenges of using parallelism. Then, we study the performance and computation shape of a set of web applications that are representative for the emerging web. We identify performance bottlenecks and examine memory access patterns to determine possible data parallelism. \ \ Our findings indicate that emerging web applications do have latent data parallelism, and JavaScript developers\u27 programming style are not a significant impediment to exploiting this parallelism

Study and Refactoring of Android Asynchronous Programming

To avoid unresponsiveness, a core part of mobile development is asynchronous programming. Android provides several async constructs that developers can use. However, developers can still use the inappropriate async constructs, which result in memory leaks, lost results, and wasted energy. Fortunately, refactoring tools can eliminate these problems by transforming async code to use the appropriate constructs. In this paper we conducted a formative study on a corpus of 611 widely-used Android apps to map the asynchronous landscape of Android apps, understand how developers retrofit asynchrony, and learn about barriers encountered by developers. Based on this study, we designed, implemented, and evaluated ASYNCDROID, a refactoring tool which enables Android developers to transform existing improperly-used async constructs into correct constructs. Our empirical evaluation shows that ASYNCDROID is applicable, accurate, and saves developers effort. We submitted 30 refactoring patches, and developers consider that the refactorings are useful.Ope

Analysis and Transformations in Support of Android Privacy

To protect user’s privacy and system’s integrity, mobile platforms use permission models to control accesses to protected resources such as GPS location, Contacts, etc. The previous major version of Android used a static permission model, which compromised the security and privacy of apps. Android 6 overhauled its permission model to ask permissions at runtime which reduces the risk of permission abuse. However, migrating to the runtime permission model requires significant effort from the app developers. In this paper we first present a large-scale formative study to understand how app developers use and migrate to the new permission model. Inspired by these findings, we designed, implemented, and evaluated a tool suite that (i) recommends locations where to insert permission requests and (ii) automatically inserts all the permission-related code. Our empirical evaluations on a diverse corpus of real-world apps show that our tools are highly applicable and accurate.Keywords: software analysis, Android, permission

How do programs become more concurrent? A story of program transformations

ABSTRACT In the multi-core era, programmers need to resort to parallelism if they want to improve program performance. Thus, a major maintenance task will be to make sequential programs more concurrent. Must concurrency be designed into a program, or can it be retrofitted later? What are the most common transformations to retrofit concurrency into sequential programs? Are these transformations random, or do they belong to certain categories? How can we automate these transformations? To answer these questions we analyzed the source code of five open-source Java projects and looked at a total of 14 versions. We analyzed qualitatively and quantitatively the concurrency-related transformations. We found that these transformations belong to four categories: transformations that improve the responsiveness, the throughput, the scalability, or correctness of the applications. In 73.9% of these transformations, concurrency was retrofitted on existing program elements. In 20.5% of the transformations, concurrency was designed into new program elements. Our findings educate software developers on how to parallelize sequential programs, and provide hints for tool vendors about what transformations are worth automating