Annual Report on CHOReOS Dissemination - 1st year (D9.3.1)

This report summarizes achievement of the CHOReOS project in terms of disseminating project's goals and results during the first year. It further provides links to the concrete material that has been disseminated so far, hence enabling the interested reader to get access to the published material to know more about CHOReOS

How do programs become more concurrent? A story of program transformations

ABSTRACT In the multi-core era, programmers need to resort to parallelism if they want to improve program performance. Thus, a major maintenance task will be to make sequential programs more concurrent. Must concurrency be designed into a program, or can it be retrofitted later? What are the most common transformations to retrofit concurrency into sequential programs? Are these transformations random, or do they belong to certain categories? How can we automate these transformations? To answer these questions we analyzed the source code of five open-source Java projects and looked at a total of 14 versions. We analyzed qualitatively and quantitatively the concurrency-related transformations. We found that these transformations belong to four categories: transformations that improve the responsiveness, the throughput, the scalability, or correctness of the applications. In 73.9% of these transformations, concurrency was retrofitted on existing program elements. In 20.5% of the transformations, concurrency was designed into new program elements. Our findings educate software developers on how to parallelize sequential programs, and provide hints for tool vendors about what transformations are worth automating

FaCoY - A Code-to-Code Search Engine

Code search is an unavoidable activity in software development. Various approaches and techniques have been explored in the literature to support code search tasks. Most of these approaches focus on serving user queries provided as natural language free-form input. However, there exists a wide range of use-case scenarios where a code-to-code approach would be most beneficial. For example, research directions in code transplantation, code diversity, patch recommendation can leverage a code-to-code search engine to find essential ingredients for their techniques. In this paper, we propose FaCoY, a novel approach for statically finding code fragments which may be semantically similar to user input code. FaCoY implements a query alternation strategy: instead of directly matching code query tokens with code in the search space, FaCoY first attempts to identify other tokens which may also be relevant in implementing the functional behavior of the input code. With various experiments, we show that (1) FaCoY is more effective than online code-to-code search engines; (2) FaCoY can detect more semantic code clones (i.e., Type-4) in BigCloneBench than the state-of-theart; (3) FaCoY, while static, can detect code fragments which are indeed similar with respect to runtime execution behavior; and (4) FaCoY can be useful in code/patch recommendation

μ-DSU:A Micro-Language Based Approach to Dynamic Software Updating

Today software systems play a critical role in society’s infrastructures and many are required to provide uninterrupted services in their constantly changing environments. As the problem domain and the operational context of such software changes, the software itself must be updated accordingly. In this paper we propose to support dynamic software updating through language semantic adaptation; this is done through use of micro-languages that confine the effect of the introduced change to specific application features. Micro-languages provide a logical layer over a programming language and associate an application feature with the portion of the programming language used to implement it. Thus, they permit to update the application feature by updating the underlying programming constructs without affecting the behaviour of the other application features. Such a linguistic approach provides the benefit of easy addition/removal of application features (with a special focus on non-functional features) to/from a running application by separating the implementation of the new feature from the original application, allowing for the application to remain unaware of any extensions. The feasibility of this approach is demonstrated with two studies; its benefits and drawbacks are also analysed

Reuse-Based Test Recommendation in Software Engineering

Still today, the development of effective and high-quality software tests is an expensive and very labor intensive process. It demands a high amount of problem awareness, domain knowledge and concentration from human software testers. Therefore, any technology that can help reduce the manual effort involved in the software testing process -- while ensuring at least the same level of quality -- has the potential to significantly reduce software development and maintenance costs. In this dissertation, we present a new idea for achieving this by reusing the knowledge bound up in existing tests. Over the last two decades, software reuse and code recommendation has received a wide variety of attention in academia and industry, but the research conducted in this area to date has focused on the reuse of application code rather than on the reuse of tests. By switching this focus, this thesis paves the way for the automated extraction of test data and knowledge from previous software projects. In particular, it presents a recommendation approach for software tests that leverages lessons learned from traditional software reuse to make test case reuse suggestions to software engineers while they are working. In contrast to most existing testing-assistance tools, which provide ex post assistance to test developers in the form of coverage assessments and test quality evaluations, our approach offers an automated, proactive, non-intrusive test recommendation system for efficient software test development

Automatic Detection and Repair of Input Validation and Sanitization Bugs

A crucial problem in developing dependable web applications is thecorrectness of the input validation and sanitization. Bugs in stringmanipulation operations used for validation and sanitization are common,resulting in erroneous application behavior and vulnerabilities that areexploitable by malicious users. In this dissertation, we investigate theproblem of automatic detection and repair of validation and sanitization bugsboth at the client-side (JavaScript) and the server-side (PHP or Java) code.We first present a formal model for input validation and sanitizationfunctions along with a new domain specific intermediate languageto represent them. Then, we show how to extract input validation andsanitization functions in our intermediate language from both client andserver-side code in web applications. After the extraction phase, we useautomata-based static string-analysis techniques to automatically verifyand fix the extracted functions. One of our contributions is the developmentof efficient automata-based string analysis techniques for frequently used,complex string operations.We developed two basic approaches to bug detection and repair: 1)policy-based, and 2) differential. In the policy-based approach, inputvalidation and sanitization policies are expressed using two regularexpressions, one specifying the maximum policy (the upper bound for theset of strings that should be allowed) and the other specifying the minimumpolicy (the lower bound for the set of strings that should be allowed). Usingour string analysis techniques we can identify two types of errors inan input validation and sanitization function: 1) it accepts a set of strings thatis not permitted by the maximum policy (i.e., it is under-constrained),or 2) it rejects a set of strings that is permitted by the minimum policy(i.e., it is over-constrained).Our differential bug detection and repair approach does not require anypolicy specifications. It exploits the fact that, in web applications,developers typically perform redundant input validation and sanitizationin both the client and the server-side since client-side checks canbe by-passed. Using automata-based string analysis, we compare theinput validation and sanitization functions extracted from the client- andserver-side code, and identify and report the inconsistencies between them.Finally, we present an automated differential repair technique that canrepair client and server-side code with respect to each other, or acrossapplications in order to strengthen the validation and sanitizationchecks. Given a reference and a target function, our differential repairtechnique strengthens the validation and sanitization operations in thetarget function based on the reference function by automatically generatinga set of patches.We experimented with a number of real world web applications and found manybugs and vulnerabilities. Our analysis generates counter-example behaviorsdemonstrating the detected bugs and vulnerabilities to help the developerswith the debugging process. Moreover, we automatically generate patchesthat can be used to mitigate the detected bugs and vulnerabilities untildevelopers write their own patches

Rétro ingénierie d'applications web javascript pour aider à la compréhension et à la documentation

Ce mémoire s’intéresse à la rétro-ingénierie comme solution pour aider les développeurs à comprendre, modifier et documenter la structure de leurs applications web. Pour retrouver la structure d’une application, il faut souvent recourir à de l’analyse statique du code source pour retrouver les différents éléments et les différentes relations qui composent l’application. Le développement web présente ici des défis particuliers puisqu’il fait intervenir plusieurs langages. Certains de ces langages, comme HTML et CSS sont relativement simples; d’autres le sont moins. En particulier, JavaScript, un langage clé de la technologie Web, présente des aspects dynamiques importants (p. ex.: typage dynamique, évaluation dynamique de chaines de caractères) qui pourraient rendre très inefficace une analyse statique du code source. En effet, la récupération des éléments constituant l’application et de leurs liens pourrait devoir nécessiter une analyse dite dynamique qui se ferait sur des scénarios d’exécution. Cependant, de telles analyses dynamiques ne garantissent pas une couverture complète de l’application et ne peuvent se faire que si le code est exécutable. Nous avons donc conduit une étude empirique sur la viabilité de l’analyse statique pour la rétro-ingénierie de JavaScript. Forts de ces résultats ainsi que des constats sur les techniques et outils existants, nous proposons nos propres pistes de solutions sous forme d’une nouvelle approche de rétro-ingénierie (Web application Viewer). Cet outil est subséquemment utilisé pour performer des expérimentations de visualisation de structure à l’aide de diagrammes de force dirigée et diagrammes de classes. L’outil de rétro-ingénierie créé permet d’extraire les principaux éléments de la structure d’une application web pour les langages JavaScript, Node.js, HTML et CSS. Les résultats sont satisfaisants et permettent au développeur de documenter leurs applications rapidement à l’aide de diagrammes