Automated migration of build scripts using dynamic analysis and search-based refactoring

The efficiency of a build system is an important factor for developer productivity. As a result, developer teams have been increasingly adopting new build systems that allow higher build parallelization. However, migrating the existing legacy build scripts to new build systems is a tedious and error-prone process. Unfortunately, there is insufficient support for automated migration of build scripts, making the migration more problematic. We propose the first dynamic approach for automated migration of build scripts to new build systems. Our approach works in two phases. First, from a set of execution traces, we synthesize build scripts that accurately capture the intent of the original build. The synthesized build scripts are typically long and hard to maintain. Second, we apply refactorings that raise the abstraction level of the synthesized scripts (e.g., introduce functions for similar fragments). As different refactoring sequences may lead to different build scripts, we use a search-based approach that explores various sequences to identify the best (e.g., shortest) build script. We optimize search-based refactoring with partial-order reduction to faster explore refactoring sequences. We implemented the proposed two phase migration approach in a tool called METAMORPHOSIS that has been recently used at Microsoft

Identifying Bugs in Make and JVM-Oriented Builds

Incremental and parallel builds are crucial features of modern build systems. Parallelism enables fast builds by running independent tasks simultaneously, while incrementality saves time and computing resources by processing the build operations that were affected by a particular code change. Writing build definitions that lead to error-free incremental and parallel builds is a challenging task. This is mainly because developers are often unable to predict the effects of build operations on the file system and how different build operations interact with each other. Faulty build scripts may seriously degrade the reliability of automated builds, as they cause build failures, and non-deterministic and incorrect build results. To reason about arbitrary build executions, we present buildfs, a generally-applicable model that takes into account the specification (as declared in build scripts) and the actual behavior (low-level file system operation) of build operations. We then formally define different types of faults related to incremental and parallel builds in terms of the conditions under which a file system operation violates the specification of a build operation. Our testing approach, which relies on the proposed model, analyzes the execution of single full build, translates it into buildfs, and uncovers faults by checking for corresponding violations. We evaluate the effectiveness, efficiency, and applicability of our approach by examining hundreds of Make and Gradle projects. Notably, our method is the first to handle Java-oriented build systems. The results indicate that our approach is (1) able to uncover several important issues (245 issues found in 45 open-source projects have been confirmed and fixed by the upstream developers), and (2) orders of magnitude faster than a state-of-the-art tool for Make builds

A heuristic-based approach to code-smell detection

Encapsulation and data hiding are central tenets of the object oriented paradigm. Deciding what data and behaviour to form into a class and where to draw the line between its public and private details can make the difference between a class that is an understandable, flexible and reusable abstraction and one which is not. This decision is a difficult one and may easily result in poor encapsulation which can then have serious implications for a number of system qualities. It is often hard to identify such encapsulation problems within large software systems until they cause a maintenance problem (which is usually too late) and attempting to perform such analysis manually can also be tedious and error prone. Two of the common encapsulation problems that can arise as a consequence of this decomposition process are data classes and god classes. Typically, these two problems occur together – data classes are lacking in functionality that has typically been sucked into an over-complicated and domineering god class. This paper describes the architecture of a tool which automatically detects data and god classes that has been developed as a plug-in for the Eclipse IDE. The technique has been evaluated in a controlled study on two large open source systems which compare the tool results to similar work by Marinescu, who employs a metrics-based approach to detecting such features. The study provides some valuable insights into the strengths and weaknesses of the two approache

Rohelisema tarkvaratehnoloogia poole tarkvaraanalüüsi abil

Mobiilirakendused, mis ei tühjenda akut, saavad tavaliselt head kasutajahinnangud. Mobiilirakenduste energiatõhusaks muutmiseks on avaldatud mitmeid refaktoreerimis- suuniseid ja tööriistu, mis aitavad rakenduse koodi optimeerida. Neid suuniseid ei saa aga seoses energiatõhususega üldistada, sest kõigi kontekstide kohta ei ole piisavalt energiaga seotud andmeid. Olemasolevad energiatõhususe parandamise tööriistad/profiilid on enamasti prototüübid, mis kohalduvad ainult väikese alamhulga energiaga seotud probleemide suhtes. Lisaks käsitlevad olemasolevad suunised ja tööriistad energiaprobleeme peamiselt a posteriori ehk tagantjärele, kui need on juba lähtekoodi sees. Android rakenduse koodi saab põhijoontes jagada kaheks osaks: kohandatud kood ja korduvkasutatav kood. Kohandatud kood on igal rakendusel ainulaadne. Korduvkasutatav kood hõlmab kolmandate poolte teeke, mis on rakendustesse lisatud arendusprotessi kiirendamiseks. Alustuseks hindame mitmete lähtekoodi halbade lõhnade refaktoreerimiste energiatarbimist Androidi rakendustes. Seejärel teeme empiirilise uuringu Androidi rakendustes kasutatavate kolmandate osapoolte võrguteekide energiamõju kohta. Pakume üldisi kontekstilisi suuniseid, mida võiks rakenduste arendamisel kasutada. Lisaks teeme süstemaatilise kirjanduse ülevaate, et teha kindlaks ja uurida nüüdisaegseid tugitööriistu, mis on rohelise Androidi arendamiseks saadaval. Selle uuringu ja varem läbi viidud katsete põhjal toome esile riistvarapõhiste energiamõõtmiste jäädvustamise ja taasesitamise probleemid. Arendame tugitööriista ARENA, mis võib aidata koguda energiaandmeid ja analüüsida Androidi rakenduste energiatarbimist. Viimasena töötame välja tugitööriista REHAB, et soovitada arendajatele energiatõhusaid kolmanda osapoole võrguteekeMobile apps that do not drain the battery usually get good user ratings. To make mobile apps energy efficient many refactoring guidelines and tools are published that help optimize the app code. However, these guidelines cannot be generalized w.r.t energy efficiency, as there is not enough energy-related data for every context. Existing energy enhancement tools/profilers are mostly prototypes applicable to only a small subset of energy-related problems. In addition, the existing guidelines and tools mostly address the energy issues a posteriori, i.e., once they have already been introduced into the code. Android app code can be roughly divided into two parts: the custom code and the reusable code. Custom code is unique to each app. Reusable code includes third-party libraries that are included in apps to speed up the development process. We start by evaluating the energy consumption of various code smell refactorings in native Android apps. Then we conduct an empirical study on the energy impact of third-party network libraries used in Android apps. We provide generalized contextual guidelines that could be used during app development Further, we conduct a systematic literature review to identify and study the current state of the art support tools available to aid green Android development. Based on this study and the experiments we conducted before, we highlight the problems in capturing and reproducing hardware-based energy measurements. We develop the support tool ‘ARENA’ that could help gather energy data and analyze the energy consumption of Android apps. Last, we develop the support tool ‘REHAB’ to recommend energy efficient third-party network libraries to developers.https://www.ester.ee/record=b547174

What Java Developers Know About Compatibility, And Why This Matters

Real-world programs are neither monolithic nor static -- they are constructed using platform and third party libraries, and both programs and libraries continuously evolve in response to change pressure. In case of the Java language, rules defined in the Java Language and Java Virtual Machine Specifications define when library evolution is safe. These rules distinguish between three types of compatibility - binary, source and behavioural. We claim that some of these rules are counter intuitive and not well-understood by many developers. We present the results of a survey where we quizzed developers about their understanding of the various types of compatibility. 414 developers responded to our survey. We find that while most programmers are familiar with the rules of source compatibility, they generally lack knowledge about the rules of binary and behavioural compatibility. This can be problematic when organisations switch from integration builds to technologies that require dynamic linking, such as OSGi. We have assessed the gravity of the problem by studying how often linkage-related problems are referenced in issue tracking systems, and find that they are common

On the Feasibility of Transfer-learning Code Smells using Deep Learning

Context: A substantial amount of work has been done to detect smells in source code using metrics-based and heuristics-based methods. Machine learning methods have been recently applied to detect source code smells; however, the current practices are considered far from mature. Objective: First, explore the feasibility of applying deep learning models to detect smells without extensive feature engineering, just by feeding the source code in tokenized form. Second, investigate the possibility of applying transfer-learning in the context of deep learning models for smell detection. Method: We use existing metric-based state-of-the-art methods for detecting three implementation smells and one design smell in C# code. Using these results as the annotated gold standard, we train smell detection models on three different deep learning architectures. These architectures use Convolution Neural Networks (CNNs) of one or two dimensions, or Recurrent Neural Networks (RNNs) as their principal hidden layers. For the first objective of our study, we perform training and evaluation on C# samples, whereas for the second objective, we train the models from C# code and evaluate the models over Java code samples. We perform the experiments with various combinations of hyper-parameters for each model. Results: We find it feasible to detect smells using deep learning methods. Our comparative experiments find that there is no clearly superior method between CNN-1D and CNN-2D. We also observe that performance of the deep learning models is smell-specific. Our transfer-learning experiments show that transfer-learning is definitely feasible for implementation smells with performance comparable to that of direct-learning. This work opens up a new paradigm to detect code smells by transfer-learning especially for the programming languages where the comprehensive code smell detection tools are not available

30 Years of Software Refactoring Research: A Systematic Literature Review

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/155872/4/30YRefactoring.pd

30 Years of Software Refactoring Research:A Systematic Literature Review

Due to the growing complexity of software systems, there has been a dramatic increase and industry demand for tools and techniques on software refactoring in the last ten years, defined traditionally as a set of program transformations intended to improve the system design while preserving the behavior. Refactoring studies are expanded beyond code-level restructuring to be applied at different levels (architecture, model, requirements, etc.), adopted in many domains beyond the object-oriented paradigm (cloud computing, mobile, web, etc.), used in industrial settings and considered objectives beyond improving the design to include other non-functional requirements (e.g., improve performance, security, etc.). Thus, challenges to be addressed by refactoring work are, nowadays, beyond code transformation to include, but not limited to, scheduling the opportune time to carry refactoring, recommendations of specific refactoring activities, detection of refactoring opportunities, and testing the correctness of applied refactorings. Therefore, the refactoring research efforts are fragmented over several research communities, various domains, and objectives. To structure the field and existing research results, this paper provides a systematic literature review and analyzes the results of 3183 research papers on refactoring covering the last three decades to offer the most scalable and comprehensive literature review of existing refactoring research studies. Based on this survey, we created a taxonomy to classify the existing research, identified research trends, and highlighted gaps in the literature and avenues for further research.Comment: 23 page

Best Practices for Implementing Agile Methods: A Guide for Department of Defense Software Developers

Traditional plan-driven software development has been widely used in the government because it\u27s considered to be less risky, more consistent, and structured. But there has been a shift from this approach to Agile methods which are more flexible, resulting in fast releases by working in an incremental fashion to adapt to the reality of the changing or unclear requirements. This report describes the Agile software development philosophy, methods, and best practices in launching software design projects using the Agile approach. It is targeted to Defense Department software developers because they face broad challenges in creating enterprise-wide information systems, where Agile methods could be used most effectively. Though not a panacea, agile methods offer a solution to an important class of problems faced by organizations today. Technology and E-Government