Scaling Testing of Refactoring Engines

Defining and implementing refactorings is a nontrivial task since it is difficult to define preconditions to guarantee that the transformation preserves the program behavior. Therefore, refactoring engines may apply incorrect transformations in which the resulting program does not compile, preserve behavior, or follow the refactoring definitions. These engines may also prevent correct transformations due to overly strong preconditions. We find that 84% of the test suites of Eclipse and JRRT are concerned to detect those kinds of bugs. However, the engines still have them. Researchers have proposed a number of techniques for testing refactoring engines. Nevertheless, they may have limitations related to the bug type, program generation, time consumption, and number of refactoring engines necessary to evaluate the implementations. We propose and implement a technique to scale testing of refactoring engines. We improve expressiveness of a program generator and use a technique to skip some test inputs to improve performance. Moreover, we propose new oracles to detect behavioral changes using change impact analysis, overly strong preconditions by disabling preconditions, and transformation issues. We evaluate our technique in 28 refactoring implementations of Java (Eclipse and JRRT) and C (Eclipse) and find 119 bugs. The technique reduces the time in 96% using skips while missing only 6% of the bugs. Additionally, it finds the first failure in general in a few seconds using skips. Finally, we evaluate our proposed technique by using other test inputs, such as the input programs of Eclipse and JRRT refactoring test suites. We find 31 bugs not detected by the developers.Sociedad Argentina de Informática e Investigación Operativa (SADIO

Scaling testing of refactoring engines.

Definir e implementar refatoramentos não é uma tarefa trivial, pois é difícil definir todas as pré-condições necessárias para garantir que a transformação preserve o comportamento observável do programa. Com isso, ferramentas de refatoramentos podem ter condições muito fracas, condições muito fortes e podem aplicar transformações que não seguem a definição do refatoramento. Na prática, desenvolvedores escrevem casos de testes para checar suas implementações de refatoramentos e se preocupam em evitar esses tipos de bugs, pois 84% das asserções de testes do Eclipse e JRRT testam as ferramentas com relação aos bugs citados anteriormente. No entanto, as ferramentas ainda possuem esses bugs. Existem algumas técnicas automáticas para testar ferramentas de refatoramentos, mas elas podem ter limitações relacionadas com tipos de bugs que podem ser detectados, geração de entradas de testes, automação e performance. Este trabalho propõe uma técnica para escalar testes de ferramentas de refatoramentos. A técnica contém DOLLY um gerador automático de programas Java e C, no qual foram adicionadas mais construções de Java (classes e métodos abstratos e interface) e uma estratégia de pular algumas entradas de testes com o propósito de reduzir o tempo de testar as implementações de refatoramentos. Foi proposto um conjunto de oráculos para avaliar a corretude das transformações, dentre eles SAFEREFACTORIMPACT que identifica falhas relacionadas com mudanças comportamentais. SAFEREFACTORIMPACT gera testes apenas para os métodos impactados pela transformação. Além disso, foi proposto um novo oráculo para identificar transformações que não seguem a definição do refatoramento e uma nova técnica para identificar condições muito fortes. A técnica proposta foi avaliada em 28 implementações de refatoramentos de Java (Eclipse e JRRT) e C (Eclipse) e detectou 119 bugs relacionados com erros de compilação, mudanças comportamentais, condições muito fortes, e transformações que não seguem a definição do refatoramento. Usando pulos de 10 e 25 no gerador de programas, a técnica reduziu em 90% e 96% o tempo para testar as implementações de refatoramentos, enquanto deixou de detectar apenas 3% e 6% dos bugs, respectivamente. Além disso, detectou a primeira falha geralmente em alguns segundos. Por fim, com o objetivo de avaliar a técnica proposta com outras entradas de testes, foram avaliadas implementações do Eclipse e JRRT usando os programas de entrada das suas coleções de testes. Neste estudo, nossa técnica detectou mais 31 bugs não detectados pelos desenvolvedores das ferramentas.Defining and implementing refactorings is a nontrivial task since it is difficult to define preconditions to guarantee that the transformation preserves the program behavior. There fore, refactoring engines may have overly weak preconditions, overly strong preconditions, and transformation issues related to the refactoring definition. In practice, developers manually write test cases to check their refactoring implementations. We find that 84% of the test suites of Eclipse and JRRT are concerned with identifying these kinds of bugs. However, bugs are still present. Researchers have proposed a number of techniques for testing refactoring engines. Nevertheless, they may have limitations related to the bug type, program generation, time consumption, and number of refactoring engines necessary to evaluate the implementations. In this work, we propose a technique to scale testing of refactoring engines by extending a previous technique. It automatically generates programs as test inputs using Dolly, a Java and C program generator. We add more Java constructs in DOLLY, such abstract classes and methods and interface, and a skip parameter to reduce the time to test the refactoring implementations by skipping some consecutive test inputs. Our technique uses SAFEREFACTORIMPACT to identify failures related to behavioral changes. It generates test cases only for the methods impacted by a transformation. Also, we propose a new oracle to evaluate whether refactoring preconditions are overly strong by disabling a subset of them. Finally, we present a technique to identify transformation issues related to the refactoring definition. We evaluate our technique in 28 refactoring implementations of Java (Eclipse and JRRT) and C (Eclipse) and find 119 bugs related to compilation errors, behavioral changes, overly strong preconditions, and transformation issues. The technique reduces the time in 90% and 96% using skips of 10 and 25 in Dolly while missing only 3% and 6% of the bugs, respectively. Additionally, it finds the first failure in general in a few seconds using skips. Finally, we evaluate our proposed technique by using other test inputs, such as the input programs of Eclipse and JRRT refactoring test suites. We find 31 bugs not detected by the developers.Cape

On Preserving the Behavior in Software Refactoring: A Systematic Mapping Study

Context: Refactoring is the art of modifying the design of a system without altering its behavior. The idea is to reorganize variables, classes and methods to facilitate their future adaptations and comprehension. As the concept of behavior preservation is fundamental for refactoring, several studies, using formal verification, language transformation and dynamic analysis, have been proposed to monitor the execution of refactoring operations and their impact on the program semantics. However, there is no existing study that examines the available behavior preservation strategies for each refactoring operation. Objective: This paper identifies behavior preservation approaches in the research literature. Method: We conduct, in this paper, a systematic mapping study, to capture all existing behavior preservation approaches that we classify based on several criteria including their methodology, applicability, and their degree of automation. Results: The results indicate that several behavior preservation approaches have been proposed in the literature. The approaches vary between using formalisms and techniques, developing automatic refactoring safety tools, and performing a manual analysis of the source code. Conclusion: Our taxonomy reveals that there exist some types of refactoring operations whose behavior preservation is under-researched. Our classification also indicates that several possible strategies can be combined to better detect any violation of the program semantics

A Model Driven Approach for Refactoring Heterogeneous Software Artefacts

Refactoring is the process of transforming a software system to improve its overall structure while preserving its observable behaviour. Refactoring engines are normally used to perform these transformations for efficiency and in order to avoid introducing behavioural changes into the program due to human error. Although these engines do not verify that behaviour is preserved, it is widely accepted that automated transformations are less likely to introduce errors in comparison to manual refactoring. Despite the advantages provided by refactoring engines they fall foul of certain weaknesses. Here we hypothesise that Model Driven Engineering can be used to produce improved refactoring engines that are less vulnerable to those weaknesses. We develop a Domain Specific Transformation Language for defining new composite refactorings from a set of built–in primitives and to script their application. We also develop an interpreter for the language, effectively providing an operational semantics, in the guise of an extensible transformation framework. We evaluate our approach with a case study examining the correlation between actual and predicted measurements of the Coupling Between Objects metric for classes that undergo the extract class refactoring. The results show that our approach is promising

Reflective and relativistic refactoring with feature-awareness

Refactoring is a core technology in modern software development. It is central to popular software design movements, such as Extreme Programming [23] and Agile software development [91], and all major Integrated Development Environments (IDEs) today offer some form of refactoring support. Despite this, refactoring engines have languished behind research. Modern IDEs offer no means to sequence refactorings to automate program changes. Further, current refactoring engines exhibit problems of speed and expressivity, which makes writing composite refactorings such as design patterns infeasible. Even worse, existing refactoring tools for Object-Oriented languages are unaware of configurations in Software Product Lines (SPLs) codebases. With this motivation in mind, this dissertation makes three contributions to address these issues: First, we present the Java API library, called R2, to script Eclipse refactorings to retrofit design patterns into existing programs. We encoded 18 out of 23 design patterns described by Gang-of-Four [57] as R2 scripts and explain why the remaining refactorings are inappropriate for refactoring engines. R2 sheds light on why refactoring speed and expressiveness are critical issues for scripting. Second, we present a new Java refactoring engine, called R3, that addresses an Achilles heel in contemporary refactoring technology, namely scripting performance. Unlike classical refactoring techniques that modify Abstract Syntax Trees (ASTs), R3 refactors programs by rendering ASTs via pretty printing. AST rendering never changes the AST; it only displays different views of the AST/program. Coupled with new ways to evaluate refactoring preconditions, R3 increases refactoring speed by an order of magnitude over Eclipse and facilitates computing views of a program where the original behavior is preserved. Third, we provide a feature-aware refactoring tool, called X15, for SPL codebases written in Java. X15 takes advantage of R3's view rendering to implement a projection technology in Feature-Oriented Software Development, which produces subprograms of the original SPL by hiding unneeded feature code. X15 is the first feature-aware refactoring tool for Java that implements a theory of refactoring feature modules, and allows users to edit and refactor SPL programs via “views”. In the most demanding experiments, X15 barely runs a second slower than R3, giving evidence that refactoring engines for SPL codebases can indeed be efficient.Computer Science

Revisiting the refactoring names.

Refactoring é uma prática chave em metodologias ágeis utilizadas por vários desenvolvedores e disponível em IDEs profissionais. Existem livros e artigos que explicam os refactorings e analisam problemas relacionados aos nomes. Alguns trabalhos identificaram que os nomes de refactorings em ferramentas automatizadas de refactoring podem confundir os desenvolvedores. No entanto, não sabemos até que ponto os nomes dos refactorings são confusos no contexto de transformações de pequena granularidade. Neste trabalho, conduzimos um estudo de método misto a partir de diferentes perspectivas para entender melhor o significado dos nomes dos refactorings para desenvolvedores e desenvolvedores de ferramentas (implementações de refactorings e ferramentas de detecção de refactorings). No primeiro estudo, revisitamos os nomes dos refactorings através de uma pesquisa com 107 desenvolvedores de projetos Java populares no GitHub. Perguntamos a eles sobre o resultado de sete tipos de refatoração aplicados a pequenos programas. Esse estudo identifica que os desenvolvedores não esperam a mesma saída para todas as perguntas, mesmo usando pequenos programas Java como entrada. O significado dos nomes dos refactorings é baseado na experiência dos desenvolvedores para um número deles (71.02%). No segundo estudo, observamos até que ponto as implementações de refatoração têm o mesmo significado dos nomes dos refactorings. Aplicamos 10 tipos de refactorings em 157,339 programas usando 27 implementações de refactorings de três ferramentas, usando a mesma entrada e parâmetros, e comparando as saídas. Categorizamos as diferenças em 17 tipos que ocorrem em 9 de 10 tipos de refactorings implementados por Eclipse, NetBeans e JRRT. No terceiro estudo, comparamos o significado dos nomes dos refactorings usados em uma ferramenta (RMiner) que detecta refactorings com implementações de refactorings implementadas por três ferramentas. RMiner não produz o mesmo conjunto de refactorings aplicados pelas implementações do Eclipse, NetBeans e JRRT em 48.57%, 35% e 9.22% dos casos, respectivamente. Em geral, desenvolvedores e desenvolvedores de ferramentas usam diferentes significados para os nomes dos refactorings, e isso pode afetar a comunicação entre desenvolvedores e pesquisadores.Refactoring is a key practice in agile methodologies used by a number of developers, and available in professional IDEs. There are some books and papers explaining the refactoring names. Some works identified that the names of some automated refactoring tools are a distraction to developers. However, we do not know to what extent the refactoring names are confusing in the context of small-grained transformations. In this work, we conduct a mixedmethod study from different perspectives to better understand the meaning of refactoring names for developers, and tool developers (refactoring implementations, and refactoring detection tools). In the first study, we revisit the refactoring names by conducting a survey with 107 developers of popular Java projects on GitHub. We asked them about the output of seven refactoring types applied to small programs. It finds that developers do not expect the same output to all questions, even using small Java programs as input. The meaning of refactoring names is based on developers’ experience for a number of them (71.02%). In the second study, we observe to what extent refactoring implementations have the same meaning of the refactoring names. We apply 10 types of refactorings to 157,339 programs using 27 refactoring implementations from three tools using the same input and parameters, and compare the outputs. We categorize the differences into 17 types that occur in 9 out of 10 refactoring types implemented by Eclipse, NetBeans, and JRRT. In the third study, we compare the meaning of the refactoring names used in a tool (RMiner) that detects refactorings to refactoring implementations implemented by three tools. RMiner does not yield the same set of refactorings applied by implementations from Eclipse, NetBeans, and JRRT in 48.57%, 35%, and 9.22% of the cases, respectively. Overall, developers and tool developers use different meanings for refactoring names, and this may impact developers’ and researchers’ communication.Cape

State of Refactoring Adoption: Towards Better Understanding Developer Perception of Refactoring

Context: Refactoring is the art of improving the structural design of a software system without altering its external behavior. Today, refactoring has become a well-established and disciplined software engineering practice that has attracted a significant amount of research presuming that refactoring is primarily motivated by the need to improve system structures. However, recent studies have shown that developers may incorporate refactoring strategies in other development-related activities that go beyond improving the design especially with the emerging challenges in contemporary software engineering. Unfortunately, these studies are limited to developer interviews and a reduced set of projects. Objective: We aim at exploring how developers document their refactoring activities during the software life cycle. We call such activity Self-Affirmed Refactoring (SAR), which is an indication of the developer-related refactoring events in the commit messages. After that, we propose an approach to identify whether a commit describes developer-related refactoring events, to classify them according to the refactoring common quality improvement categories. To complement this goal, we aim to reveal insights into how reviewers develop a decision about accepting or rejecting a submitted refactoring request, what makes such review challenging, and how to the efficiency of refactoring code review. Method: Our empirically driven study follows a mixture of qualitative and quantitative methods. We text mine refactoring-related documentation, then we develop a refactoring taxonomy, and automatically classify a large set of commits containing refactoring activities, and identify, among the various quality models presented in the literature, the ones that are more in-line with the developer\u27s vision of quality optimization, when they explicitly mention that they are refactoring to improve them to obtain an enhanced understanding of the motivation behind refactoring. After that, we performed an industrial case study with professional developers at Xerox to study the motivations, documentation practices, challenges, verification, and implications of refactoring activities during code review. Result: We introduced SAR taxonomy on how developers document their refactoring strategies in commit messages and proposed a SAR model to automate the detection of refactoring. Our survey with code reviewers has revealed several difficulties related to understanding the refactoring intent and implications on the functional and non-functional aspects of the software. Conclusion: Our SAR taxonomy and model, can work in conjunction with refactoring detectors, to report any early inconsistency between refactoring types and their documentation and can serve as a solid background for various empirical investigations. In light of our findings of the industrial case study, we recommended a procedure to properly document refactoring activities, as part of our survey feedback

Combining Static and Dynamic Analysis for Bug Detection and Program Understanding

This work proposes new combinations of static and dynamic analysis for bug detection and program understanding. There are 3 related but largely independent directions: a) In the area of dynamic invariant inference, we improve the consistency of dynamically discovered invariants by taking into account second-order constraints that encode knowledge aboutinvariants; the second-order constraints are either supplied by the programmer or vetted by the programmer (among candidate constraints suggested automatically); b) In the area of testing dataflow (esp. map-reduce) programs, our tool, SEDGE, achieves higher testing coverage by leveraging existinginput data and generalizing them using a symbolic reasoning engine (a powerful SMT solver); c) In the area of bug detection, we identify and present the concept of residual investigation: a dynamic analysis that serves as theruntime agent of a static analysis. Residual investigation identifies with higher certainty whether an error reported by the static analysis is likely true