An empirical investigation into the impact of refactoring on regression testing

It is widely believed that refactoring improves software quality and developer’s productivity by making it easier to maintain and understand software systems. On the other hand, some believe that refactoring has the risk of functionality regression and increased testing cost. This paper investigates the impact of refactoring edits on regression tests using the version history of Java open source projects: (1) Are there adequate regression tests for refactoring in practice? (2) How many of existing regression tests are relevant to refactoring edits and thus need to be re-run for the new version? (3) What proportion of failure-inducing changes are relevant to refactorings? By using a refactoring reconstruction analysis and a change impact analysis in tandem, we investigate the relationship between the types and locations of refactoring edits identified by RefFinder and the affecting changes and affected tests identified by the FaultTracer change impact analysis. The results on three open source projects, JMeter, XMLSecurity, and ANT, show that only 22% of refactored methods and fields are tested by existing regression tests. While refactorings only constitutes 8% of atomic changes, 38% of affected tests are relevant to refactorings. Furthermore, refactorings are involved in almost a half of failed test cases. These results call for new automated regression test augmentation and selection techniques for validating refactoring edits.Electrical and Computer Engineerin

Handling High-Level Model Changes Using Search Based Software Engineering

Model-Driven Engineering (MDE) considers models as first-class artifacts during the software lifecycle. The number of available tools, techniques, and approaches for MDE is increasing as its use gains traction in driving quality, and controlling cost in evolution of large software systems. Software models, defined as code abstractions, are iteratively refined, restructured, and evolved. This is due to many reasons such as fixing defects in design, reflecting changes in requirements, and modifying a design to enhance existing features. In this work, we focus on four main problems related to the evolution of software models: 1) the detection of applied model changes, 2) merging parallel evolved models, 3) detection of design defects in merged model, and 4) the recommendation of new changes to fix defects in software models. Regarding the first contribution, a-posteriori multi-objective change detection approach has been proposed for evolved models. The changes are expressed in terms of atomic and composite refactoring operations. The majority of existing approaches detects atomic changes but do not adequately address composite changes which mask atomic operations in intermediate models. For the second contribution, several approaches exist to construct a merged model by incorporating all non-conflicting operations of evolved models. Conflicts arise when the application of one operation disables the applicability of another one. The essence of the problem is to identify and prioritize conflicting operations based on importance and context – a gap in existing approaches. This work proposes a multi-objective formulation of model merging that aims to maximize the number of successfully applied merged operations. For the third and fourth contributions, the majority of existing works focuses on refactoring at source code level, and does not exploit the benefits of software design optimization at model level. However, refactoring at model level is inherently more challenging due to difficulty in assessing the potential impact on structural and behavioral features of the software system. This requires analysis of class and activity diagrams to appraise the overall system quality, feasibility, and inter-diagram consistency. This work focuses on designing, implementing, and evaluating a multi-objective refactoring framework for detection and fixing of design defects in software models.Ph.D.Information Systems Engineering, College of Engineering and Computer ScienceUniversity of Michigan-Dearbornhttp://deepblue.lib.umich.edu/bitstream/2027.42/136077/1/Usman Mansoor Final.pdfDescription of Usman Mansoor Final.pdf : Dissertatio

Detection of Rename Local Variable Refactoring Instances in Commit History

Detecting refactoring instances occurred in successive revisions of software systems can provide wealthy information for several purposes, e.g., to facilitate the code review process, to devise more accurate code merging techniques, to help the developers of API clients to ease their adaptation to API changes, and to enable more accurate empirical studies on the refactoring practice. In the literature there are several techniques proposed for refactoring detection, supporting a wide variety of refactoring types. Yet, almost all of them have missed an extensively-applied refactoring type, i.e., Rename Local Variable refactoring. In addition, all these techniques rely on similarity thresholds (which are difficult to tune), or need the systems under analysis to be fully built (which is usually a daunting task), or depend on specific IDEs (which drastically limits their effectiveness and usability). In this thesis, we extend the state-of-the-art refactoring detection tool, RefactoringMiner, by defining necessary rules and extending its core algorithms to tailor it for accurately detecting Rename Local Variable refactoring instances. We have evaluated the proposed technique on two large-scale open-source systems, namely dnsjava and Tomcat. Our comparison with REPENT, the state-of-the-art tool in detecting Rename Local Variable refactoring instances, shows that our approach is superior in terms of precision and recall. Moreover, to automatically create a reference corpus of refactoring instances which is required for the evaluation, we have built a fully-automated infrastructure (called RefBenchmark) that is able to invoke several refactoring detection tools and find the agreements/disagreements between their results

Improving the Accuracy of Refactoring Detection

With the development of refactoring technology, refactoring detection technology as its reverse technology has also been greatly progressed and applied, the technology has important significance and role for code optimisation, code review and code reliability. Over the past 20 years, the refactoring detection technique has evolved from a theoretical concept to be mature approaches and tools. However, due to the various complexities that arise when refactoring code, there are still some problems with these detection tools at work: selection of tools, detection of nested refactorings, false negatives due to matching algorithms, etc. As the requirements for detection increase, the pursuit of better detection performance (precision and recall) and more generalised detection tools has become a major research goal for my PhD. The main research components of this paper include: Firstly, at the beginning of my research I conducted a meta-analysis of refactoring detection and evaluated the detection performance of four common refactoring detection tools under the same benchmark, analysed and compared their strengths and weaknesses, and identified new research questions and research directions. Secondly, I identified the study and detection of nested refactorings as a research blind spot in existing approaches, so I conducted a demonstration of the feasibility of nesting multiple refactor types with each other; in addition, I created an approach that can detect nested refactorings based on a single refactoring data using manually defined refactoring features combined with a random forest algorithm, thus being able to detect all 35 semantically meaningful nestings of them with 91.4% accuracy. Then I focused on the features that emerged during the refactoring process, mining the refactoring information in the diff to help RefDiff improve detection performance. During the research I developed Diff Extractor and Diff Encoder for extracting and encoding diffs, transformed diffs into arrays for refactoring information mining, and trained two models: 1. Diff Structure Feature Model, which determines the type of refactoring based on the structural features of the refactored diffs and can be used as a result checker, which improves the overall performance of RefDiff by checking for false positives in the RefDiff detection results. 2. Diff Feature Matching Network, which is trained based on the correspondence of the removed and added parts of the refactored diff, has excellent robustness and can solve the problem of missing matching caused by word frequency matching approach. Finally, I design an approach that integrates the two models, optimises Diff Extractor and Diff Encoder based on the characteristics of diff features, adds flags to diff based on the refactoring property, designs new encoding approach emphasising token uniqueness, trains better models and builds a model cross-validation mechanism that allows us to obtain detection results with high levels of confidence. We have shown that our approach, called RefDiff-Model, not only improves the precision of RefDiff 2.0 to 100% and increases the recall to 96.1%, but also continues to support detection tasks in multiple programming languages