Analyzing Clone Evolution for Identifying the Important Clones for Management

Code clones (identical or similar code fragments in a code-base) have dual but contradictory impacts (i.e., both positive and negative impacts) on the evolution and maintenance of a software system. Because of the negative impacts (such as high change-proneness, bug-proneness, and unintentional inconsistencies), software researchers consider code clones to be the number one bad-smell in a code-base. Existing studies on clone management suggest managing code clones through refactoring and tracking. However, a software system's code-base may contain a huge number of code clones, and it is impractical to consider all these clones for refactoring or tracking. In these circumstances, it is essential to identify code clones that can be considered particularly important for refactoring and tracking. However, no existing study has investigated this matter. We conduct our research emphasizing this matter, and perform five studies on identifying important clones by analyzing clone evolution history. In our first study we detect evolutionary coupling of code clones by automatically investigating clone evolution history from thousands of commits of software systems downloaded from on-line SVN repositories. By analyzing evolutionary coupling of code clones we identify a particular clone change pattern, Similarity Preserving Change Pattern (SPCP), such that code clones that evolve following this pattern should be considered important for refactoring. We call these important clones the SPCP clones. We rank SPCP clones considering their strength of evolutionary coupling. In our second study we further analyze evolutionary coupling of code clones with an aim to assist clone tracking. The purpose of clone tracking is to identify the co-change (i.e. changing together) candidates of code clones to ensure consistency of changes in the code-base. Our research in the second study identifies and ranks the important co-change candidates by analyzing their evolutionary coupling. In our third study we perform a deeper analysis on the SPCP clones and identify their cross-boundary evolutionary couplings. On the basis of such couplings we separate the SPCP clones into two disjoint subsets. While one subset contains the non-cross-boundary SPCP clones which can be considered important for refactoring, the other subset contains the cross-boundary SPCP clones which should be considered important for tracking. In our fourth study we analyze the bug-proneness of different types of SPCP clones in order to identify which type(s) of code clones have high tendencies of experiencing bug-fixes. Such clone-types can be given high priorities for management (refactoring or tracking). In our last study we analyze and compare the late propagation tendencies of different types of code clones. Late propagation is commonly regarded as a harmful clone evolution pattern. Findings from our last study can help us prioritize clone-types for management on the basis of their tendencies of experiencing late propagations. We also find that late propagation can be considerably minimized by managing the SPCP clones. On the basis of our studies we develop an automatic system called AMIC (Automatic Mining of Important Clones) that identifies the important clones for management (refactoring and tracking) and ranks these clones considering their evolutionary coupling, bug-proneness, and late propagation tendencies. We believe that our research findings have the potential to assist clone management by pin-pointing the important clones to be managed, and thus, considerably minimizing clone management effort

Change Impact Analysis of Code Clones

Copying a code fragment and reusing it with or without modifications is known to be a frequent activity in software development. This results in exact or closely similar copies of code fragments, known as code clones, to exist in the software systems. Developers leverage the code reuse opportunity by code cloning for increased productivity. However, different studies on code clones report important concerns regarding the impacts of clones on software maintenance. One of the key concerns is to maintain consistent evolution of the clone fragments as inconsistent changes to clones may introduce bugs. Challenges to the consistent evolution of clones involve the identification of all related clone fragments for change propagation when a cloned fragment is changed. The task of identifying the ripple effects (i.e., all the related components to change) is known as Change Impact Analysis (CIA). In this thesis, we evaluate the impacts of clones on software systems from new perspectives and then we propose an evolutionary coupling based technique for change impact analysis of clones. First, we empirically evaluate the comparative stability of cloned and non-cloned code using fine-grained syntactic change types. Second, we assess the impacts of clones from the perspective of coupling at the domain level. Third, we carry out a comprehensive analysis of the comparative stability of cloned and non-cloned code within a uniform framework. We compare stability metrics with the results from the original experimental settings with respect to the clone detection tools and the subject systems. Fourth, we investigate the relationships between stability and bug-proneness of clones to assess whether and how stability contribute to the bug-proneness of different types of clones. Next, in the fifth study, we analyzed the impacts of co-change coupling on the bug-proneness of different types of clones. After a comprehensive evaluation of the impacts of clones on software systems, we propose an evolutionary coupling based CIA approach to support the consistent evolution of clones. In the sixth study, we propose a solution to minimize the effects of atypical commits (extra large commits) on the accuracy of the detection of evolutionary coupling. We propose a clustering-based technique to split atypical commits into pseudo-commits of related entities. This considerably reduces the number of incorrect couplings introduced by the atypical commits. Finally, in the seventh study, we propose an evolutionary coupling based change impact analysis approach for clones. In addition to handling the atypical commits, we use the history of fine-grained syntactic changes extracted from the software repositories to detect typed evolutionary coupling of clones. Conventional approaches consider only the frequency of co-change of the entities to detect evolutionary coupling. We consider both change frequencies and the fine-grained change types in the detection of evolutionary coupling. Findings from our studies give important insights regarding the impacts of clones and our proposed typed evolutionary coupling based CIA approach has the potential to support the consistent evolution of clones for better clone management

An empirical investigation into contributory factors of change and fault propensity in large-scale commercial object-oriented software

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel UniversityObject-Oriented design and development dominates both commercial and open source software projects. One of the principal goals of object-oriented design is to aid reuse, and hence, reduce future maintenance efforts of software systems. However, the on-going maintenance of large-scale software systems (both changes and faults) continues to be a significant proportion of the lifecycle of the system and the total investment cost. Understanding and thus being able to predict - or even reduce - the impact of the contributing factors of future maintenance efforts of a software system is thus highly beneficial to software practitioners. In this Thesis we empirically study a large, commercial software system with the principal aim to determine the contributing factors to the change and fault propensity over a three-year period. We consider the object-oriented design context of the software, specifically its inheritance characteristics, coupling and cohesion properties, object-oriented design pattern participation, and size. We also explore the effect of refactoring and test classes in the software. Our results show that several aspects of the design context of a class have an impact to the change and fault-proneness of the software. Specifically, we show that classes with high afferent or efferent coupling are more change and fault-prone; we also identify a number of design patterns whose participants tend to have a higher change and fault propensity than non-participants and we identify a range of inheritance characteristics (in terms of depth of inheritance and number of children) that result in an increase to change and fault-proneness. Furthermore we show that refactoring is a commonly occurring maintenance activity, although it is largely limited to simpler types of refactorings. Finally, we provide some insight into the co-evolution of production and test code during refactoring

Analysis of Human Affect and Bug Patterns to Improve Software Quality and Security

The impact of software is ever increasing as more and more systems are being software operated. Despite the usefulness of software, many instances software failures have been causing tremendous losses in lives and dollars. Software failures take place because of bugs (i.e., faults) in the software systems. These bugs cause the program to malfunction or crash and expose security vulnerabilities exploitable by malicious hackers. Studies confirm that software defects and vulnerabilities appear in source code largely due to the human mistakes and errors of the developers. Human performance is impacted by the underlying development process and human affects, such as sentiment and emotion. This thesis examines these human affects of software developers, which have drawn recent interests in the community. For capturing developers’ sentimental and emotional states, we have developed several software tools (i.e., SentiStrength-SE, DEVA, and MarValous). These are novel tools facilitating automatic detection of sentiments and emotions from the software engineering textual artifacts. Using such an automated tool, the developers’ sentimental variations are studied with respect to the underlying development tasks (e.g., bug-fixing, bug-introducing), development periods (i.e., days and times), team sizes and project sizes. We expose opportunities for exploiting developers’ sentiments for higher productivity and improved software quality. While developers’ sentiments and emotions can be leveraged for proactive and active safeguard in identifying and minimizing software bugs, this dissertation also includes in-depth studies of the relationship among various bug patterns, such as software defects, security vulnerabilities, and code smells to find actionable insights in minimizing software bugs and improving software quality and security. Bug patterns are exposed through mining software repositories and bug databases. These bug patterns are crucial in localizing bugs and security vulnerabilities in software codebase for fixing them, predicting portions of software susceptible to failure or exploitation by hackers, devising techniques for automated program repair, and avoiding code constructs and coding idioms that are bug-prone. The software tools produced from this thesis are empirically evaluated using standard measurement metrics (e.g., precision, recall). The findings of all the studies are validated with appropriate tests for statistical significance. Finally, based on our experience and in-depth analysis of the present state of the art, we expose avenues for further research and development towards a holistic approach for developing improved and secure software systems

Understanding the Impact of Poor Coding Practices on the Quality of Deep Learning Systems

RÉSUMÉ:Les applications basées sur l’apprentissage profond, en anglais Deep Learning (DL), sont de plus en plus utilisées pour résoudre diverses tâches de notre quotidien grâce aux recentes prouesses des modèles d’apprentissage profond qui surpassent déjà les compétences humaines dans un large éventail de tâches, de la classification des images à la reconnaissance de la parole et au traitement du langage naturel. Ces progrès tendent à élargir l’application de l’apprentissage profond dans des domaines aussi critiques en termes de sécurité comme les voitures autonomes et la santé. Les spécialistes de l’apprentissage profond partagent les mêmes préoccupations que les ingénieurs logiciels d’ autres domaines en ce qui concerne l’efficacité, la complexité et la maintenabilité des systèmes logiciels. En revanche, le processus de développement continu des systèmes d’apprentissage profond, caractérisé par un rythme rapide et une complexité accrue, pourrait conduire à de mauvais choix de conception par le développeur.---------- ABSTRACT: Deep Learning (DL) based applications are increasingly being used in our society to solve a variety of tasks, thanks to the recent progress of deep learning models, which are now outperforming humans on a wide range of tasks, from image classification to speech recognition and natural language processing. This progress is being made towards the widespread application of DL in safety-critical applications such as autonomous cars and healthcare. Deep learning practitioners share similar concerns as software engineers in other domains with regards to efficiency, complexity, and maintainability. On the other hand, the continuous development of deep learning systems which takes place at a rapid pace as well as their increasing complexity could lead to bad design choices on the part of the developers. Furthermore, due to the prevalent use of similar frameworks and repeated coding of similar tasks, deep learning developers, therefore, tend to use copy-paste practice, creating clones in deep learning cod

A systematic literature review on the code smells datasets and validation mechanisms

The accuracy reported for code smell-detecting tools varies depending on the dataset used to evaluate the tools. Our survey of 45 existing datasets reveals that the adequacy of a dataset for detecting smells highly depends on relevant properties such as the size, severity level, project types, number of each type of smell, number of smells, and the ratio of smelly to non-smelly samples in the dataset. Most existing datasets support God Class, Long Method, and Feature Envy while six smells in Fowler and Beck's catalog are not supported by any datasets. We conclude that existing datasets suffer from imbalanced samples, lack of supporting severity level, and restriction to Java language.Comment: 34 pages, 10 figures, 12 tables, Accepte

30 Years of Software Refactoring Research: A Systematic Literature Review

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/155872/4/30YRefactoring.pd