Investigating Automatic Static Analysis Results to Identify Quality Problems: an Inductive Study

Background: Automatic static analysis (ASA) tools examine source code to discover "issues", i.e. code patterns that are symptoms of bad programming practices and that can lead to defective behavior. Studies in the literature have shown that these tools find defects earlier than other verification activities, but they produce a substantial number of false positive warnings. For this reason, an alternative approach is to use the set of ASA issues to identify defect prone files and components rather than focusing on the individual issues. Aim: We conducted an exploratory study to investigate whether ASA issues can be used as early indicators of faulty files and components and, for the first time, whether they point to a decay of specific software quality attributes, such as maintainability or functionality. Our aim is to understand the critical parameters and feasibility of such an approach to feed into future research on more specific quality and defect prediction models. Method: We analyzed an industrial C# web application using the Resharper ASA tool and explored if significant correlations exist in such a data set. Results: We found promising results when predicting defect-prone files. A set of specific Resharper categories are better indicators of faulty files than common software metrics or the collection of issues of all issue categories, and these categories correlate to different software quality attributes. Conclusions: Our advice for future research is to perform analysis on file rather component level and to evaluate the generalizability of categories. We also recommend using larger datasets as we learned that data sparseness can lead to challenges in the proposed analysis proces

Change decision support:extraction and analysis of late architecture changes using change characterization and software metrics

Software maintenance is one of the most crucial aspects of software development. Software engineering researchers must develop practical solutions to handle the challenges presented in maintaining mature software systems. Research that addresses practical means of mitigating the risks involved when changing software, reducing the complexity of mature software systems, and eliminating the introduction of preventable bugs is paramount to today’s software engineering discipline. Giving software developers the information that they need to make quality decisions about changes that will negatively affect their software systems is a key aspect to mitigating those risks. This dissertation presents work performed to assist developers to collect and process data that plays a role in change decision-making during the maintenance phase. To address these problems, developers need a way to better understand the effects of a change prior to making the change. This research addresses the problems associated with increasing architectural complexity caused by software change using a twoold approach. The first approach is to characterize software changes to assess their architectural impact prior to their implementation. The second approach is to identify a set of architecture metrics that correlate to system quality and maintainability and to use these metrics to determine the level of difficulty involved in making a change. The two approaches have been combined and the results presented provide developers with a beneficial analysis framework that offers insight into the change process

Technical Debt Decision-Making Framework

Software development companies strive to produce high-quality software. In commercial software development environments, due to resource and time constraints, software is often developed hastily which gives rise to technical debt. Technical debt refers to the consequences of taking shortcuts when developing software. These consequences include making the system difficult to maintain and defect prone. Technical debt can have financial consequences and impede feature enhancements. Identifying technical debt and deciding which debt to address is challenging given resource constraints. Project managers must decide which debt has the highest priority and is most critical to the project. This decision-making process is not standardized and sometimes differs from project to project. My research goal is to develop a framework that project managers can use in their decision-making process to prioritize technical debt based on its potential impact. To achieve this goal, we survey software practitioners, conduct literature reviews, and mine software repositories for historical data to build a framework to model the technical debt decision-making process and inform practitioners of the most critical debt items

Technical Debt Decision-Making Framework

Software development companies strive to produce high-quality software. In commercial software development environments, due to resource and time constraints, software is often developed hastily which gives rise to technical debt. Technical debt refers to the consequences of taking shortcuts when developing software. These consequences include making the system difficult to maintain and defect prone. Technical debt can have financial consequences and impede feature enhancements. Identifying technical debt and deciding which debt to address is challenging given resource constraints. Project managers must decide which debt has the highest priority and is most critical to the project. This decision-making process is not standardized and sometimes differs from project to project. My research goal is to develop a framework that project managers can use in their decision-making process to prioritize technical debt based on its potential impact. To achieve this goal, we survey software practitioners, conduct literature reviews, and mine software repositories for historical data to build a framework to model the technical debt decision-making process and inform practitioners of the most critical debt items

Study of metrics and practices for improving object oriented software quality

Modern software systems are large and complex products, consisting in thousands lines of code, developed, often in a distributed environment, by dozens of developers and produced through an industrial process, usu- ally with short time to market. To manage such kind of complexity and to keep the development process under control measurements and metrics are required. The present thesis collects the outcomes of the research the author carried on in the field of software metrics during the three years of the Ph.D. studies. Software metrics are used to measure various aspects of software development, including software features, processes execution, developers' efforts, software quality, just to name a few. The first part of the present thesis reports the results of the studies performed on product metrics, with the final goal of helping software engineers better manage the programmers efforts and particularly to assess software quality dur- ing software development. The second part of this dissertation presents the outcomes of the research aimed at shedding some light on the effec- tiveness and impact of some development practices on software systems. To perform these studies I used a novel approach, based on the concept of complex network. Complex networks are in fact one of the best can- didates to represent software systems, enabling researchers to obtain a deeper knowledge of the structure and evolution of a software system. We found some meaningful statistical correlations between network metrics and software properties. Both the theoretical framework and the reported findings might, in principle, have a practical application to assist software engineers dealing with specific development tasks, like bug discovery or refactoring

Study of metrics and practices for improving object oriented software quality

Modern software systems are large and complex products, consisting in thousands lines of code, developed, often in a distributed environment, by dozens of developers and produced through an industrial process, usu- ally with short time to market. To manage such kind of complexity and to keep the development process under control measurements and metrics are required. The present thesis collects the outcomes of the research the author carried on in the field of software metrics during the three years of the Ph.D. studies. Software metrics are used to measure various aspects of software development, including software features, processes execution, developers' efforts, software quality, just to name a few. The first part of the present thesis reports the results of the studies performed on product metrics, with the final goal of helping software engineers better manage the programmers efforts and particularly to assess software quality dur- ing software development. The second part of this dissertation presents the outcomes of the research aimed at shedding some light on the effec- tiveness and impact of some development practices on software systems. To perform these studies I used a novel approach, based on the concept of complex network. Complex networks are in fact one of the best can- didates to represent software systems, enabling researchers to obtain a deeper knowledge of the structure and evolution of a software system. We found some meaningful statistical correlations between network metrics and software properties. Both the theoretical framework and the reported findings might, in principle, have a practical application to assist software engineers dealing with specific development tasks, like bug discovery or refactoring

System design and the cost of architectural complexity

Thesis (Ph. D.)--Massachusetts Institute of Technology, Engineering Systems Division, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 159-166).Many modern systems are so large that no one truly understands how they work. It is well known in the engineering community that architectural patterns (including hierarchies, modules, and abstraction layers) should be used in design because they play an important role in controlling complexity. These patterns make a system easier to evolve and keep its separate portions within the bounds of human understanding so that distributed teams can operate independently while jointly fashioning a coherent whole. This study set out to measure the link between architectural complexity (the complexity that arises within a system due to a lack or breakdown of hierarchy or modularity) and a variety of costs incurred by a development organization. A study was conducted within a successful software firm. Measures of architectural complexity were taken from eight versions of their product using techniques recently developed by MacCormack, Baldwin, and Rusnak. Significant cost drivers including defect density, developer productivity, and staff turnover were measured as well. The link between cost and complexity was explored using a variety of statistical techniques. Within this research setting, we found that differences in architectural complexity could account for 50% drops in productivity, three-fold increases in defect density, and order-of-magnitude increases in staff turnover. Using the techniques developed in this thesis, it should be possible for firms to estimate the financial cost of their complexity by assigning a monetary value to the decreased productivity, increased defect density, and increased turnover it causes. As a result, it should be possible for firms to more accurately estimate the potential dollar-value of refactoring efforts aimed at improving architecture.by Daniel J. Sturtevant.Ph.D

Using Agile Software Development Practices in a Research Oriented Distributed Simulation

Although sometimes controversial, agile methodologies have proven to be a viable choice for some software development projects. Projects suited to agile methodologies are those that involve new technology, have requirements that change rapidly, and are controlled by small, talented teams. Much literature about agile software development leans towards business products and non-government entities. Only a handful of literature resources mention agile software development being used in government contracts and even fewer resources mention research projects. NASA\u27s Airspace and Traffic Operations Simulation (ATOS) is a research oriented simulation that doesn\u27t follow the traditional business project mold. In an effort to gain a better understanding if agile could be used effectively in a NASA contract for a research oriented simulation project, this research looked at what agile practices could be effectively used to help gain simulation reliability while simultaneously allowing routine maintenance, current experiment support, new modeling additions, and comprehensive architectural changes

Towards the detection and analysis of performance regression introducing code changes

In contemporary software development, developers commonly conduct regression testing to ensure that code changes do not affect software quality. Performance regression testing is an emerging research area from the regression testing domain in software engineering. Performance regression testing aims to maintain the system\u27s performance. Conducting performance regression testing is known to be expensive. It is also complex, considering the increase of committed code and developing team members working simultaneously. Many automated regression testing techniques have been proposed in prior research. However, challenges in the practice of locating and resolving performance regression still exist. Directing regression testing to the commit level provides solutions to locate the root cause, yet it hinders the development process. This thesis outlines motivations and solutions to address locating performance regression root causes. First, we challenge a deterministic state-of-art approach by expanding the testing data to find improvement areas. The deterministic approach was found to be limited in searching for the best regression-locating rule. Thus, we presented two stochastic approaches to develop models that can learn from historical commits. The goal of the first stochastic approach is to view the research problem as a search-based optimization problem seeking to reach the highest detection rate. We are applying different multi-objective evolutionary algorithms and conducting a comparison between them. This thesis also investigates whether simplifying the search space by combining objectives would achieve comparative results. The second stochastic approach addresses the severity of class imbalance any system could have since code changes introducing regression are rare but costly. We formulate the identification of problematic commits that introduce performance regression as a binary classification problem that handles class imbalance. Further, the thesis provides an exploratory study on the challenges developers face in resolving performance regression. The study is based on the questions posted on a technical form directed to performance regression. We collected around 2k questions discussing the regression of software execution time, and all were manually analyzed. The study resulted in a categorization of the challenges. We also discussed the difficulty level of performance regression issues within the development community. This study provides insights to help developers during the software design and implementation to avoid regression causes