Achievements, open problems and challenges for search based software testing

Search Based Software Testing (SBST) formulates testing as an optimisation problem, which can be attacked using computational search techniques from the field of Search Based Software Engineering (SBSE). We present an analysis of the SBST research agenda, focusing on the open problems and challenges of testing non-functional properties, in particular a topic we call 'Search Based Energy Testing' (SBET), Multi-objective SBST and SBST for Test Strategy Identification. We conclude with a vision of FIFIVERIFY tools, which would automatically find faults, fix them and verify the fixes. We explain why we think such FIFIVERIFY tools constitute an exciting challenge for the SBSE community that already could be within its reach

Study of Code Smells: A Review and Research Agenda

Code Smells have been detected, predicted and studied by researchers from several perspectives. This literature review is conducted to understand tools and algorithms used to detect and analyze code smells to summarize research agenda. 114 studies have been selected from 2009 to 2022 to conduct this review. The studies are deeply analyzed under the categorization of machine learning and non-machine learning, which are found to be 25 and 89 respectively. The studies are analyzed to gain insight into algorithms, tools and limitations of the techniques. Long Method, Feature Envy, and Duplicate Code are reported to be the most popular smells. 38% of the studies focused their research on the enhancement of tools and methods. Random Forest and JRip algorithms are found to give the best results under machine learning techniques. We extended the previous studies on code smell detection tools, reporting a total 87 tools during the review. Java is found to be the dominant programming language during the study of smells

Achievements, Open Problems and Challenges for Search Based Software Testing

testing as an optimisation problem, which can be attacked using computational search techniques from the field of Search Based Software Engineering (SBSE). We present an analysis of the SBST research agenda1, focusing on the open problems and chal-lenges of testing non-functional properties, in particular a topic we call ‘Search Based Energy Testing ’ (SBET), Multi-objective SBST and SBST for Test Strategy Identification. We conclude with a vision of FIFIVERIFY tools, which would automatically find faults, fix them and verify the fixes. We explain why we think such FIFIVERIFY tools constitute an exciting challenge for the SBSE community that already could be within its reach. I

Software Process Evaluation from User Perceptions and Log Data

Companies often claim to follow specific software development methodologies (SDM) when performing their software development process. These methodologies are often supported by dedicated tools that keep track of work activities carried out by developers. The purpose of this paper is to provide a novel approach that integrates analytical insights from both the perceptions of SDM stakeholders and software development tools logs to provide SDM improvement recommendations. This paper develops a new process improvement approach that combines two significantly different sources of data on the same phenomenon. First, it uses a questionnaire to gather software development stakeholder SDM perceptions (managers and developers). Second, it leverages process mining to analyze software development tools logs to obtain additional information on software development activities. Finally, it develops recommendations based on concurrent analysis of both sources. Our novel process improvement approach is evaluated in three directions: Does the presented approach (RQ1) enable managers to gain additional insights into employees' performance, (RQ2) deliver additional insights into project performance, and (RQ3) enable development of additional SDM improvement recommendations? We find that integrated analysis of software development perception data and software development tools logs opens new possibilities to more precisely identify and improve specific SDM elements. The evaluation of our novel process improvement approach follows a single case study design. Our approach can only be used in enterprises in which software development tools logs are available. The study should be repeated in different cultural settings. We practically show how concurrently analyzing data about developer SDM perceptions and event log data from software development tools enables management to gain additional insights in the software development process regarding the performance of individual developers. The main theoretical contribution of our paper is a novel process improvement approach that effectively integrates data from management and developer perspectives and software development tools logs.Einstein Foundation Berlin http://dx.doi.org/10.13039/501100006188Peer Reviewe

A Study on Software Testability and the Quality of Testing in Object-Oriented Systems

Software testing is known to be important to the delivery of high-quality systems, but it is also challenging, expensive and time-consuming. This has motivated academic and industrial researchers to seek ways to improve the testability of software. Software testability is the ease with which a software artefact can be effectively tested. The first step towards building testable software components is to understand the factors – of software processes, products and people – that are related to and can influence software testability. In particular, the goal of this thesis is to provide researchers and practitioners with a comprehensive understanding of design and source code factors that can affect the testability of a class in object oriented systems. This thesis considers three different views on software testability that address three related aspects: 1) the distribution of unit tests in relation to the dynamic coupling and centrality of software production classes, 2) the relationship between dynamic (i.e., runtime) software properties and class testability, and 3) the relationship between code smells, test smells and the factors related to smells distribution. The thesis utilises a combination of source code analysis techniques (both static and dynamic), software metrics, software visualisation techniques and graph-based metrics (from complex networks theory) to address its goals and objectives. A systematic mapping study was first conducted to thoroughly investigate the body of research on dynamic software metrics and to identify issues associated with their selection, design and implementation. This mapping study identified, evaluated and classified 62 research works based on a pre-tested protocol and a set of classification criteria. Based on the findings of this study, a number of dynamic metrics were selected and used in the experiments that were then conducted. The thesis demonstrates that by using a combination of visualisation, dynamic analysis, static analysis and graph-based metrics it is feasible to identify central classes and to diagrammatically depict testing coverage information. Experimental results show that, even in projects with high test coverage, some classes appear to be left without any direct unit testing, even though they play a central role during a typical execution profile. It is contended that the proposed visualisation techniques could be particularly helpful when developers need to maintain and reengineer existing test suites. Another important finding of this thesis is that frequently executed and tightly coupled classes are correlated with the testability of the class – such classes require larger unit tests and more test cases. This information could inform estimates of the effort required to test classes when developing new unit tests or when maintaining and refactoring existing tests. An additional key finding of this thesis is that test and code smells, in general, can have a negative impact on class testability. Increasing levels of size and complexity in code are associated with the increased presence of test smells. In addition, production classes that contain smells generally require larger unit tests, and are also likely to be associated with test smells in their associated unit tests. There are some particular smells that are more significantly associated with class testability than other smells. Furthermore, some particular code smells can be seen as a sign for the presence of test smells, as some test and code smells are found to co-occur in the test and production code. These results suggest that code smells, and specifically certain types of smells, as well as measures of size and complexity, can be used to provide a more comprehensive indication of smells likely to emerge in test code produced subsequently (or vice versa in a test-first context). Such findings should contribute positively to the work of testers and maintainers when writing unit tests and when refactoring and maintaining existing tests

Investigating Cultural Dimensions via Developers Artefacts: The Utility of Repository Mining

A growing body of research is using artefacts from online development communities to explore the impact of developers’ behaviours on the software development process. Although this research has produced many insights, researchers have yet to fully explore the impact of developers’ cultural backgrounds on their behaviours in an online community, although such understandings could be useful for helping the community to understand and plan for team dynamics. This study utilised a pragmatic case study to explore the relationship between culture and online behaviour among developers from the United States (U.S.), China, and Russia—three countries that differ in their orientations as individualistic or collectivist cultures. The data for the study comprised artefacts supplied over an 11-year period by users of Stack Overflow1, a popular online programming community that addresses questions from members by providing them with rapid access to the knowledge and expertise of their peers. Artefacts consisted of developers’ questions and answers, personal profiles, Up and Down voting records, online reputations, and earned badges. Data mining techniques, as well as statistical, linguistic, and content analysis were used to compare artefacts from the three groups of developers based on their cultural orientation as individualistic or collectivistic, attitudes, and interaction and knowledge sharing patterns. The findings revealed differences among the three groups that were consistent with their cultural backgrounds. U.S. developers, who are from an individualistic culture, asked and responded to more questions, had higher average reputations, used the pronoun “I” more frequently, and were more task- focused. Conversely, Chinese developers, who are from a collectivistic culture, provided more extensive commenting and editing of posts, used the pronouns “we” and “you” more frequently, and were more likely to engage in information exchange. Russian developers had been using Stack Overflow the longest and were the most reflective. The cultural patterns identified in this study have numerous implications for enhancing in- group interactions and behaviour management among software development communities