Assigning Test Priority to Modules Using Code-Content and Bug History

Regression testing is a process that is repeated after every change in the program. Prioritization of test cases is an important process during regression test execution. Nowadays, there exist several techniques that decide which of the test cases will run first as per their priority levels, while increasing the probability of finding bugs earlier in the test life cycle. However, sometimes algorithms used to select important test cases may stop searching in local minima while missing the rest of the tests that might be important for a given change. To address this limitation further, we propose a domain-specific model that assigns testing priority to classes in applications based on developers\u27 judgments for priority. Moreover, our technique which takes into consideration applications\u27 code content and bug history, relates these features to overall class priority for testing. In the end, we test the proposed approach with a new (unknown) dataset of 20 instances. The predicted results are compared with developers\u27 priority score and saw that this metric can prioritize correctly 70% of classes under test

Improving Readability in Automatic Unit Test Generation

In object-oriented programming, quality assurance is commonly provided through writing unit tests, to exercise the operations of each class. If unit tests are created and maintained manually, this can be a time-consuming and laborious task. For this reason, automatic methods are often used to generate tests that seek to cover all paths of the tested code. Search may be guided by criteria that are opaque to the programmer, resulting in test sequences that are long and confusing. This has a negative impact on test maintenance. Once tests have been created, the job is not done: programmers need to reason about the tests throughout the lifecycle, as the tested software units evolve. Maintenance includes diagnosing failing tests (whether due to a software fault or an invalid test) and preserving test oracles (ensuring that checked assertions are still relevant). Programmers also need to understand the tests created for code that they did not write themselves, in order to understand the intent of that code. If generated tests cannot be easily understood, then they will be extremely difficult to maintain. The overall objective of this thesis is to reaffirm the importance of unit test maintenance; and to offer novel techniques to improve the readability of automatically generated tests. The first contribution is an empirical survey of 225 developers from different parts of the world, who were asked to give their opinions about unit testing practices and problems. The survey responses confirm that unit testing is considered important; and that there is an appetite for higher-quality automated test generation, with a view to test maintenance. The second contribution is a domain-specific model of unit test readability, based on human judgements. The model is used to augment automated unit test generation to produce test suites with both high coverage and improved readability. In evaluations, 30 programmers preferred our improved tests and were able to answer maintenance questions 14level of accuracy. The third contribution is a novel algorithm for generating descriptive test names that summarise API- level coverage goals. Test optimisation ensures that each test is short, bears a clear relation to the covered code, and can be readily identified by programmers. In evaluations, 47 programmers agreed with the choice of synthesised names and that these were as descriptive as manually chosen names. Participants were also more accurate and faster at matching generated tests against the tested code, compared to matching with manually-chosen test names

Application of Machine Learning in Software Testing: Survey

The process of testing software applications must always be repeated to achieve the desired results. Also systems today in general have grown and expanded so much that this process needs to be automated in some form. Software testing has achieved its research interest for many years now. There are developed different testing techniques that reduces human effort, and increases software quality. In meanwhile in order to make it even more attractive and productive, researchers came with novel approaches in applying Machine Learning (ML) in testing techniques. Machine learning as a branch of artificial intelligence (AI) focuses on building applications that learn from data and improve their accuracy over time without being programmed to do so. The better the algorithm, the more accurate the decisions and predictions will become as it processes more data. The focus of this paper is surveying the literature about usage of Machine Leaning in Software Testing area. This survey will cover almost every stage of testing, starting from test data collection, test input generation, to test oracle and reporting

Benefits and Challenges of Applying Machine Learning in Adaptive Systems

Adaptive software systems have the ability to configure their process based on the input. This means that, they can change their behavior based on internal changes, or changes in the running environment. Different user requirement, big data inputs, large scale services, and always available processes made adaptive systems evolve even more. However, there is always a gap that can be filled in order to improve them further. Machine learning is one of the techniques that is widely used to improve different software systems. Its main idea is to use previous knowledge, train it, and predict new results. There exist a bunch of algorithms that are being used in this evolving area, and adaptive systems are good targets where they can have a great effect. This paper reviewed the main machine learning algorithms that are being used for adaptive systems, and further it considered the impact and challenge of these techniques in the usability of the systems

A model for predicting the probability of code beauty

Software maintenance is one of the most expensive phases of the software development life cycle. This cost increases more if maintenance is performed on poorly written code (less aesthetic). There exist a set of code writing patterns that developers need to follow to write good looking code. However, coding conforms ‘rules’ is not always possible. During software evolution, code goes through different changes, which are the main reasons for breaking rules of beautiful code. In this paper, we propose an AI (artificial intelligence) based model which will measure the beauty of a written code. The model is built on a set of code- based features that are used to assign the probability of being a beautiful code

DeepTC-Enhancer: Improving the Readability of Automatically Generated Tests

Automated test case generation tools have been successfully proposed to reduce the amount of human and infrastructure resources required to write and run test cases. However, recent studies demonstrate that the readability of generated tests is very limited due to (i) uninformative identifiers and (ii) lack of proper documentation. Prior studies proposed techniques to improve test readability by either generating natural language summaries or meaningful methods names. While these approaches are shown to improve test readability, they are also affected by two limitations: (1) generated summaries are often perceived as too verbose and redundant by developers, and (2) readable tests require both proper method names but also meaningful identifiers (within-method readability). In this work, we combine template based methods and Deep Learning (DL) approaches to automatically generate test case scenarios (elicited from natural language patterns of test case statements) as well as to train DL models on path-based representations of source code to generate meaningful identifier names. Our approach, called DeepTC-Enhancer, recommends documentation and identifier names with the ultimate goal of enhancing readability of automatically generated test cases. An empirical evaluation with 36 external and internal developers shows that (1) DeepTC-Enhancer outperforms significantly the baseline approach for generating summaries and performs equally with the baseline approach for test case renaming, (2) the transformation proposed by DeepTC-Enhancer results in a significant increase in readability of automatically generated test cases, and (3) there is a significant difference in the feature preferences between external and internal developers.Accepted author manuscriptSoftware Engineerin

Generating Highly-structured Input Data by Combining Search-based Testing and Grammar-based Fuzzing

Software testing is an important and time-consuming task that is often done manually. In the last decades, researchers have come up with techniques to generate input data (e.g., fuzzing) and automate the process of generating test cases (e.g., search-based testing). However, these techniques are known to have their own limitations: search-based testing does not generate highly-structured data; grammar-based fuzzing does not generate test case structures. To address these limitations, we combine these two techniques. By applying grammar-based mutations to the input data gathered by the search-based testing algorithm, it allows us to co-evolve both aspects of test case generation. We evaluate our approach, called G-EvoSuite, by performing an empirical study on 20 Java classes from the three most popular JSON parsers across multiple search budgets. Our results show that the proposed approach on average improves branch coverage for JSON related classes by 15% (with a maximum increase of 50%) without negatively impacting other classes.Virtual/online event due to COVID-19Software EngineeringSoftware Technolog