Mutation testing and self/peer assessment: analyzing their effect on students in a software testing course

Testing is a crucial activity in the development of software systems. With the increasing complexity of software projects, the industry requires incorporating graduates with adequate testing skills and preparation in this field. A challenge in software testing education is to make students perceive the benefits of writing tests and assess their quality with advanced testing techniques. In this paper, we present an experience integrating both mutation testing and self/peer assessment –two of the most used techniques to that end in the past– into a software testing course during three years. This experience allowed us to analyze the effect of applying these strategies on the students’ perception of their manually-written test suites. Noticeably, the computation of the mutation score significantly undermined the initial expectations they had on the developed test suites. Also, the application of peer testing helped them estimate the relative quality of two comparable test suites, as we found a notable correspondence with their respective mutation coverage. Besides, a more in-depth analysis revealed that the students' test suites with more test cases did not always achieve the highest scores, that they found more readable their own tests, and that they tended to cover the basic operations while forgetting about more advanced features. An opinion survey confirmed the impact that the use of mutants had on their perception about testing, and they mostly supported paying a higher level of attention to testing concepts in software engineering degree plans.The work was partially funded by the European Commission (FEDER), the Spanish Ministry of Science, Innovation and Universities under the project FAME (RTI2018-093608-B-C33), the European project ASSETs (612678-EPP-1-2019-1-IT-EPPKA2-SSA-B), and the University of Cádiz

The development of a program analysis environment for Ada

A unit level, Ada software module testing system, called Query Utility Environment for Software Testing of Ada (QUEST/Ada), is described. The project calls for the design and development of a prototype system. QUEST/Ada design began with a definition of the overall system structure and a description of component dependencies. The project team was divided into three groups to resolve the preliminary designs of the parser/scanner: the test data generator, and the test coverage analyzer. The Phase 1 report is a working document from which the system documentation will evolve. It provides history, a guide to report sections, a literature review, the definition of the system structure and high level interfaces, descriptions of the prototype scope, the three major components, and the plan for the remainder of the project. The appendices include specifications, statistics, two papers derived from the current research, a preliminary users' manual, and the proposal and work plan for Phase 2

Gamifying Software Testing – A Focus on Strategy & Tools Development

This study aims to introduce new software testing strategies and tools with the aim of creating a more engaging and rewarding environment for software testers. For this purpose, gamification has been selected as a potential solution to raise the performances of testers. Empirical experiments were conducted to validate key factors and metrics influencing the design and development of a gamified software testing system

A Guided Chatbot Learning Experience in the Science Classroom

This dissertation describes a practitioner’s design-based development of a prototype chatbot to guide students in learning biological concepts of genetic mutations and protein synthesis. This chatbot’s architecture provides learning activities, feedback, and support throughout a series of short, connected lessons. The chatbot is designed to scaffold learners through a predict, observe, explain model of inquiry learning. It utilizes real-world phenomena to lead students through biology core ideas, science and engineering practices, and crosscutting concepts. Results of prototype testing include survey results in support of the proof of concept among both students and teachers, as well as accuracy measurements of chatbot intents. Descriptive statistics and suggestions were collected from both groups to evaluate the relevancy, consistency, practicality, and effectiveness of the project as well as speak to improvements for future projects. The designer finds that the construction of chatbots as guided learning experiences holds untapped potential in science educational technology. Advisor: Guy Traini

A Fault-Based Model of Fault Localization Techniques

Every day, ordinary people depend on software working properly. We take it for granted; from banking software, to railroad switching software, to flight control software, to software that controls medical devices such as pacemakers or even gas pumps, our lives are touched by software that we expect to work. It is well known that the main technique/activity used to ensure the quality of software is testing. Often it is the only quality assurance activity undertaken, making it that much more important. In a typical experiment studying these techniques, a researcher will intentionally seed a fault (intentionally breaking the functionality of some source code) with the hopes that the automated techniques under study will be able to identify the fault\u27s location in the source code. These faults are picked arbitrarily; there is potential for bias in the selection of the faults. Previous researchers have established an ontology for understanding or expressing this bias called fault size. This research captures the fault size ontology in the form of a probabilistic model. The results of applying this model to measure fault size suggest that many faults generated through program mutation (the systematic replacement of source code operators to create faults) are very large and easily found. Secondary measures generated in the assessment of the model suggest a new static analysis method, called testability, for predicting the likelihood that code will contain a fault in the future. While software testing researchers are not statisticians, they nonetheless make extensive use of statistics in their experiments to assess fault localization techniques. Researchers often select their statistical techniques without justification. This is a very worrisome situation because it can lead to incorrect conclusions about the significance of research. This research introduces an algorithm, MeansTest, which helps automate some aspects of the selection of appropriate statistical techniques. The results of an evaluation of MeansTest suggest that MeansTest performs well relative to its peers. This research then surveys recent work in software testing using MeansTest to evaluate the significance of researchers\u27 work. The results of the survey indicate that software testing researchers are underreporting the significance of their work

Establishing a Performance Testing Approach for E-Learning Applications

Most of the E-Learning applications perform poorly in motivating employees to learn. To solve this problem, we need to examine what workplace e-learning requires and how workplace e-learning systems should be developed in line with those requirements. We investigated the problem by identifying the fundamental elements of the workplace learning environment including the learner, organization, learning content and social context, and their relationships. We found that workplace e-learning should align individual and organizational learning needs, connect learning and work performance, and support social interaction among individuals. To achieve this, a performance testing approach is proposed. Key performance indicators are utilized to clarify organizational goals, make sense of work context and requests on work performance, and accordingly help employees set up rational learning objectives and enhance their learning process. Using this approach, prototype system has been developed and a set of experiments have been conducted to demonstrate the effectiveness of the approach. This paper also presents the use of software verification, validation and testing technique, traditionally used in the software development, in the design and implementation of E-Learning products. We examine the ways one can apply testing techniques in E-Learning life cycle. This includes the strategy adoption for the selection of testing technique along with tool acquisition and measurement. The objective is to develop a collaborative approach involving software testing and educational methodology

Kean Program Review - BA Biology 2019

Kean Program Review - BA Biology 201

Explainable Automated Debugging via Large Language Model-driven Scientific Debugging

Automated debugging techniques have the potential to reduce developer effort in debugging, and have matured enough to be adopted by industry. However, one critical issue with existing techniques is that, while developers want rationales for the provided automatic debugging results, existing techniques are ill-suited to provide them, as their deduction process differs significantly from that of human developers. Inspired by the way developers interact with code when debugging, we propose Automated Scientific Debugging (AutoSD), a technique that given buggy code and a bug-revealing test, prompts large language models to automatically generate hypotheses, uses debuggers to actively interact with buggy code, and thus automatically reach conclusions prior to patch generation. By aligning the reasoning of automated debugging more closely with that of human developers, we aim to produce intelligible explanations of how a specific patch has been generated, with the hope that the explanation will lead to more efficient and accurate developer decisions. Our empirical analysis on three program repair benchmarks shows that AutoSD performs competitively with other program repair baselines, and that it can indicate when it is confident in its results. Furthermore, we perform a human study with 20 participants, including six professional developers, to evaluate the utility of explanations from AutoSD. Participants with access to explanations could judge patch correctness in roughly the same time as those without, but their accuracy improved for five out of six real-world bugs studied: 70% of participants answered that they wanted explanations when using repair tools, while 55% answered that they were satisfied with the Scientific Debugging presentation