CTJ: Input-Output Based Relation Combinatorial Testing Strategy Using Jaya Algorithm

            ويكاد يكون من المستحيل اختبار كل مجموعة من المدخلات نظرًا لأن تنفيذ حالات الاختبار يتطلب وقتا طويلا للغاية. الأختبار الاندماجي هو السبيل لتخطي عقبات الاختبار الشامل من خلال أختبار كل قيم المدخلات لكل المعاملات المركبة المتعددة طرق الترتيب.   يمكن تقسيم الاختبار التجميعي إلى ثلاثة أنواع هي تفاعل القوة الموحد ، والتفاعل المتغير والقوة ، والعلاقة القائمة على المدخلات والمخرجات . ان الطريقة الاخيرة الانفة الذكر تختزل الفحص الاندماجي الى مجموعة ضمن اختيار الشخص الفاحص. معظم الابحاث في الاختبار الاندماجي طبقت في تفاعل القوة الموحدة وقوة التفاعل المتغيرة ، ومع ذلك ، هناك اهتمام قليل جدا بالعلاقة بين المدخلات والمخرجات. لذا تم اقتراح خوارزمية جايا في هذا البحث  كخوارزمية مثلي لانشاء جدول الفحص الاندماجي باستراتيجية تعتمد على العلاقة بين المدخلات والمخرجات. نتيجة تطبيق خوارزمية جايا في الاختبار الاندماجي القائم على المدخلات والمخرجات مقبولة لأنها تنتج العدد الأمثل تقريبًا لحالات الاختبار في نطاق زمني مقبول.Software testing is a vital part of the software development life cycle. In many cases, the system under test has more than one input making the testing efforts for every exhaustive combination impossible (i.e. the time of execution of the test case can be outrageously long). Combinatorial testing offers an alternative to exhaustive testing via considering the interaction of input values for every t-way combination between parameters. Combinatorial testing can be divided into three types which are uniform strength interaction, variable strength interaction and input-output based relation (IOR). IOR combinatorial testing only tests for the important combinations selected by the tester. Most of the researches in combinatorial testing applied the uniform and the variable interaction strength, however, there is still a lack of work addressing IOR. In this paper, a Jaya algorithm is proposed as an optimization algorithm engine to construct a test list based on IOR in the proposed combinatorial test list generator strategy into a tool called CTJ. The result of applying the Jaya algorithm in input-output based combinatorial testing is acceptable since it produces a nearly optimum number of test cases in a satisfactory time range

A framework for cots software evaluation and selection for COTS mismatches handling and non-functional requirements

The decision to purchase Commercial Off-The-Shelf (COTS) software needs systematic guidelines so that the appropriate COTS software can be selected in order to provide a viable and effective solution to the organizations. However, the existing COTS software evaluation and selection frameworks focus more on functional aspects and do not give adequate attention to accommodate the mismatch between user requirements and COTS software specification, and also integration with non functional requirements of COTS software. Studies have identified that these two criteria are important in COTS software evaluation and selection. Therefore, this study aims to develop a new framework of COTS software evaluation and selection that focuses on handling COTS software mismatches and integrating the nonfunctional requirements. The study is conducted using mixed-mode methodology which involves survey and interview. The study is conducted in four main phases: a survey and interview of 63 organizations to identify COTS software evaluation criteria, development of COTS software evaluation and selection framework using Evaluation Theory, development of a new decision making technique by integrating Analytical Hierarchy Process and Gap Analysis to handle COTS software mismatches, and validation of the practicality and reliability of the proposed COTS software Evaluation and Selection Framework (COTS-ESF) using experts’ review, case studies and yardstick validation. This study has developed the COTS-ESF which consists of five categories of evaluation criteria: Quality, Domain, Architecture, Operational Environment and Vendor Reputation. It also provides a decision making technique and a complete process for performing the evaluation and selection of COTS software. The result of this study shows that the evaluated aspects of the framework are feasible and demonstrate their potential and practicality to be applied in the real environment. The contribution of this study straddles both the research and practical perspectives of software evaluation by improving decision making and providing a systematic guidelines for handling issue in purchasing viable COTS software

Harmonizing CMMI-DEV 1.2 and XP Method to Improve The Software Development Processes in Small Software Development Firms

Most software development organizations are small firms, and they have realized the need to manage and improve their software development and management activities. Traditional Software Process Improvement (SPI) models and standards are not realistic for these firms because of high cost, limited resources and strict project deadlines. Therefore, these firms need a lightweight software development method and an appropriate SPI model to manage and improve their software development and management processes. This study aims to construct a suitable software development process improvement framework for Small Software Development Firms (SSDFs) based on eXtreme Programming (XP) method and Capability Maturity Model Integration for Development Version 1.2 (CMMI-Dev1.2) model. Four stages are involved in developing the framework: (1) aligning XP practices to the specific goals of CMMI-Dev1.2 Key Process Areas (KPAs); (2) developing the proposed software development process improvement framework based on extending XP method by adapting the Extension-Based Approach (EBA), CMMI-Dev1.2, and generic elements of the SPI framework; (3) verifying the compatibility of the proposed framework to the KPAs of CMMI-Dev1.2 by using focus group method coupled with Delphi technique; and (4) validating the modified framework by using CMMI-Dev1.2 questionnaire as a main item to validate the suitability of the modified framework for SSDFs, and conducting two case studies to validate the applicability and effectiveness of this framework for these firms. The result of aligning XP practices to the KPAs of CMMI-Dev1.2 shows that twelve KPAs are largely supported by XP practices, eight KPAs are partially supported by XP practices, and two KPAs are not-supported by XP practices. The main contributions of this study are: software development process improvement framework for SSDFs, elicit better understanding of how to construct the framework, and quality improvement of the software development processes. There are possible avenues for extending this research to fulfil the missing specific practices of several KPAs, examining other agile practices and using CMMI-Dev1.3 to improve the framework, and conducting more case studie

Evaluating and comparing fault-based testing strategies for general Boolean specifications: A series of experiments

A great amount of fault-based testing strategies have been proposed to generate test cases for detecting certain types of faults in Boolean specifications. However, most of the previous studies on these strategies were focused on the Boolean expressions in the disjunctive normal form (DNF), even the irredundant DNF (IDNF)-little work has been conducted to comprehensively investigate their performance on general Boolean specifications. In this study, we conducted a series of experiments to evaluate and compare 18 fault-based testing strategies using over 4000 randomly generated fault-seeded Boolean expressions. In the experiments, a testing strategy is regarded as effective and efficient if it can detect most of the seeded faults using a small number of test cases. Our experimental results show that if a testing strategy is highly effective and efficient when testing the Boolean expressions in the IDNF, it also shows high effectiveness and efficiency on general Boolean expressions. It is found that one family of fault-based testing strategies, namely MUMCUT, normally deliver the best performance among all the 18 strategies. Our study provides an in-depth understanding and insight of fault-based testing for general Boolean expressions

Adaptive assessment in the class of programming

Διπλωματική εργασία--Πανεπιστήμιο Μακεδονίας, Θεσσαλονίκη, 2009.This paper presents P.A.T. (Programming Adaptive Testing), a computerized adaptive testing system for assessing students’ programming knowledge. P.A.T. was used in two high school programming classes by 73 students. After research was carried out, it was found helpful in increasing students’ cognitive domain skills

Prioritization of combinatorial test cases by incremental interaction coverage

Combinatorial testing is a well-recognized testing method, and has been widely applied in practice. To facilitate analysis, a common approach is to assume that all test cases in a combinatorial test suite have the same fault detection capability. However, when testing resources are limited, the order of executing the test cases is critical. To improve testing cost-effectiveness, prioritization of combinatorial test cases is employed. The most popular approach is based on interaction coverage, which prioritizes combinatorial test cases by repeatedly choosing an unexecuted test case that covers the largest number on uncovered parameter value combinations of a given strength (level of interaction among parameters). However, this approach suffers from some drawbacks. Based on previous observations that the majority of faults in practical systems can usually be triggered with parameter interactions of small strengths, we propose a new strategy of prioritizing combinatorial test cases by incrementally adjusting the strength values. Experimental results show that our method performs better than the random prioritization technique and the technique of prioritizing combinatorial test suites according to test case generation order, and has better performance than the interaction-coverage-based test prioritization technique in most cases

Bytecode-based Multiple Condition Coverage: An Initial Investigation

Masking occurs when one condition prevents another condition from influencing the output of a Boolean expression. Logic-based adequacy criteria such as Multiple Condition Coverage (MCC) are designed to overcome masking at the within-expression level, but can offer no guarantees about masking in subsequent expressions. As a result, a Boolean expression written as a single complex statement will yield test cases that are more likely to overcome masking than when the expression is written as series of simple statements. Many approaches to automated analysis and test case generation for Java systems operate not on the source code representation of code, but on the bytecode. The transformation from source code to bytecode requires simplifying code elements, introducing the risk of masking. We propose Bytecode-MCC, designed to group related Boolean expressions from the bytecode, reformulate the expressions into a single complex expression, and produce test cases satisfying each combination of conditions in the constructed expression. Bytecode-MCC should produce test obligations that—when satisfied—are more likely to reveal faults in the program logic than tests providing coverage of existing criteria over the simplified bytecode. A preliminary study has hinted at the potential of this approach. However, Bytecode-MCC is more difficult to achieve than Branch Coverage, and means of increasing coverage are needed to truly test the fault-detection potential of this technique. We propose methods of improving Bytecode-MCC coverage through automated generation that we will explore in future work

HydroShare – A Case Study of the Application of Modern Software Engineering to a Large Distributed Federally-Funded Scientific Software Development Project

HydroShare is an online collaborative system under development to support the open sharing of hydrologic data, analytical tools, and computer models. With HydroShare, scientists can easily discover, access, and analyze hydrologic data and thereby enhance the production and reproducibility of hydrologic scientific results. HydroShare also takes advantage of emerging social media functionality to enable users to enhance information about and collaboration around hydrologic data and models. HydroShare is being developed by an interdisciplinary collaborative team of domain scientists, university software developers, and professional software engineers from ten institutions located across the United States. While the combination of non–co-located, diverse stakeholders presents communication and management challenges, the interdisciplinary nature of the team is integral to the project’s goal of improving scientific software development and capabilities in academia. This chapter describes the challenges faced and lessons learned with the development of HydroShare, as well as the approach to software development that the HydroShare team adopted on the basis of the lessons learned. The chapter closes with recommendations for the application of modern software engineering techniques to large, collaborative, scientific software development projects, similar to the National Science Foundation (NSF)–funded HydroShare, in order to promote the successful application of the approach described herein by other teams for other projects

MiSFIT: Mining Software Fault Information and Types

As software becomes more important to society, the number, age, and complexity of systems grow. Software organizations require continuous process improvement to maintain the reliability, security, and quality of these software systems. Software organizations can utilize data from manual fault classification to meet their process improvement needs, but organizations lack the expertise or resources to implement them correctly. This dissertation addresses the need for the automation of software fault classification. Validation results show that automated fault classification, as implemented in the MiSFIT tool, can group faults of similar nature. The resulting classifications result in good agreement for common software faults with no manual effort. To evaluate the method and tool, I develop and apply an extended change taxonomy to classify the source code changes that repaired software faults from an open source project. MiSFIT clusters the faults based on the changes. I manually inspect a random sample of faults from each cluster to validate the results. The automatically classified faults are used to analyze the evolution of a software application over seven major releases. The contributions of this dissertation are an extended change taxonomy for software fault analysis, a method to cluster faults by the syntax of the repair, empirical evidence that fault distribution varies according to the purpose of the module, and the identification of project-specific trends from the analysis of the changes