Input Modeling Prioritization Using Statistically User Profile for Pairwise Test Case Generation with Constraints Handling

Pairwise testing is a widely used technique for software testing with reduce size of the test suite and able to detect interactions that trigger the system’s faults. In addition, pairwise testing test suites must be able to deal with constraints between input parameters and values. In current practice, selecting input parameters and values usually depends on tester skills that might not be sufficient. Input parameters and values modeling and tools for easily guiding and prioritizing the selection of optimal input parameters and values for the SUT is also required. In this work, we present an approach for prioritizing input parameters and values modeling using statistical user profile. Our approach is implemented in a tool called UPPTCT which provides ability to handle constraints on input parameters and values for pairwise testing in order to generate test cases. We conduct experiments to evaluate test case effectiveness and compare our tool with other renowned pairwise test generation and constraints handling tools. The experimental results show that the effectiveness of our approach is significantly more efficient and effective than random testing as large portion of reported defects with regard to statically user profile were caught by our approach. Furthermore, our tool performs better in some cases and performs comparable results for generating test cases upon input parameters and values for both with constraints handling and without constraints handling.Pairwise testing is a widely used technique for software testing with the reduced size of the test suite and able to detect interactions that trigger the system’s faults. In addition, pairwise testing test suites must be able to take constraints between input parameters and parameter values into account. In current practice, identifying and selecting input parameters and parameter values usually depends on tester skills that might not be sufficient. Input parameters and parameter values modeling and tools for easily guiding and prioritizing the selection of optimal input parameters and parameter values for the SUT is also required. In this work, we present an approach for prioritizing input parameters and parameter values modeling using statistical user profile. Our approach is implemented in a tool called UPPTCT which provides the ability to handle constraints on input parameters and parameter values for pairwise testing in order to generate test cases. We conduct experiments to evaluate test case effectiveness and compare our tool with other renowned pairwise test generation and constraints handling tools. The experimental results show that the effectiveness of our approach is significantly more efficient and effective than random testing as a large portion of reported defects with regard to statically user profile were caught by our approach. Furthermore, our tool performs better in some cases and performs comparable results for generating test cases upon input parameters and parameter values for both with constraints handling and without constraints handling

Robustness-Driven Resilience Evaluation of Self-Adaptive Software Systems

An increasingly important requirement for certain classes of software-intensive systems is the ability to self-adapt their structure and behavior at run-time when reacting to changes that may occur to the system, its environment, or its goals. A major challenge related to self-adaptive software systems is the ability to provide assurances of their resilience when facing changes. Since in these systems, the components that act as controllers of a target system incorporate highly complex software, there is the need to analyze the impact that controller failures might have on the services delivered by the system. In this paper, we present a novel approach for evaluating the resilience of self-adaptive software systems by applying robustness testing techniques to the controller to uncover failures that can affect system resilience. The approach for evaluating resilience, which is based on probabilistic model checking, quantifies the probability of satisfaction of system properties when the target system is subject to controller failures. The feasibility of the proposed approach is evaluated in the context of an industrial middleware system used to monitor and manage highly populated networks of devices, which was implemented using the Rainbow framework for architecture-based self-adaptation

Testing of OrgPlan Conversion Planning software (OF0331)

OrgPlan is a computer package designed to support farmers and consultants in planning a conversion to organic farming. It consists of two main elements: the basic planning module and a database with data for organic, in-conversion and conventional data. It was developed with DEFRA funding (OF 0159) by a partnership between the University of Wales, Aberystwyth, the University of Hertfordshire, Elm Farm Research Centre and SAC. The objective of this work was to obtain feedback before its general release on the suitability of OrgPlan in supporting the process of planning a conversion to organic farming. Given the risks of the organic conversion process and the sensitive nature of the financial reports that OrgPlan can generate, further testing with consultants experienced in organic conversion planning was carried out. The work was broken down in four objectives. Independent of this, OrgPlan has been used by the contractor in the context of research work, in particular the Modelling of Strategies of Organic Milk Production (OF 0146). Objective 1: Update of standard data The contractor updated the OrgPlan database with data from the 2002/03 Organic Farm Management Handbook and other sources. Objective 2: Workshops and Field testing of the software Three workshops with a total 22 consultants were held during which they were given a basic introduction to the use of OrgPlan and had a first opportunity to use the software on their own computer or appropriate workstations. OrgPlan can effectively support several aspects of a first broad brush planning of an organic conversion (rotation planning, cropping and livestock enterprises, feasibility of a proposed organic scenario in terms of financial output, nutrient and forage budgets) and can assist with more detailed financial planning of investments, leading to Profit and Loss and Cash-Flow forecasts. OrgPlan could have a wider application in whole farm planning, but this would require extending the database to cover a wider range of enterprises common on conventional farms. Key strengths identified by the consultants (not in order of importance) • Financial planning • Availability of basic enterprise data set • Rotation planning and nutrient budgets • Combination of financial and nutrient data in one package • Create different scenarios giving instant access for reassessment of options • Possibility to ‘tweak' a scenario • Library, navigation around the collection is excellent • Help topics clear and straightforward • Broad brush planning, particularly for farms planning new enterprises Key weaknesses (not in order of importance) • Limited range of enterprises in the database, particularly for horticultural crops • Problems with set-up, use of database and understanding all functions • Need for regular updates of the dataset • P and K Fertilisers routinely included in organic enterprises • Data entry in some sections is long-winded Objective 3: Essential corrections to the software and update of advisory section • A list of problems and suggestions was compiled. All essential changes will be implemented before a release of the software. Other suggestions, which entail more complicated programming work, are included in a as ideas for future development of OrgPlan. Objective 4: Final report This is the final report submitted to DEFRA. The contractor will also submit to DEFRA a concept outlining the steps to be taken for the release of the software, which is planned for autumn 2003

Multi-objective improvement of software using co-evolution and smart seeding

Optimising non-functional properties of software is an important part of the implementation process. One such property is execution time, and compilers target a reduction in execution time using a variety of optimisation techniques. Compiler optimisation is not always able to produce semantically equivalent alternatives that improve execution times, even if such alternatives are known to exist. Often, this is due to the local nature of such optimisations. In this paper we present a novel framework for optimising existing software using a hybrid of evolutionary optimisation techniques. Given as input the implementation of a program or function, we use Genetic Programming to evolve a new semantically equivalent version, optimised to reduce execution time subject to a given probability distribution of inputs. We employ a co-evolved population of test cases to encourage the preservation of the program’s semantics, and exploit the original program through seeding of the population in order to focus the search. We carry out experiments to identify the important factors in maximising efficiency gains. Although in this work we have optimised execution time, other non-functional criteria could be optimised in a similar manner

Towards Grid Interoperability

The Grid paradigm promises to provide global access to computing resources, data storage and experimental instruments. It also provides an elegant solution to many resource administration and provisioning problems while offering a platform for collaboration and resource sharing. Although substantial progress has been made towards these goals, nevertheless there is still a lot of work to be done until the Grid can deliver its promises. One of the central issues is the development of standards and Grid interoperability. Job execution is one of the key capabilities in all Grid environments. This is a well understood, mature area with standards and implementations. This paper describes some proof of concept experiments demonstrating the interoperability between various Grid environments

Modeling software design diversity

Design diversity has been used for many years now as a means of achieving a degree of fault tolerance in software-based systems. Whilst there is clear evidence that the approach can be expected to deliver some increase in reliability compared with a single version, there is not agreement about the extent of this. More importantly, it remains difficult to evaluate exactly how reliable a particular diverse fault-tolerant system is. This difficulty arises because assumptions of independence of failures between different versions have been shown not to be tenable: assessment of the actual level of dependence present is therefore needed, and this is hard. In this tutorial we survey the modelling issues here, with an emphasis upon the impact these have upon the problem of assessing the reliability of fault tolerant systems. The intended audience is one of designers, assessors and project managers with only a basic knowledge of probabilities, as well as reliability experts without detailed knowledge of software, who seek an introduction to the probabilistic issues in decisions about design diversity

An Adaptive Design Methodology for Reduction of Product Development Risk

Embedded systems interaction with environment inherently complicates understanding of requirements and their correct implementation. However, product uncertainty is highest during early stages of development. Design verification is an essential step in the development of any system, especially for Embedded System. This paper introduces a novel adaptive design methodology, which incorporates step-wise prototyping and verification. With each adaptive step product-realization level is enhanced while decreasing the level of product uncertainty, thereby reducing the overall costs. The back-bone of this frame-work is the development of Domain Specific Operational (DOP) Model and the associated Verification Instrumentation for Test and Evaluation, developed based on the DOP model. Together they generate functionally valid test-sequence for carrying out prototype evaluation. With the help of a case study 'Multimode Detection Subsystem' the application of this method is sketched. The design methodologies can be compared by defining and computing a generic performance criterion like Average design-cycle Risk. For the case study, by computing Average design-cycle Risk, it is shown that the adaptive method reduces the product development risk for a small increase in the total design cycle time.Comment: 21 pages, 9 figure