A Survey of Constrained Combinatorial Testing

Combinatorial Testing (CT) is a potentially powerful testing technique, whereas its failure revealing ability might be dramatically reduced if it fails to handle constraints in an adequate and efficient manner. To ensure the wider applicability of CT in the presence of constrained problem domains, large and diverse efforts have been invested towards the techniques and applications of constrained combinatorial testing. In this paper, we provide a comprehensive survey of representations, influences, and techniques that pertain to constraints in CT, covering 129 papers published between 1987 and 2018. This survey not only categorises the various constraint handling techniques, but also reviews comparatively less well-studied, yet potentially important, constraint identification and maintenance techniques. Since real-world programs are usually constrained, this survey can be of interest to researchers and practitioners who are looking to use and study constrained combinatorial testing techniques

An overview of decision table literature 1982-1995.

This report gives an overview of the literature on decision tables over the past 15 years. As much as possible, for each reference, an author supplied abstract, a number of keywords and a classification are provided. In some cases own comments are added. The purpose of these comments is to show where, how and why decision tables are used. The literature is classified according to application area, theoretical versus practical character, year of publication, country or origin (not necessarily country of publication) and the language of the document. After a description of the scope of the interview, classification results and the classification by topic are presented. The main body of the paper is the ordered list of publications with abstract, classification and comments.

Automatic test case generation for WS-Agreements using combinatorial testing

In the scope of the applications developed under the service-based paradigm, Service Level Agreements (SLAs) are a standard mechanism used to flexibly specify the Quality of Service (QoS) that must be delivered. These agreements contain the conditions negotiated between the service provider and consumers as well as the potential penalties derived from the violation of such conditions. In this context, it is important to assure that the service based application (SBA) behaves as expected in order to avoid potential consequences like penalties or dissatisfaction between the stakeholders that have negotiated and signed the SLA. In this article we address the testing of SLAs specified using the WS-Agreement standard by means of applying testing techniques such as the Classification Tree Method and Combinatorial Testing to generate test cases. From the content of the individual terms of the SLA, we identify situations that need to be tested. We also obtain a set of constraints based on the SLA specification and the behavior of the SBA in order to guarantee the testability of the test cases. Furthermore, we define three different coverage strategies with the aim at grading the intensity of the tests. Finally, we have developed a tool named SLACT (SLA Combinatorial Testing) in order to automate the process and we have applied the whole approach to an eHealth case study

Interactive Cost Configuration Over Decision Diagrams

Abstract In many AI domains such as product configuration, a user should interactively specify a solution that must satisfy a set of constraints. In such scenarios, offline compilation of feasible solutions into a tractable representation is an important approach to delivering efficient backtrack-free user interaction online. In particular, binary decision diagrams (BDDs) have been successfully used as a compilation target for product and service configuration. In this paper we discuss how to extend BDD-based configuration to scenarios involving cost functions which express user preferences. We first show that an efficient, robust and easy to implement extension is possible if the cost function is additive, and feasible solutions are represented using multi-valued decision diagrams (MDDs). We also discuss the effect on MDD size if the cost function is non-additive or if it is encoded explicitly into MDD. We then discuss interactive configuration in the presence of multiple cost functions. We prove that even in its simplest form, multiple-cost configuration is NP-hard in the input MDD. However, for solving two-cost configuration we develop a pseudo-polynomial scheme and a fully polynomial approximation scheme. The applicability of our approach is demonstrated through experiments over real-world configuration models and product-catalogue datasets. Response times are generally within a fraction of a second even for very large instances

Experimental Design in Game Testing

The gaming industry has been on constant rise over the last few years. Companies invest huge amounts of money for the release of their games. A part of this money is invested in testing the games. Current game testing methods include manual execution of pre-written test cases in the game. Each test case may or may not result in a bug. In a game, a bug is said to occur when the game does not behave according to its intended design. The process of writing the test cases to test games requires standardization. We believe that this standardization can be achieved by implementing experimental design to video game testing. In this thesis, we discuss the implementation of combinatorial testing to test games. Combinatorial testing is a method of experimental design that is used to generate test cases and is primarily used for commercial software testing. In addition to the discussion of the implementation of combinatorial testing techniques in video game testing, we present a method for finding combinations resulting in video game bugs

Building High Strength Mixed Covering Arrays with Constraints

Covering arrays have become a key piece in Combinatorial Testing. In particular, we focus on the efficient construction of Covering Arrays with Constraints of high strength. SAT solving technology has been proven to be well suited when solving Covering Arrays with Constraints. However, the size of the SAT reformulations rapidly grows up with higher strengths. To this end, we present a new incomplete algorithm that mitigates substantially memory blow-ups. The experimental results confirm the goodness of the approach, opening avenues for new practical applications

Topology Network Optimization of Facility Planning and Design Problems

The research attempts to provide a graphical theory-based approach to solve the facility layout problem. Which has generally been approached using Quadratic Assignment Problem (QAP) in the past, an algebraic method. It is a very complex problem and is part of the NP-Hard optimization class, because of the nonlinear quadratic objective function and (0,1) binary variables. The research is divided into three phases which together provide an optimal facility layout, block plan solution consisting of MHS (material handling solution) projected onto the block plan. In phase one, we solve for the position of departments in a facility based on flow and utility factor (weight for location). The position of all the departments is identified on the vertices of MPG (maximal planar graph), which maximizes the possibility of flow. We use named MPG produced in literature, throughout the research. The grouping of the department is achieved through GMAFLAD, a QSP (quadratic set packing) based optimizer. In Phase 2, the dual for the MPG’s is solved consisting of department location as per phase 1, to generate Voronoi graphs. These graphs are then, expanded by an ingenious parameter optimization formulation to achieve area fitting for individual cases. Optimization modeling software, Lingo17.0 is used for solving the parameter optimization for generating coordinates of the block plan. The plotting of coordinates for the block plan graphics is done via Autodesk inventor 2019. In phase 3, the solution for MHS is achieved using an RSMT (Rectilinear Steiner minimal tree) graph approach. The Voronoi seed coordinates produced through phase 2 results are computed by GeoSteiner package to generated the RSMT graph for projection onto the block plan (Also, done by Inventor 2019). The graphical method employed in this research, itself has complex and NP-hard problem segments in it, which have been relaxed to polynomial time complexity by fragmenting into groups and solving them in sections. Solving for MPG & RSMT are a class of NP-Hard problem, which have been restricted to N=32 here. Finally, to validate the research and its methodology a real-life case study of a shipyard building for the data set of PDVSA, Venezuela is performed and verified