Spinal Test Suites for Software Product Lines

A major challenge in testing software product lines is efficiency. In particular, testing a product line should take less effort than testing each and every product individually. We address this issue in the context of input-output conformance testing, which is a formal theory of model-based testing. We extend the notion of conformance testing on input-output featured transition systems with the novel concept of spinal test suites. We show how this concept dispenses with retesting the common behavior among different, but similar, products of a software product line.Comment: In Proceedings MBT 2014, arXiv:1403.704

Towards Statistical Prioritization for Software Product Lines Testing

Software Product Lines (SPL) are inherently difficult to test due to the combinatorial explosion of the number of products to consider. To reduce the number of products to test, sampling techniques such as combinatorial interaction testing have been proposed. They usually start from a feature model and apply a coverage criterion (e.g. pairwise feature interaction or dissimilarity) to generate tractable, fault-finding, lists of configurations to be tested. Prioritization can also be used to sort/generate such lists, optimizing coverage criteria or weights assigned to features. However, current sampling/prioritization techniques barely take product behavior into account. We explore how ideas of statistical testing, based on a usage model (a Markov chain), can be used to extract configurations of interest according to the likelihood of their executions. These executions are gathered in featured transition systems, compact representation of SPL behavior. We discuss possible scenarios and give a prioritization procedure illustrated on an example.Comment: Extended version published at VaMoS '14 (http://dx.doi.org/10.1145/2556624.2556635

Trusted product lines

This thesis describes research undertaken into the application of software product line approaches to the development of high-integrity, embedded real-time software systems that are subject to regulatory approval/certification. The motivation for the research arose from a real business need to reduce cost and lead time of aerospace software development projects. The thesis hypothesis can be summarised as follows: It is feasible to construct product line models that allow the specification of required behaviour within a reference architecture that can be transformed into an effective product implementation, whilst enabling suitable supporting evidence for certification to be produced. The research concentrates on the following four main areas: 1. Construction of an argument framework in which the application of product line techniques to high-integrity software development can be assessed and critically reviewed. 2. Definition of a product-line reference architecture that can host components containing variation. 3. Design of model transformations that can automatically instantiate products from a set of components hosted within the reference architecture. 4. Identification of verification approaches that may provide evidence that the transformations designed in step 3 above preserve properties of interest from the product line model into the product instantiations. Together, these areas form the basis of an approach we term “Trusted Product Lines”. The approach has been evaluated and validated by deployment on a real aerospace project; the approach has been used to produce DO-178B/ED-12B Level A applications of over 300 KSLOC in size. The effect of this approach on the software development process has been critically evaluated in this thesis, both quantitatively (in terms of cost and relative size of process phases) and qualitatively (in terms of software quality). The “Trusted Product Lines” approach, as described within the thesis, shows how product line approaches can be applied to high-integrity software development, and how certification evidence created and arguments constructed for products instantiated from the product line. To the best of our knowledge, the development and effective application of product line techniques in a certification environment is novel and unique

A Systematic Review of Tracing Solutions in Software Product Lines

Software Product Lines are large-scale, multi-unit systems that enable massive, customized production. They consist of a base of reusable artifacts and points of variation that provide the system with flexibility, allowing generating customized products. However, maintaining a system with such complexity and flexibility could be error prone and time consuming. Indeed, any modification (addition, deletion or update) at the level of a product or an artifact would impact other elements. It would therefore be interesting to adopt an efficient and organized traceability solution to maintain the Software Product Line. Still, traceability is not systematically implemented. It is usually set up for specific constraints (e.g. certification requirements), but abandoned in other situations. In order to draw a picture of the actual conditions of traceability solutions in Software Product Lines context, we decided to address a literature review. This review as well as its findings is detailed in the present article.Comment: 22 pages, 9 figures, 7 table

Probabilistic Model Checking for Energy Analysis in Software Product Lines

In a software product line (SPL), a collection of software products is defined by their commonalities in terms of features rather than explicitly specifying all products one-by-one. Several verification techniques were adapted to establish temporal properties of SPLs. Symbolic and family-based model checking have been proven to be successful for tackling the combinatorial blow-up arising when reasoning about several feature combinations. However, most formal verification approaches for SPLs presented in the literature focus on the static SPLs, where the features of a product are fixed and cannot be changed during runtime. This is in contrast to dynamic SPLs, allowing to adapt feature combinations of a product dynamically after deployment. The main contribution of the paper is a compositional modeling framework for dynamic SPLs, which supports probabilistic and nondeterministic choices and allows for quantitative analysis. We specify the feature changes during runtime within an automata-based coordination component, enabling to reason over strategies how to trigger dynamic feature changes for optimizing various quantitative objectives, e.g., energy or monetary costs and reliability. For our framework there is a natural and conceptually simple translation into the input language of the prominent probabilistic model checker PRISM. This facilitates the application of PRISM's powerful symbolic engine to the operational behavior of dynamic SPLs and their family-based analysis against various quantitative queries. We demonstrate feasibility of our approach by a case study issuing an energy-aware bonding network device.Comment: 14 pages, 11 figure