Complex situations formed by mixes of versatile environments, various user needs and time-to-market constraints led to the development of highly configurable systems. In line with the emergence of such systems, software development is increasingly moving from the production of a single, yet configurable software to the development of families of software products. Such families of related software are called Software Product Lines (SPLs), and they allow the automation of the configuration, deployment and management of tailored software products through the combination of software features. These features and the constraints defining their legal combinations are usually encoded in a feature model (FM), which is used to represent a SPL.
One main challenge with SPLs is testing them, a task which is even more difficult as the number of features proposed is important. Ideally, all the possible products that can be configured from a SPL should be tested. This, however, is unfeasible in practice since only 270 optional features allows configuring more products than the number of atoms in the universe. Considering that realistic SPLs involve thousands of features and that testing capabilities are limited by time and budget constraints, only a subset of all the configurable products can actually be tested, introducing the needs for strategies to test such SPLs.
To reduce the testing effort, techniques using combinatorial interaction testing (CIT) have been proposed and proven to be successful. However, they fail at scaling to large and heavily constrained SPLs. In addition, CIT is costly to apply due to the combinatorial explosion induced by calculating the feature combinations. Besides, existing approaches do not consider multiple and potentially conflicting testing objectives such as minimizing the number of configurations and their cost. In this respect, this dissertation introduces scalable techniques for both generating and prioritizing relevant SPL product configurations for CIT by using a similarity heuristic which avoids the combinatorial explosion. In a second step, methods for handling multiple testing objectives are presented.
The following part of this thesis focuses on the quality assessment of given product configurations prior to testing. The objective here is to evaluate how good is a given set of configurations according to different testing criteria, whatever the way these configurations have been selected. This situation arises when configurations that have to be tested are already available. Since testing these software products individually is a costly and time consuming task, methodologies to evaluate them prior testing are introduced, thus allowing to discard unnecessary ones and save testing sessions. In particular, an approach based on mutation of the SPL FM which can form viable and cheaper alternative to CIT is presented.
The next part of this dissertation investigates the reverse-engineering of a SPL and its FM from existing source code of software product variants. Since SPLs allows us to reduce development costs and quickly derive tailored products for specific market share, automated techniques to migrate similar product variants into a whole SPL are necessary. In particular, the challenge of reverse-engineering a SPL which is concordant with the underlying software products is tackled by a fully automated approach. In addition, since reverse-engineering approaches (whether manually or automatically performed) are inherently error-prone, a methodology for evaluating and fixing reverse-engineered SPL FM is presented.
The final part of this dissertation describes the application of the introduced theoretical advances to an industrial case with the CETREL company. In this project, a credit card authorization system is tested by using credit card authorizations. The testing process is optimized by modeling credit card authorizations as a SPL, enabling the application of the above-mentioned generation and evaluation approaches. All the proposed approaches use search-based techniques combined with constraint solvers and have been validated through rigorous experiments performed on moderate to large scale SPLs
