Multi-objective Optimal Test Suite Computation for Software Product Line Pairwise Testing

Lopez-Herrejon, R. E., Chicano F., Ferrer J., Egyed A., & Alba E. (2013). Multi-objective Optimal Test Suite Computation for Software Product Line Pairwise Testing. 2013 IEEE International Conference on Software Maintenance, Eindhoven, The Netherlands, September 22-28, 2013. 404–407.Software Product Lines (SPLs) are families of related software products, which usually provide a large number of feature combinations, a fact that poses a unique set of challenges for software testing. Recently, many SPL testing approaches have been proposed, among them pair wise combinatorial techniques that aim at selecting products to test based on the pairs of feature combinations such products provide. These approaches regard SPL testing as an optimization problem where either coverage (maximize) or test suite size (minimize) are considered as the main optimization objective. Instead, we take a multi-objective view where the two objectives are equally important. In this exploratory paper we propose a zero-one mathematical linear program for solving the multi-objective problem and present an algorithm to compute the true Pareto front, hence an optimal solution, from the feature model of a SPL. The evaluation with 118 feature models revealed an interesting trade-off between reducing the number of constraints in the linear program and the runtime which opens up several venues for future research.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. Austrian Science Fund (FWF) project P21321-N15 and Lise Meitner Fellowship M1421-N15. Spanish Ministry of Economy and Competitiveness and FEDER under contract TIN2011-28194 and fellowship BES-2012-055967

Comparative analysis of classical multi-objective evolutionary algorithms and seeding strategies for pairwise testing of Software Product Lines

Lopez-Herrejon, R. Erick, Ferrer J., Chicano F., Egyed A., & Alba E. (2014). Comparative analysis of classical multi-objective evolutionary algorithms and seeding strategies for pairwise testing of Software Product Lines. Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2014, Beijing, China, July 6-11, 2014. 387–396.Software Product Lines (SPLs) are families of related software products, each with its own set of feature combinations. Their commonly large number of products poses a unique set of challenges for software testing as it might not be technologically or economically feasible to test of all them individually. SPL pairwise testing aims at selecting a set of products to test such that all possible combinations of two features are covered by at least one selected product. Most approaches for SPL pairwise testing have focused on achieving full coverage of all pairwise feature combinations with the minimum number of products to test. Though useful in many contexts, this single-objective perspective does not reflect the prevailing scenario where software engineers do face trade-offs between the objectives of maximizing the coverage or minimizing the number of products to test. In contrast and to address this need, our work is the first to propose a classical multi-objective formalisation where both objectives are equally important. In this paper, we study the application to SPL pairwise testing of four classical multi-objective evolutionary algorithms. We developed three seeding strategies — techniques that leverage problem domain knowledge — and measured their performance impact on a large and diverse corpus of case studies using two well-known multi-objective quality measures. Our study identifies the performance differences among the algorithms and corroborates that the more domain knowledge leveraged the better the search results. Our findings enable software engineers to select not just one solution (as in the case of single-objective techniques) but instead to select from an array of test suite possibilities the one that best matches the economical and technological constraints of their testing context.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. Austrian Science Fund (FWF) project P25289- N15 and Lise Meitner Fellowship M1421-N15. Spanish Ministry of Economy and Competitiveness and FEDER under contract TIN2011-28194 and fellowship BES-2012-055967. Project 8.06/5.47.4142 in collaboration with the VSB-Tech. Univ. of Ostrava and Universidad de Málaga, Andalucía Tech

Reverse Engineering Feature Models with Evolutionary Algorithms: An Exploratory Study

Successful software evolves, more and more commonly, from a single system to a set of system variants tailored to meet the similiar and yet di erent functionality required by the distinct clients and users. Software Product Line Engineering (SPLE) is a software development paradigm that has proven e ective for coping with this scenario. At the core of SPLE is variability modeling which employs Feature Models (FMs) as the de facto standard to represent the combinations of features that distinguish the systems variants. Reverse engineering FMs consist in constructing a feature model from a set of products descriptions. This research area is becoming increasingly active within the SPLE community, where the problem has been addressed with di erent perspectives and approaches ranging from analysis of con guration scripts, use of propositional logic or natural language techniques, to ad hoc algorithms. In this paper, we explore the feasibility of using Evolutionary Algorithms (EAs) to synthesize FMs from the feature sets that describe the system variants. We analyzed 59 representative case studies of di erent characteristics and complexity. Our exploratory study found that FMs that denote proper supersets of the desired feature sets can be obtained with a small number of generations. However, reducing the di erences between these two sets with an e ective and scalable tness function remains an open question.We believe that this work is a rst step towards leveraging the extensive wealth of Search-Based Software Engineering techniques to address this and other variability management challenges.CICYT TIN2009- 07366Junta de Andalucía TIC-590

Multi-objective test case prioritization in highly configurable systems: A case study

Test case prioritization schedules test cases for execution in an order that attempts to accelerate the detection of faults. The order of test cases is determined by prioritization objectives such as covering code or critical components as rapidly as possible. The importance of this technique has been recognized in the context of Highly-Configurable Systems (HCSs), where the potentially huge number of configurations makes testing extremely challenging. However, current approaches for test case prioritization in HCSs suffer from two main limitations. First, the prioritization is usually driven by a single objective which neglects the potential benefits of combining multiple criteria to guide the detection of faults. Second, instead of using industry-strength case studies, evaluations are conducted using synthetic data, which provides no information about the effectiveness of different prioritization objectives. In this paper, we address both limitations by studying 63 combinations of up to three prioritization objectives in accelerating the detection of faults in the Drupal framework. Results show that non–functional properties such as the number of changes in the features are more effective than functional metrics extracted from the configuration model. Results also suggest that multi-objective prioritization typically results in faster fault detection than mono-objective prioritization.CICYT TIN2012-32273CICYT TIN2015-70560-RJunta de Andalucía P12-TIC- 186

An assessment of search-based techniques for reverse engineering feature models

Successful software evolves from a single system by adding and changing functionality to keep up with users’ demands and to cater to their similar and different requirements. Nowadays it is a common practice to offer a system in many variants such as community, professional, or academic editions. Each variant provides different functionality described in terms of features. Software Product Line Engineering (SPLE) is an effective software development paradigm for this scenario. At the core of SPLE is variability modelling whose goal is to represent the combinations of features that distinguish the system variants using feature models, the de facto standard for such task. As SPLE practices are becoming more pervasive, reverse engineering feature models from the feature descriptions of each individual variant has become an active research subject. In this paper we evaluated, for this reverse engineering task, three standard search based techniques (evolutionary algorithms, hill climbing, and random search) with two objective functions on 74 SPLs. We compared their performance using precision and recall, and found a clear trade-off between these two metrics which we further reified into a third objective function based on Fβ, an information retrieval measure, that showed a clear performance improvement. We believe that this work sheds light on the great potential of search-based techniques for SPLE tasks.Ministerio de Economía y Competitividad TIN2012-32273Junta de Andalucía TIC-186

Tag relatedness in image folksonomies

Folksonomies - networks of users, resources, and tags allow users to easily retrieve, organize and browse web contents. However, their advantages are still limited mainly due to the noisiness of user provided tags. To overcome this issue, we propose an approach for characterizing related tags in folksonomies: we use tag co-occurrence statistics and Laplacian score based feature selection in order to create empirical co-occurrence probability distribution for each tag; then we identify related tags on the basis of the dissimilarity between their distributions. For this purpose, we introduce variant of the Jensen-Shannon Divergence, which is more robust to statistical noise. We experimentally evaluate our approach using WordNet and compare it to a common tag-relatedness approach based on the cosine similarity. The results show the effectiveness of our approach and its advantage over the competing method

The fundamental constants and their variation: observational status and theoretical motivations

This article describes the various experimental bounds on the variation of the fundamental constants of nature. After a discussion on the role of fundamental constants, of their definition and link with metrology, the various constraints on the variation of the fine structure constant, the gravitational, weak and strong interactions couplings and the electron to proton mass ratio are reviewed. This review aims (1) to provide the basics of each measurement, (2) to show as clearly as possible why it constrains a given constant and (3) to point out the underlying hypotheses. Such an investigation is of importance to compare the different results, particularly in view of understanding the recent claims of the detections of a variation of the fine structure constant and of the electron to proton mass ratio in quasar absorption spectra. The theoretical models leading to the prediction of such variation are also reviewed, including Kaluza-Klein theories, string theories and other alternative theories and cosmological implications of these results are discussed. The links with the tests of general relativity are emphasized.Comment: 56 pages, l7 figures, submitted to Rev. Mod. Phy