Analysis of Feature Models Using Alloy: A Survey

Feature Models (FMs) are a mechanism to model variability among a family of closely related software products, i.e. a software product line (SPL). Analysis of FMs using formal methods can reveal defects in the specification such as inconsistencies that cause the product line to have no valid products. A popular framework used in research for FM analysis is Alloy, a light-weight formal modeling notation equipped with an efficient model finder. Several works in the literature have proposed different strategies to encode and analyze FMs using Alloy. However, there is little discussion on the relative merits of each proposal, making it difficult to select the most suitable encoding for a specific analysis need. In this paper, we describe and compare those strategies according to various criteria such as the expressivity of the FM notation or the efficiency of the analysis. This survey is the first comparative study of research targeted towards using Alloy for FM analysis. This review aims to identify all the best practices on the use of Alloy, as a part of a framework for the automated extraction and analysis of rich FMs from natural language requirement specifications.Comment: In Proceedings FMSPLE 2016, arXiv:1603.0857

FM fact label: a configurable and interactive visualization of feature model characterizations

Recognizing specific characteristics of feature models (FM) can be challenging due to the different nature and domains of the models. There are several metrics to characterize FMs. However, there is no standard way to visualize and identify the properties that make an FM unique and distinguishable. We propose FM Fact Label as a tool to visualize an FM characterization based on its metadata, structural measures, and analytical metrics. Although existing tools can provide a visualization of the FM and report some metrics, the feature diagram of large-scale FMs becomes ineffective to take an overall shape of the FM properties. Moreover, the reported metrics are often embedded in the tool user interface, preventing further analysis. FM Fact Label is a standalone web-based tool that provides a configurable and interactive visualization of FM characterizations that can be exported to several formats. Our contribution becomes important because the Universal Variability Language (UVL) is starting to gain attraction in the software product line community as a unified textual language to specify FMs and share knowledge. With this contribution, we help to advance the UVL ecosystem one step forward while providing a common representation for the results of existing analysis toolsMinisterio de Ciencia, Innovación y Universidades RTI2018-101204-B-C22 (OPHELIA)Junta de Andalucía P20-01224 (COPERNICA)Junta de Andalucía METAMORFOSIS (US-1381375)Ministerio de Economía y Competitividad RTI2018-099213-B-I00 (MEDEA)Ministerio de Ciencia e Innovación PID2021-122812OB-I00 (IRIS)Junta de Andalucía P18-FR-1081 (RHEA)Junta de Andalucía UMA18-FEDERIA-157 (LEIA)European Union H2020 101017109 (DAEMON

On the Hardness of SAT with Community Structure

Recent attempts to explain the effectiveness of Boolean satisfiability (SAT) solvers based on conflict-driven clause learning (CDCL) on large industrial benchmarks have focused on the concept of community structure. Specifically, industrial benchmarks have been empirically found to have good community structure, and experiments seem to show a correlation between such structure and the efficiency of CDCL. However, in this paper we establish hardness results suggesting that community structure is not sufficient to explain the success of CDCL in practice. First, we formally characterize a property shared by a wide class of metrics capturing community structure, including "modularity". Next, we show that the SAT instances with good community structure according to any metric with this property are still NP-hard. Finally, we study a class of random instances generated from the "pseudo-industrial" community attachment model of Gir\'aldez-Cru and Levy. We prove that, with high probability, instances from this model that have relatively few communities but are still highly modular require exponentially long resolution proofs and so are hard for CDCL. We also present experimental evidence that our result continues to hold for instances with many more communities. This indicates that actual industrial instances easily solved by CDCL may have some other relevant structure not captured by the community attachment model.Comment: 23 pages. Full version of a SAT 2016 pape

Aplicabilidad de la Caracterización de Benchmarks a Modelos de Variabilidad

Los benchmarks utilizados para comparar el rendimiento de diferentes sistemas presentan una alta variabilidad que puede ser representada en modelos de variabilidad como los Feature Models. En este artículo presentamos los problemas de escalabilidad y complejidad de selección por objetivos de los Feature Models, y una posible solución, mostrando las ventajas de la caracterización de benchmarks (dada por sus cargas de trabajo). Para esta mejora formalizamos un modelo de caracterización de paquetes de cargas de trabajo para Feature Models, basándonos en ocho atributos abstractos (operaciones matemáticas, memoria, ...). Este modelo y sus ventajas son evaluados en el eco-asistente HADAS, junto a un benchmark PHP, y al benchmark de sistemas empotrados BEEBS, obteniendo una capacidad de selección más intuitiva, y un decremento en el tiempo de obtención de configuraciones válidas y sus métricas en HADAS, con respecto a la representación estándar.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Uniform Random Sampling Product Configurations of Feature Models That Have Numerical Features

Analyses of Software Product Lines (SPLs) rely on automated solvers to navigate complex dependencies among features and find legal configurations. Often these analyses do not support numerical features with constraints because propositional formulas use only Boolean variables. Some automated solvers can represent numerical features natively, but are limited in their ability to count and Uniform Random Sample (URS) conigurations, which are key operations to derive unbiased statistics on configuration spaces. Bit-blasting is a technique to encode numerical constraints as propositional formulas. We use bit-blasting to encode Boolean and numerical constraints so that we can exploit existing #SAT solvers to count and URS conigurations. Compared to state-of-art Satisfiability Modulo Theory and Constraint Programming solvers, our approach has two advantages: 1) faster and more scalable coniguration counting and 2) reliable URS of SPL configurations. We also show that our work can be used to extend prior SAT-based SPL analyses to support numerical features and constraints.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Preprocessing is What You Need: Understanding and Predicting the Complexity of SAT-based Uniform Random Sampling

peer reviewedDespite its NP-completeness, the Boolean satisfiability problem gave birth to highly efficient tools that are able to find solutions to a Boolean formula and compute their number. Boolean formulae compactly encode huge, constrained search spaces for variability-intensive systems, e.g., the possible configurations of the Linux kernel. These search spaces are generally too big to explore exhaustively, leading most testing approaches to sample a few solutions before analysing them. A desirable property of such samples is uniformity: each solution should get the same selection probability. This property motivated the design of uniform random samplers, relying on SAT solvers and counters and achieving different tradeoffs between uniformity and scalability. Though we can observe their performance in practice, understanding the complexity these tools face and accurately predicting it is an under-explored problem. Indeed, structural metrics such as the number of variables and clauses involved in a formula poorly predict the sampling complexity. More elaborated ones, such as minimal independent support (MIS), are intractable to compute on large formulae. We provide an efficient parallel algorithm to compute a related metric, the number of equivalence classes, and demonstrate that this metric is highly correlated to time and memory usage of uniform random sampling and model counting tools. We explore the role of formula preprocessing on various metrics and show its positive influence on correlations. Relying on these correlations, we train an efficient classifier (F1-score 0.97) to predict whether uniformly sampling a given formula will exceed a specified budget. Our results allow us to characterise the similarities and differences between (uniform) sampling, solving and counting. c 202

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

International audienceUniform or near-uniform generation of solutions for large satisfiability formulas is a problem of theoretical and practical interest for the testing community. Recent works proposed two algorithms (namely UniGen and QuickSampler) for reaching a good compromise between execution time and uniformity guarantees, with empirical evidence on SAT benchmarks. In the context of highly-configurable software systems (e.g., Linux), it is unclear whether UniGen and QuickSampler can scale and sample uniform software configurations. In this paper, we perform a thorough experiment on 128 real-world feature models. We find that UniGen is unable to produce SAT solutions out of such feature models. Furthermore, we show that QuickSampler does not generate uniform samples and that some features are either never part of the sample or too frequently present. Finally, using a case study, we characterize the impacts of these results on the ability to find bugs in a configurable system. Overall, our results suggest that we are not there: more research is needed to explore the cost-effectiveness of uniform sampling when testing large configurable systems

Anwendungen von #SAT Solvern für Produktlinien: Masterarbeit

Product lines are widely used for managing families of similar products. Typically, product lines are complex and infeasible to analyze manually. In the last two decades, product-line analyses have been reduced to satisfiability problems which are well understood. However, there are methods for which satisfiability is not sufficient. Recently, researchers begun to reduce other problems to #SAT. Yet, only few applications have been considered and those are fairly limited in their scope. Furthermore, the authors mainly propose ad-hoc solutions that are only applicable under certain restrictions or do not scale to large product lines. In this thesis, we aim show the benefits of applying #SAT for the analysis of product lines. To this end, we make the following contributions: First, we summarize applications dependent on #AT considered in the literature and propose new applications to motivate the usage of #SAT technology. Second, we present a variety of algorithms and optimizations for these applications including new proposals. Third, we empirically evaluate 10 proposed algorithms with 14 off-the-shelf #SAT solvers on 131 industrial feature models to identify the fastest algorithms and solvers. Our results show that for each analysis at least one algorithm and solver scale on a vast majority of the feature models, whereas Linux and an automotive model not be analyzed at all. In addition, our results further reveal the benefits of knowledge compilation to deterministic decomposable negation normal form for performing counting-based analyses. Overall, our work shows that #SAT dependent analyses for feature models open a new variety of different applications and scale to a large number of industrial feature models.Produktlinien sind weit verbreitet für die Verwaltung von Familien verwandter Pro- dukte. In der Regel sind Produktlinien komplex und manuell schwer zu analysieren. In den letzten zwei Jahrzehnten wurden Produktlinienanalysen auf Erfüllbarkeit- sprobleme reduziert, für welche es eine Vielzahl an effizienten Werkzeugen gibt. Allerdings ist Erfüllbarkeit nicht für alle Analysen hinreichend. Kürzlich haben Forscher damit begonnen, andere Probleme auf #SAT zu reduzieren. Es wur- den jedoch nur wenige Anwendungen in Betracht gezogen und auch der Anwen- dungsbereich ist begrenzt. Darüber hinaus schlagen die Autoren hauptsächlich Ad-hoc-Lösungen vor, die nur unter bestimmten Einschränkungen der Produktlin- ien anwendbar sind oder nicht für große Produktlinien skalieren. In dieser Arbeit zeigen wir die Vorteile von #SAT Anwendungen für Produtlinien auf. Unser wis- senschaftlicher Beitrag besteht aus den folgenden drei Punkten: Zuerst fassen wir die in der Literatur betrachteten #SAT-Anwendungen zusammen und schlagen neue Anwendungen vor, um den Einsatz von #SAT-Technologien zu motivieren. Zweit- ens stellen wir eine Vielzahl von Algorithmen und Optimierungen für diese Anwen- dungen vor, einschließlich neuer Vorschläge. Drittens führen wir eine empirische Evaluation von 10 der vorgeschlagenen Algorithmen mit 14 #SAT-Solvern auf 131 industriellen Feature-Modellen aus, um die schnellsten Algorithmen und Solver zu identifizieren. Die Ergebnisse der Evaluation zeigen, dass wir für jede Analyse wenig- stens einen Algorithmus und Solver identifiziert haben, die für industrielle Feature- Modelle skalieren. Dazu sind die Ergebnisse ein starker Indikator für die Vorteile des Einsatzes von d-DNNFs bei #SAT-Anwendungen. Insgesamt zeigt unsere Ar- beit, dass #SAT-abhängige Analysen für Feature-Modelle eine Vielzahl neuer un- terschiedlicher Anwendungen ermöglicht und für viele industirelle Feature-Modelle skaliert