The state of adoption and the challenges of systematic variability management in industry

Handling large-scale software variability is still a challenge for many organizations. After decades of research on variability management concepts, many industrial organizations have introduced techniques known from research, but still lament that pure textbook approaches are not applicable or efficient. For instance, software product line engineering—an approach to systematically develop portfolios of products—is difficult to adopt given the high upfront investments; and even when adopted, organizations are challenged by evolving their complex product lines. Consequently, the research community now mainly focuses on re-engineering and evolution techniques for product lines; yet, understanding the current state of adoption and the industrial challenges for organizations is necessary to conceive effective techniques. In this multiple-case study, we analyze the current adoption of variability management techniques in twelve medium- to large-scale industrial cases in domains such as automotive, aerospace or railway systems. We identify the current state of variability management, emphasizing the techniques and concepts they adopted. We elicit the needs and challenges expressed for these cases, triangulated with results from a literature review. We believe our results help to understand the current state of adoption and shed light on gaps to address in industrial practice.This work is supported by Vinnova Sweden, Fond Unique Interminist´eriel (FUI) France, and the Swedish Research Council. Open access funding provided by University of Gothenbur

Software product line testing - a systematic mapping study

Context: Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped. Objective: This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research. Method: A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type. Results: A majority of the papers are of proposal research types (64 %). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority. Conclusions: More validation and evaluation research is needed to provide a better foundation for SPL testing

Generalizing production testing operations for IoT devices

Abstract. A rapidly increasing number of new IoT products entering the market puts strain on the testing effort required to manufacture them. Every device needs to be tested at the manufacturing site before it can be shipped to the customer. This testing process during manufacturing is called Production Testing. If test automation systems running these tests are developed with a single system engineering approach, the number of test automation systems becomes unmaintainable. In addition to this, the development cost of such test automation system has to be covered by each product. Reusability of test automation and test assets is low when resources cannot be efficiently shared between products. Existing solutions for generalizing testing effort from a single system approach to support multiple products were reviewed from the literature. Software Product Line Engineering is recognized as a possible solution, but its adoption requires organizational, economical, and technical changes. This thesis studied technical solutions for how test automation system could be developed to support the testing of multiple products. Test automation system was designed based on existing literature, and two example products were used to mimic realistic IoT products. Work followed to define test requirements for two example products, implement tests, and execute them for the first example product. After tests passed for the first example product, they were executed for the second product. Test failures and evident problems were marked as variation points, and they were analysed. Test attributes that needed to be varied were recognized, and four different sources for that information were identified. Test logic was identified as one of the sources for attributes, and there was no need to variate it. Matching configuration was created for other sources: component, test, and hardware configuration. Tests were successfully executed for the second example product after introducing the variation via configuration files. Prototype implementation succeeded in its goal to create production test automation system capable of testing two different example products using common test assets. Needed variation was introduced successfully through configuration files. This thesis shows that general test assets can be created for production testing, despite the fact that production testing is tightly coupled to the target hardware. Future work continues by testing additional hardware platforms to reveal more variation points. This helps to develop production testing test automation to support a wider range of hardware platforms and components. Storing the hardware-specific configuration data to the device looks promising topic for further study

From concept to validation of a wireless environmental sensor for the integral application of preventive conservation methodologies in low-budget museums

[EN] The effective implementation of preventive conservation strategies requires tools to continuously measure the environmental conditions to which the cultural objects are exposed. In this sense, the European Horizon 2020 project CollectionCare aims to provide an affordable preventive conservation service for individual objects focused to small museums with limited budgets. Although the use of data loggers has been a must in the past, new deployments tend to use wired and wireless sensors that provide real-time information and the ability to instantly analyse the data, allowing immediate action to be taken in the event of a threat towards a cultural object. For already constructed buildings, wireless systems have the advantage that, a priori, deployments are simpler, faster and cheaper, but have to deal with complex heritage environments with long distances to be covered and very thick walls to cross. In many cases, commercial systems of this type are not economically viable for small museums with limited budgets. Moreover, conservators who try to approach such solutions are often overwhelmed by unclear technical specifications that do not allow them to determine whether the solution fits their environment or not, giving rise to great frustration. Among others, the CollectionCare includes the design of a specific low-cost wireless sensor, being the aim of this article to present to both technical and non-technical readers, the design choices made regarding the housing, attachment method, power source, wireless transmission technique and selection of the environmental sensors following European standards. Also, the effective implementation of the device in three stages to prove the concep until to get a near-production version is presented. The developed device has been deployed in museums in Belgium, Italy, Greece, Latvia, Denmark and Spain, and the validation results are presented, showing that it is feasible to have a cost-effective proposal that it is easy to install and configure and can operate for 10 years without the need for maintenance or battery replacement except if it is needed to comply with annual recalibration if standards such as ISO 11799:2015 are required.This research was funded by the European Union's Horizon 2020 research and innovation programme under Grant agreement No.814624.Laborda, J.; García-Castillo, AM.; Mercado Romero, R.; Peiró-Vitoria, A.; Perles, A. (2022). From concept to validation of a wireless environmental sensor for the integral application of preventive conservation methodologies in low-budget museums. Heritage Science. 10(1):1-17. https://doi.org/10.1186/s40494-022-00837-911710

Supporting the grow-and-prune model for evolving software product lines

207 p.Software Product Lines (SPLs) aim at supporting the development of a whole family of software products through a systematic reuse of shared assets. To this end, SPL development is separated into two interrelated processes: (1) domain engineering (DE), where the scope and variability of the system is defined and reusable core-assets are developed; and (2) application engineering (AE), where products are derived by selecting core assets and resolving variability. Evolution in SPLs is considered to be more challenging than in traditional systems, as both core-assets and products need to co-evolve. The so-called grow-and-prune model has proven great flexibility to incrementally evolve an SPL by letting the products grow, and later prune the product functionalities deemed useful by refactoring and merging them back to the reusable SPL core-asset base. This Thesis aims at supporting the grow-and-prune model as for initiating and enacting the pruning. Initiating the pruning requires SPL engineers to conduct customization analysis, i.e. analyzing how products have changed the core-assets. Customization analysis aims at identifying interesting product customizations to be ported to the core-asset base. However, existing tools do not fulfill engineers needs to conduct this practice. To address this issue, this Thesis elaborates on the SPL engineers' needs when conducting customization analysis, and proposes a data-warehouse approach to help SPL engineers on the analysis. Once the interesting customizations have been identified, the pruning needs to be enacted. This means that product code needs to be ported to the core-asset realm, while products are upgraded with newer functionalities and bug-fixes available in newer core-asset releases. Herein, synchronizing both parties through sync paths is required. However, the state of-the-art tools are not tailored to SPL sync paths, and this hinders synchronizing core-assets and products. To address this issue, this Thesis proposes to leverage existing Version Control Systems (i.e. git/Github) to provide sync operations as first-class construct

Visualizing the customization endeavor in product-based-evolving software product lines: a case of action design research

[EN] Software Product Lines (SPLs) aim at systematically reusing software assets, and deriving products (a.k.a., variants) out of those assets. However, it is not always possible to handle SPL evolution directly through these reusable assets. Time-to-market pressure, expedited bug fixes, or product specifics lead to the evolution to first happen at the product level, and to be later merged back into the SPL platform where the core assets reside. This is referred to as product-based evolution. In this scenario, deciding when and what should go into the next SPL release is far from trivial. Distinct questions arise. How much effort are developers spending on product customization? Which are the most customized core assets? To which extent is the core asset code being reused for a given product? We refer to this endeavor as Customization Analysis, i.e., understanding the functional increments in adjusting products from the last SPL platform release. The scale of the SPLs' code-base calls for customization analysis to be conducted through Visual Analytics tools. This work addresses the design principles for such tools through a joint effort between academia and industry, specifically, Danfoss Drives, a company division in charge of the P400 SPL. Accordingly, we adopt an Action Design Research approach where answers are sought by interacting with the practitioners in the studied situations. We contribute by providing informed goals for customization analysis as well as an intervention in terms of a visual analytics tool. We conclude by discussing to what extent this experience can be generalized to product-based evolving SPL organizations other than Danfoss Drives.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work is supported by the Spanish Ministry of Science, Innovation and Universities grant number RTI2018099818-B-I00 and MCIU-AEI TIN2017-90644-REDT (TASOVA). ONEKIN enjoys support from the program 'Grupos de Investigacion del Sistema Univesitario Vasco 2019-2021' under contract IT1235-19. Raul Medeiros enjoys a doctoral grant from the Spanish Ministry of Science and Innovation

Exploring regression testing and software product line testing - research and state of practice

In large software organizations with a product line development approach a selective testing of product variants is necessary in order to keep pace with the decreased development time for new products, enabled by the systematic reuse. The close relationship between products in product line indicates an option to reduce the testing effort due to redundancy. In many cases test selection is performed manually, based on test leaders’ expertise. This makes the cost and quality of the testing highly dependent on the skills and experience of the test leaders. There is a need in industry for systematic approaches to test selection. The goal of our research is to improve the control of the testing and reduce the amount of redundant testing in the product line context by applying regression test selection strategies. In this thesis, the state of art of regression testing and software product line testing are explored. Two extensive systematic reviews are conducted as well as an industrial survey of regression testing state of practice and an industrial evaluation of a pragmatic regression test selection strategy. Regression testing is not an isolated one-off activity, but rather an activity of varying scope and preconditions, strongly dependent on the context in which it is applied. Several techniques for regression test selection are proposed and evaluated empirically but in many cases the context is too specific for a technique to be easily applied directly by software developers. In order to improve the possibility for generalizing empirical results on regression test selection, guidelines for reporting the testing context are discussed in this thesis. Software product line testing is a relatively new research area. The understanding about challenges is well established but when looking for solutions to these challenges, we mostly find proposals, and empirical evaluations are sparse. Regression test selection strategies proposed in literature are not easily applicable in the product line context. Instead, control may be increased by increased visibility of the effects of testing and proper measurements of software quality. Focus of our future work will be on how to guide the planning and assessment of regression testing activities in large, complex reuse based systems, by visualizing the quality achieved in different parts of the system and evaluating the effects of different selection strategies when applied in various regression testing situations