Quality Assurance of Integrated Business Software: An Approach to Testing Software Product Lines

The use of integrated business software can be instrumental in reducing the difficulties occurring when various information systems have to be integrated. As a downside of this and due to the fact that these systems are designed to be used in all sorts of enterprises, the internal complexity of these systems increases exponentially. Software product lines on the one hand promise remedy by the conscious use of variability, on the other hand create new demands on quality assurance. The article on hand provides a theoretical framework for evaluating approaches to software testing, regarding their use in the development of software product lines. It turns out that only a practice-oriented approach emphasizing the buyer’s view will be successful in the end

Integration of Quality Attributes in Software Product Line Development

Different approaches for building modern software systems in complex and open environments have been proposed in the last few years. Some efforts try to apply Software Product Line (SPL) approach to take advantage of the massive reuse for producing software systems that share a common set of features. In general quality assurance is a crucial activity for success in software industry, but it is even more important when talking about Software Product Lines since the intensive reuse of assets makes the quality attributes (a measurable physical or abstract property of an entity) of the assets to be transmitted to the whole SPL scope. However, despite the importance that quality has in software product line development, most of the methodologies being applied in Software Product Line Development focus only on managing the commonalities and variability within the product line and not giving support to the non--¿ functional requirements that the products must fit. The main goal of this master final work is to introduce quality attributes in early stages of software product line development processes by means of the definition of a production plan that, on one hand, integrates quality as an additional view for describing the extension of the software product line and, on the other hand introduces the quality attributes as a decision factor during product configuration and when selecting among design alternatives. Our approach has been defined following the Model--¿ Driven Software Development paradigm. Therefore all the software artifacts defined had its correspondent metamodels and the processes defined rely on automated model transformations. Finally in order to illustrate the feasibility of the approach we have integrated the quality view in an SPL example in the context of safety critical embedded systems on the automotive domain.González Huerta, J. (2011). Integration of Quality Attributes in Software Product Line Development. http://hdl.handle.net/10251/15835Archivo delegad

Colored model based testing for software product lines (CMBT-SWPL)

Over the last decade, the software product line domain has emerged as one of the mostpromising software development paradigms. The main benefits of a software product lineapproach are improvements in productivity, time to market, product quality, and customersatisfaction.Therefore, one topic that needs greater emphasis is testing of software product lines toachieve the required software quality assurance. Our concern is how to test a softwareproduct line as early as possible in order to detect errors, because the cost of error detectedIn early phases is much less compared to the cost of errors when detected later.The method suggested in this thesis is a model-based, reuse-oriented test technique calledColored Model Based Testing for Software Product Lines (CMBT-SWPL). CMBT-SWPLis a requirements-based approach for efficiently generating tests for products in a soft-ware product line. This testing approach is used for validation and verification of productlines. It is a novel approach to test product lines using a Colored State Chart (CSC), whichconsiders variability early in the product line development process. More precisely, the vari-ability will be introduced in the main components of the CSC. Accordingly, the variabilityis preserved in test cases, as they are generated from colored test models automatically.During domain engineering, the CSC is derived from the feature model. By coloring theState Chart, the behavior of several product line variants can be modeled simultaneouslyin a single diagram and thus address product line variability early. The CSC representsthe test model, from which test cases using statistical testing are derived.During application engineering, these colored test models are customized for a specificapplication of the product line. At the end of this test process, the test cases are generatedagain using statistical testing, executed and the test results are ready for evaluation. Inxaddition, the CSC will be transformed to a Colored Petri Net (CPN) for verification andsimulation purposes.The main gains of applying the CMBT-SWPL method are early detection of defects inrequirements, such as ambiguities incompleteness and redundancy which is then reflectedin saving the test effort, time, development and maintenance costs

Feature Nets: behavioural modelling of software product lines

Software product lines (SPL) are diverse systems that are developed using a dual engineering process: (a)family engineering defines the commonality and variability among all members of the SPL, and (b) application engineering derives specific products based on the common foundation combined with a variable selection of features. The number of derivable products in an SPL can thus be exponential in the number of features. This inherent complexity poses two main challenges when it comes to modelling: Firstly, the formalism used for modelling SPLs needs to be modular and scalable. Secondly, it should ensure that all products behave correctly by providing the ability to analyse and verify complex models efficiently. In this paper we propose to integrate an established modelling formalism (Petri nets) with the domain of software product line engineering. To this end we extend Petri nets to Feature Nets. While Petri nets provide a framework for formally modelling and verifying single software systems, Feature Nets offer the same sort of benefits for software product lines. We show how SPLs can be modelled in an incremental, modular fashion using Feature Nets, provide a Feature Nets variant that supports modelling dynamic SPLs, and propose an analysis method for SPL modelled as Feature Nets. By facilitating the construction of a single model that includes the various behaviours exhibited by the products in an SPL, we make a significant step towards efficient and practical quality assurance methods for software product lines

Predicting Line-Level Defects by Capturing Code Contexts with Hierarchical Transformers

Software defects consume 40% of the total budget in software development and cost the global economy billions of dollars every year. Unfortunately, despite the use of many software quality assurance (SQA) practices in software development (e.g., code review, continuous integration), defects may still exist in the official release of a software product. Therefore, prioritizing SQA efforts for the vulnerable areas of the codebase is essential to ensure the high quality of a software release. Predicting software defects at the line level could help prioritize the SQA effort but is a highly challenging task given that only ~3% of lines of a codebase could be defective. Existing works on line-level defect prediction often fall short and cannot fully leverage the line-level defect information. In this paper, we propose Bugsplorer, a novel deep-learning technique for line-level defect prediction. It leverages a hierarchical structure of transformer models to represent two types of code elements: code tokens and code lines. Unlike the existing techniques that are optimized for file-level defect prediction, Bugsplorer is optimized for a line-level defect prediction objective. Our evaluation with five performance metrics shows that Bugsplorer has a promising capability of predicting defective lines with 26-72% better accuracy than that of the state-of-the-art technique. It can rank the first 20% defective lines within the top 1-3% suspicious lines. Thus, Bugsplorer has the potential to significantly reduce SQA costs by ranking defective lines higher

The software product assurance metrics study: JPL's software systems quality and productivity

The findings are reported of the Jet Propulsion Laboratory (JPL)/Software Product Assurance (SPA) Metrics Study, conducted as part of a larger JPL effort to improve software quality and productivity. Until recently, no comprehensive data had been assembled on how JPL manages and develops software-intensive systems. The first objective was to collect data on software development from as many projects and for as many years as possible. Results from five projects are discussed. These results reflect 15 years of JPL software development, representing over 100 data points (systems and subsystems), over a third of a billion dollars, over four million lines of code and 28,000 person months. Analysis of this data provides a benchmark for gauging the effectiveness of past, present and future software development work. In addition, the study is meant to encourage projects to record existing metrics data and to gather future data. The SPA long term goal is to integrate the collection of historical data and ongoing project data with future project estimations

Understanding Variability-Aware Analysis in Low-Maturity Variant-Rich Systems

Context: Software systems often exist in many variants to support varying stakeholder requirements, such as specific market segments or hardware constraints. Systems with many variants (a.k.a. variant-rich systems) are highly complex due to the variability introduced to support customization. As such, assuring the quality of these systems is also challenging since traditional single-system analysis techniques do not scale when applied. To tackle this complexity, several variability-aware analysis techniques have been conceived in the last two decades to assure the quality of a branch of variant-rich systems called software product lines. Unfortunately, these techniques find little application in practice since many organizations do use product-line engineering techniques, but instead rely on low-maturity \clo~strategies to manage their software variants. For instance, to perform an analysis that checks that all possible variants that can be configured by customers (or vendors) in a car personalization system conform to specified performance requirements, an organization needs to explicitly model system variability. However, in low-maturity variant-rich systems, this and similar kinds of analyses are challenging to perform due to (i) immature architectures that do not systematically account for variability, (ii) redundancy that is not exploited to reduce analysis effort, and (iii) missing essential meta-information, such as relationships between features and their implementation in source code.Objective: The overarching goal of the PhD is to facilitate quality assurance in low-maturity variant-rich systems. Consequently, in the first part of the PhD (comprising this thesis) we focus on gaining a better understanding of quality assurance needs in such systems and of their properties.Method: Our objectives are met by means of (i) knowledge-seeking research through case studies of open-source systems as well as surveys and interviews with practitioners; and (ii) solution-seeking research through the implementation and systematic evaluation of a recommender system that supports recording the information necessary for quality assurance in low-maturity variant-rich systems. With the former, we investigate, among other things, industrial needs and practices for analyzing variant-rich systems; and with the latter, we seek to understand how to obtain information necessary to leverage variability-aware analyses.Results: Four main results emerge from this thesis: first, we present the state-of-practice in assuring the quality of variant-rich systems, second, we present our empirical understanding of features and their characteristics, including information sources for locating them; third, we present our understanding of how best developers\u27 proactive feature location activities can be supported during development; and lastly, we present our understanding of how features are used in the code of non-modular variant-rich systems, taking the case of feature scattering in the Linux kernel.Future work: In the second part of the PhD, we will focus on processes for adapting variability-aware analyses to low-maturity variant-rich systems.Keywords:\ua0Variant-rich Systems, Quality Assurance, Low Maturity Software Systems, Recommender Syste