Automated metamorphic testing on the analyses of feature models

Copyright © 2010 Elsevier B.V. All rights reserved.Context: A feature model (FM) represents the valid combinations of features in a domain. The automated extraction of information from FMs is a complex task that involves numerous analysis operations, techniques and tools. Current testing methods in this context are manual and rely on the ability of the tester to decide whether the output of an analysis is correct. However, this is acknowledged to be time-consuming, error-prone and in most cases infeasible due to the combinatorial complexity of the analyses, this is known as the oracle problem.Objective: In this paper, we propose using metamorphic testing to automate the generation of test data for feature model analysis tools overcoming the oracle problem. An automated test data generator is presented and evaluated to show the feasibility of our approach.Method: We present a set of relations (so-called metamorphic relations) between input FMs and the set of products they represent. Based on these relations and given a FM and its known set of products, a set of neighbouring FMs together with their corresponding set of products are automatically generated and used for testing multiple analyses. Complex FMs representing millions of products can be efficiently created by applying this process iteratively.Results: Our evaluation results using mutation testing and real faults reveal that most faults can be automatically detected within a few seconds. Two defects were found in FaMa and another two in SPLOT, two real tools for the automated analysis of feature models. Also, we show how our generator outperforms a related manual suite for the automated analysis of feature models and how this suite can be used to guide the automated generation of test cases obtaining important gains in efficiency.Conclusion: Our results show that the application of metamorphic testing in the domain of automated analysis of feature models is efficient and effective in detecting most faults in a few seconds without the need for a human oracle.This work has been partially supported by the European Commission(FEDER)and Spanish Government under CICYT project SETI(TIN2009-07366)and the Andalusian Government project ISABEL(TIC-2533)

ROPocop - Dynamic Mitigation of Code-Reuse Attacks

Control-flow attacks, usually achieved by exploiting a buffer-overflow vulnerability, have been a serious threat to system security for over fifteen years. Researchers have answered the threat with various mitigation techniques, but nevertheless, new exploits that successfully bypass these technologies still appear on a regular basis. In this paper, we propose ROPocop, a novel approach for detecting and preventing the execution of injected code and for mitigating code-reuse attacks such as return-oriented programming (RoP). ROPocop uses dynamic binary instrumentation, requiring neither access to source code nor debug symbols or changes to the operating system. It mitigates attacks by both monitoring the program counter at potentially dangerous points and by detecting suspicious program flows. We have implemented ROPocop for Windows x86 using PIN, a dynamic program instrumentation framework from Intel. Benchmarks using the SPEC CPU2006 suite show an average overhead of 2.4x, which is comparable to similar approaches, which give weaker guarantees. Real-world applications show only an initially noticeable input lag and no stutter. In our evaluation our tool successfully detected all 11 of the latest real-world code-reuse exploits, with no false alarms. Therefore, despite the overhead, it is a viable, temporary solution to secure critical systems against exploits if a vendor patch is not yet available

An Instance-Oriented Approach to Constructing Product Lines from Layers

The Model/View/Controller (MVC) paradigm, and its many variants, is a cornerstone of decoupling within object-oriented design. MVC leads to clear reuse benefits regarding the class hierarchies for the model and view elements. In practice, however, the controllers appear to defy reuse, most likely because they encapsulate specialized business logic. Within an effective product line, however, such specialized logic must be reused. We combine the MVC paradigm with feature-oriented programming (FOP) to produce a novel instance-oriented design pattern for layers that brings reusability back to controllers. We demonstrate the effectiveness of our approach using a product-line example of a solitaire game engine

Spinal Test Suites for Software Product Lines

A major challenge in testing software product lines is efficiency. In particular, testing a product line should take less effort than testing each and every product individually. We address this issue in the context of input-output conformance testing, which is a formal theory of model-based testing. We extend the notion of conformance testing on input-output featured transition systems with the novel concept of spinal test suites. We show how this concept dispenses with retesting the common behavior among different, but similar, products of a software product line.Comment: In Proceedings MBT 2014, arXiv:1403.704

Middleware specialization using aspect oriented programming

Standardized middleware is used to build large distributed real-time and enterprise (DRE) systems. These middleware are highly flexible and support a large number of features since they have to be applicable to a wide range of domains and applications. This generality and flexibility, however, often causes many performance and footprint overheads par-ticularly for product line architectures, which have a well-defined scope smaller than that of the middleware yet must leverage its benefits, such as reusability. To alleviate this tension thus a key objective is to specialize the middleware, which comprises removing the sources of excessive general-ity while simultaneously optimizing the required features of middleware functionality. To meet this objective this paper describes how we have applied Aspect-Oriented Program-ming (AOP) in a novel manner to address these challenges. Although AOP is primarily used for separation of concerns, we use it to specialize middleware. Aspects are used to se-lect the specific set of features needed by the product line. Aspect weaving is subsequently used to specialize the mid-dleware. This paper describes the key motivation for our research, identifies the challenges developing middleware-based product lines and shows how to resolve those using aspects. The results applying our AOP-based specialization techniques to event demultiplexing middleware for the case of single threaded implementation showed 3 % decrease in latency and 2 % increase in throughput, while in the thread pool implementation showed 4 % decrease in latency and 3% increase in throughput

Mobile Media SPL creation by Feature IDE using FODA

Software Product Lines are used in many areas, combining to form new technologies and products. A product line is a group of products that share a common development platform and vary by the composition and implementation method for the functionalities. This paper describes the implementation or creation of MobileMedia feature model using FODA (Feature Oriented Domain Analysis) methodology using FeatureIDE eclipse plug-in. The feature model created in this depicts various outlines as feature model as visual model, collaboration diagram view of model, its configuration, FeatureIDE Statistics. Basically the paper shows the concept how SPLs can be viewed as feature diagrams using various tools in order to deal with them. This modelling has been widely used by software product line communities and a number of extensions have been proposed

Automated generation of computationally hard feature models using evolutionary algorithms

This is the post-print version of the final paper published in Expert Systems with Applications. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2014 Elsevier B.V.A feature model is a compact representation of the products of a software product line. The automated extraction of information from feature models is a thriving topic involving numerous analysis operations, techniques and tools. Performance evaluations in this domain mainly rely on the use of random feature models. However, these only provide a rough idea of the behaviour of the tools with average problems and are not sufficient to reveal their real strengths and weaknesses. In this article, we propose to model the problem of finding computationally hard feature models as an optimization problem and we solve it using a novel evolutionary algorithm for optimized feature models (ETHOM). Given a tool and an analysis operation, ETHOM generates input models of a predefined size maximizing aspects such as the execution time or the memory consumption of the tool when performing the operation over the model. This allows users and developers to know the performance of tools in pessimistic cases providing a better idea of their real power and revealing performance bugs. Experiments using ETHOM on a number of analyses and tools have successfully identified models producing much longer executions times and higher memory consumption than those obtained with random models of identical or even larger size.European Commission (FEDER), the Spanish Government and the Andalusian Government

An Investigation of Modular Dependencies in Aspects, Features and Classes

The essence of software design is to construct well-defined, encapsulated modules that are composed together to build the desired software application. There are several design paradigms in use today, including traditional Object-Oriented Programming (OOP), Feature-Oriented Programming (FOP), Aspect-Oriented Programming (AOP) and Instance-Oriented Programming (IOP). FOP studies the modularity of features in product lines, where a feature is an increment in program functionality. AOP aims to separate and modularize aspects when an aspect is a crosscutting concern. IOP, as an extension to FOP, makes the layers work like object factories. While each is good at solving different types of problems, they are closely related. The composition of modules is complicated because modules have (often hidden) dependencies on other modules. This thesis aims to better understand the way dependencies are managed by each approach. Based on this, we focus on the precedence issue in AOP and FOP, that is, how designers are able to specify the order by which modules are composed together. Different precedence means different semantics, but the current tools can not guarantee the correct precedence is adopted. We first solve the precedence issue separately for AOP and FOP, then based on this, we come up with a unified model to solve the precedence issue by using source code annotations to specify the precedence. We evaluate our technique with use cases