Multi-objective optimisation for regression testing

Regression testing is the process of retesting a system after it or its environment has changed. Many techniques aim to find the cheapest subset of the regression test suite that achieves full coverage. More recently, it has been observed that the tester might want to have a range of solutions providing different trade-offs between cost and one or more forms of coverage, this being a multi-objective optimisation problem. This paper further develops the multi-objective agenda by adapting a decomposition-based multi-objective evolutionary algorithm (MOEA/D). Experiments evaluated four approaches: a classic greedy algorithm; non-dominated sorting genetic algorithm II (NSGA-II); MOEA/D with a fixed value for a parameter $c$; and MOEA/D in which tuning was used to choose the value of $c$. These used six programs from the SIR repository and one larger program, VoidAuth. In all of the experiments MOEA/D with tuning was the most effective technique. The relative performance of the other techniques varied, although MOEA/D with fixed $c$ outperformed NSGA-II on the larger programs (Space and VoidAuth)

Model fusion using fuzzy aggregation: Special applications to metal properties

To improve the modelling performance, one should either propose a new modelling methodology or make the best of existing models. In this paper, the study is concentrated on the latter solution, where a structure-free modelling paradigm is proposed. It does not rely on a fixed structure and can combine various modelling techniques in ‘symbiosis’ using a ‘master fuzzy system’. This approach is shown to be able to include the advantages of different modelling techniques altogether by requiring less training and by minimising the efforts relating optimisation of the final structure. The proposed approach is then successfully applied to the industrial problems of predicting machining induced residual stresses for aerospace alloy components as well as modelling the mechanical properties of heat-treated alloy steels, both representing complex, non-linear and multi-dimensional environments

Which Surrogate Works for Empirical Performance Modelling? A Case Study with Differential Evolution

It is not uncommon that meta-heuristic algorithms contain some intrinsic parameters, the optimal configuration of which is crucial for achieving their peak performance. However, evaluating the effectiveness of a configuration is expensive, as it involves many costly runs of the target algorithm. Perhaps surprisingly, it is possible to build a cheap-to-evaluate surrogate that models the algorithm's empirical performance as a function of its parameters. Such surrogates constitute an important building block for understanding algorithm performance, algorithm portfolio/selection, and the automatic algorithm configuration. In principle, many off-the-shelf machine learning techniques can be used to build surrogates. In this paper, we take the differential evolution (DE) as the baseline algorithm for proof-of-concept study. Regression models are trained to model the DE's empirical performance given a parameter configuration. In particular, we evaluate and compare four popular regression algorithms both in terms of how well they predict the empirical performance with respect to a particular parameter configuration, and also how well they approximate the parameter versus the empirical performance landscapes

A hierarchical Mamdani-type fuzzy modelling approach with new training data selection and multi-objective optimisation mechanisms: A special application for the prediction of mechanical properties of alloy steels

In this paper, a systematic data-driven fuzzy modelling methodology is proposed, which allows to construct Mamdani fuzzy models considering both accuracy (precision) and transparency (interpretability) of fuzzy systems. The new methodology employs a fast hierarchical clustering algorithm to generate an initial fuzzy model efficiently; a training data selection mechanism is developed to identify appropriate and efficient data as learning samples; a high-performance Particle Swarm Optimisation (PSO) based multi-objective optimisation mechanism is developed to further improve the fuzzy model in terms of both the structure and the parameters; and a new tolerance analysis method is proposed to derive the confidence bands relating to the final elicited models. This proposed modelling approach is evaluated using two benchmark problems and is shown to outperform other modelling approaches. Furthermore, the proposed approach is successfully applied to complex high-dimensional modelling problems for manufacturing of alloy steels, using ‘real’ industrial data. These problems concern the prediction of the mechanical properties of alloy steels by correlating them with the heat treatment process conditions as well as the weight percentages of the chemical compositions

Integrated system to perform surrogate based aerodynamic optimisation for high-lift airfoil

This work deals with the aerodynamics optimisation of a generic two-dimensional three element high-lift configuration. Although the high-lift system is applied only during take-off and landing in the low speed phase of the flight the cost efficiency of the airplane is strongly influenced by it [1]. The ultimate goal of an aircraft high lift system design team is to define the simplest configuration which, for prescribed constraints, will meet the take-off, climb, and landing requirements usually expressed in terms of maximum L/D and/or maximum CL. The ability of the calculation method to accurately predict changes in objective function value when gaps, overlaps and element deflections are varied is therefore critical. Despite advances in computer capacity, the enormous computational cost of running complex engineering simulations makes it impractical to rely exclusively on simulation for the purpose of design optimisation. To cut down the cost, surrogate models, also known as metamodels, are constructed from and then used in place of the actual simulation models. This work outlines the development of integrated systems to perform aerodynamics multi-objective optimisation for a three-element airfoil test case in high lift configuration, making use of surrogate models available in MACROS Generic Tools, which has been integrated in our design tool. Different metamodeling techniques have been compared based on multiple performance criteria. With MACROS is possible performing either optimisation of the model built with predefined training sample (GSO) or Iterative Surrogate-Based Optimization (SBO). In this first case the model is build independent from the optimisation and then use it as a black box in the optimisation process. In the second case is needed to provide the possibility to call CFD code from the optimisation process, and there is no need to build any model, it is being built internally during the optimisation process. Both approaches have been applied. A detailed analysis of the integrated design system, the methods as well as th

Gauging risk with higher moments : handrails in measuring and optimising conditional value at risk

The aim of the paper is to study empirically the influence of higher moments of the return distribution on conditional value at risk (CVaR). To be more exact, we attempt to reveal the extent to which the risk given by CVaR can be estimated when relying on the mean, standard deviation, skewness and kurtosis. Furthermore, it is intended to study how this relationship can be utilised in portfolio optimisation. First, based on a database of 600 individual equity returns from 22 emerging world markets, factor models incorporating the first four moments of the return distribution have been constructed at different confidence levels for CVaR, and the contribution of the identified factors in explaining CVaR was determined. Following this the influence of higher moments was examined in portfolio context, i.e. asset allocation decisions were simulated by creating emerging market portfolios from the viewpoint of US investors. This can be regarded as a normal decisionmaking process of a hedge fund focusing on investments into emerging markets. In our analysis we compared and contrasted two approaches with which one can overcome the shortcomings of the variance as a risk measure. First of all, we solved in the presence of conflicting higher moment preferences a multi-objective portfolio optimisation problem for different sets of preferences. In addition, portfolio optimisation was performed in the mean-CVaR framework characterised by using CVaR as a measure of risk. As a part of the analysis, the pair-wise comparison of the different higher moment metrics of the meanvariance and the mean-CVaR efficient portfolios were also made. Throughout the work special attention was given to implied preferences to the different higher moments in optimising CVaR. We also examined the extent to which model risk, namely the risk of wrongly assuming normally-distributed returns can deteriorate our optimal portfolio choice. JEL Classification: G11, G15, C6

Multi-objective improvement of software using co-evolution and smart seeding

Optimising non-functional properties of software is an important part of the implementation process. One such property is execution time, and compilers target a reduction in execution time using a variety of optimisation techniques. Compiler optimisation is not always able to produce semantically equivalent alternatives that improve execution times, even if such alternatives are known to exist. Often, this is due to the local nature of such optimisations. In this paper we present a novel framework for optimising existing software using a hybrid of evolutionary optimisation techniques. Given as input the implementation of a program or function, we use Genetic Programming to evolve a new semantically equivalent version, optimised to reduce execution time subject to a given probability distribution of inputs. We employ a co-evolved population of test cases to encourage the preservation of the program’s semantics, and exploit the original program through seeding of the population in order to focus the search. We carry out experiments to identify the important factors in maximising efficiency gains. Although in this work we have optimised execution time, other non-functional criteria could be optimised in a similar manner