What accuracy statistics really measure

Provides the software estimation research community with a better understanding of the meaning of, and relationship between, two statistics that are often used to assess the accuracy of predictive models: the mean magnitude relative error (MMRE) and the number of predictions within 25% of the actual, pred(25). It is demonstrated that MMRE and pred(25) are, respectively, measures of the spread and the kurtosis of the variable z, where z=estimate/actual. Thus, z is considered to be a measure of accuracy, and statistics such as MMRE and pred(25) to be measures of properties of the distribution of z. It is suggested that measures of the central location and skewness of z, as well as measures of spread and kurtosis, are necessary. Furthermore, since the distribution of z is non-normal, non-parametric measures of these properties may be needed. For this reason, box-plots of z are useful alternatives to simple summary metrics. It is also noted that the simple residuals are better behaved than the z variable, and could also be used as the basis for comparing prediction system

Why the Virtual Nature of Software Makes It Ideal for Search Based Optimization

This paper(1) provides a motivation for the application of search based optimization to Software Engineering, an area that has come to be known as Search Based Software Engineering (SBSE). SBSE techniques have already been applied to many problems throughout the Software Engineering lifecycle, with new application domains emerging on a regular basis. The approach is very generic and therefore finds wide application in Software Engineering. It facilitates automated and semi-automated solutions in situations typified by large complex problem spaces with multiple competing and conflicting objectives. Previous work has already discussed, in some detail, the advantages of the SBSE approach for Software Engineering. This paper summarises previous work and goes further, by arguing that Software Engineering provides the ideal set of application problems for which optimization algorithms are supremely well suited