What accuracy statistics really measure

Provides the software estimation research community with a better understanding of the meaning of, and relationship between, two statistics that are often used to assess the accuracy of predictive models: the mean magnitude relative error (MMRE) and the number of predictions within 25% of the actual, pred(25). It is demonstrated that MMRE and pred(25) are, respectively, measures of the spread and the kurtosis of the variable z, where z=estimate/actual. Thus, z is considered to be a measure of accuracy, and statistics such as MMRE and pred(25) to be measures of properties of the distribution of z. It is suggested that measures of the central location and skewness of z, as well as measures of spread and kurtosis, are necessary. Furthermore, since the distribution of z is non-normal, non-parametric measures of these properties may be needed. For this reason, box-plots of z are useful alternatives to simple summary metrics. It is also noted that the simple residuals are better behaved than the z variable, and could also be used as the basis for comparing prediction system

The value of coskewness in evaluating mutual funds

Recent asset pricing studies demonstrate the relevance of incorporating the coskewness in Asset Pricing Models, and illustrate how this component helps to explain the time variation of ex-ante market risk premiums. This paper analyzes the role of coskewness in mutual funds performance evaluation. We find evidence that adding a coskewness factor is economically and statistically significant. We document that some managers are managing the coskewness and show, in general, a persistent behaviour on time in their coskewness policy. One of the most striking results is that many negative (positive) alpha funds measured relative to the CAPM risk adjustments would be reclassified as positive (negative) alpha funds using a model with coskewness. Therefore, a ranking of funds based on risk adjusted returns without considering coskewness would generate an erroneous classification. Moreover, some fund characteristics, such as the turnover ratio or the category, are related to the likelihood of managing coskewness

The value of coskewness in mutual fund performance evaluation.

Recent asset pricing studies demonstrate the relevance of incorporating coskewness in asset pricing models, and illustrate how this component helps to explain the time variation of ex-ante market risk premiums. This paper analyzes the role of coskewness in mutual fund performance evaluation and finds evidence that adding a coskewness factor is economically and statistically significant. It documents that coskewness is sometimes managed and shows persistence of the coskewness policy over time. One of the most striking results is that many negative (positive) alpha funds, measured relative to the CAPM risk adjustments, would be reclassified as positive (negative) alpha funds using a model with coskewness. Therefore, performance ranking based on risk-adjusted returns without considering coskewness could generate an erroneous classification. Moreover, some fund characteristics, such as turnover ratio or category, are related to the likelihood of managing coskewness.Coskewness; Mutual funds; Performance measures;

Convex mixture regression for quantitative risk assessment

There is wide interest in studying how the distribution of a continuous response changes with a predictor. We are motivated by environmental applications in which the predictor is the dose of an exposure and the response is a health outcome. A main focus in these studies is inference on dose levels associated with a given increase in risk relative to a baseline. In addressing this goal, popular methods either dichotomize the continuous response or focus on modeling changes with the dose in the expectation of the outcome. Such choices may lead to information loss and provide inaccurate inference on dose-response relationships. We instead propose a Bayesian convex mixture regression model that allows the entire distribution of the health outcome to be unknown and changing with the dose. To balance flexibility and parsimony, we rely on a mixture model for the density at the extreme doses, and express the conditional density at each intermediate dose via a convex combination of these extremal densities. This representation generalizes classical dose-response models for quantitative outcomes, and provides a more parsimonious, but still powerful, formulation compared to nonparametric methods, thereby improving interpretability and efficiency in inference on risk functions. A Markov chain Monte Carlo algorithm for posterior inference is developed, and the benefits of our methods are outlined in simulations, along with a study on the impact of dde exposure on gestational age