Model comparison with Sharpe ratios

We show how to conduct asymptotically valid tests of model comparison when the extent of model mispricing is gauged by the squared Sharpe ratio improvement measure. This is equivalent to ranking models on their maximum Sharpe ratios, effectively extending the Gibbons, Ross, and Shanken (1989) test to accommodate the comparison of nonnested models. Mimicking portfolios can be substituted for any nontraded model factors, and estimation error in the portfolio weights is taken into account in the statistical inference. A variant of the Fama and French (2018) 6-factor model, with a monthly updated version of the usual value spread, emerges as the dominant model

Informative Features for Model Comparison

Given two candidate models, and a set of target observations, we address the problem of measuring the relative goodness of fit of the two models. We propose two new statistical tests which are nonparametric, computationally efficient (runtime complexity is linear in the sample size), and interpretable. As a unique advantage, our tests can produce a set of examples (informative features) indicating the regions in the data domain where one model fits significantly better than the other. In a real-world problem of comparing GAN models, the test power of our new test matches that of the state-of-the-art test of relative goodness of fit, while being one order of magnitude faster.Comment: Accepted to NIPS 201

Bayesian Model comparison of Higgs couplings

We investigate the possibility of contributions from physics beyond the Standard Model (SM) to the Higgs couplings, in the light of the LHC data. The work is performed within an interim framework where the magnitude of the Higgs production and decay rates are rescaled though Higgs coupling scale factors. We perform Bayesian parameter inference on these scale factors, concluding that there is good compatibility with the SM. Furthermore, we carry out Bayesian model comparison on all models where any combination of scale factors can differ from their SM values and find that typically models with fewer free couplings are strongly favoured. We consider the evidence that each coupling individually equals the SM value, making the minimal assumptions on the other couplings. Finally, we make a comparison of the SM against a single "not-SM" model, and find that there is moderate to strong evidence for the SM.Comment: 24 pages, 4 figure

Introducing doubt in Bayesian model comparison

There are things we know, things we know we dont know, and then there are things we dont know we dont know. In this paper we address the latter two issues in a Bayesian framework, introducing the notion of doubt to quantify the degree of (dis)belief in a model given observational data in the absence of explicit alternative models. We demonstrate how a properly calibrated doubt can lead to model discovery when the true model is unknown

Process model comparison based on cophenetic distance

The automated comparison of process models has received increasing attention in the last decade, due to the growing existence of process models and repositories, and the consequent need to assess similarities between the underlying processes. Current techniques for process model comparison are either structural (based on graph edit distances), or behavioural (through activity profiles or the analysis of the execution semantics). Accordingly, there is a gap between the quality of the information provided by these two families, i.e., structural techniques may be fast but inaccurate, whilst behavioural are accurate but complex. In this paper we present a novel technique, that is based on a well-known technique to compare labeled trees through the notion of Cophenetic distance. The technique lays between the two families of methods for comparing a process model: it has an structural nature, but can provide accurate information on the differences/similarities of two process models. The experimental evaluation on various benchmarks sets are reported, that position the proposed technique as a valuable tool for process model comparison.Peer ReviewedPostprint (author's final draft