Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory

The comparison of benchmark error sets is an essential tool for the evaluation of theories in computational chemistry. The standard ranking of methods by their Mean Unsigned Error is unsatisfactory for several reasons linked to the non-normality of the error distributions and the presence of underlying trends. Complementary statistics have recently been proposed to palliate such deficiencies, such as quantiles of the absolute errors distribution or the mean prediction uncertainty. We introduce here a new score, the systematic improvement probability (SIP), based on the direct system-wise comparison of absolute errors. Independently of the chosen scoring rule, the uncertainty of the statistics due to the incompleteness of the benchmark data sets is also generally overlooked. However, this uncertainty is essential to appreciate the robustness of rankings. In the present article, we develop two indicators based on robust statistics to address this problem: P_{inv}, the inversion probability between two values of a statistic, and \mathbf{P}_{r}, the ranking probability matrix. We demonstrate also the essential contribution of the correlations between error sets in these scores comparisons

State-dependent Kalman filters for robust engine control

Vehicle emissions variations impose significant challenges to the automotive industry. In these simulation studies, nonlinear estimation techniques based on state-dependent and extended Kalman filtering are developed for spark ignition engines to enhance robustness of the feedforward fuel controllers to changes in nominal system parameters and measurement errors. A model-based approach is used to derive the optimal filters. Numerical simulations indicate the superiority of estimation-based approaches to enhance robustness of in-cylinder air estimation which directly contributes to the precision of engine exhaust air-fuel ratio and, consequently the consistency of the tailpipe emissions. The results obtained are for an aggressive driving profile and are presented and discusse

The noise properties of 42 millisecond pulsars from the European Pulsar Timing Array and their impact on gravitational wave searches

The sensitivity of Pulsar Timing Arrays to gravitational waves depends on the noise present in the individual pulsar timing data. Noise may be either intrinsic or extrinsic to the pulsar. Intrinsic sources of noise will include rotational instabilities, for example. Extrinsic sources of noise include contributions from physical processes which are not sufficiently well modelled, for example, dispersion and scattering effects, analysis errors and instrumental instabilities. We present the results from a noise analysis for 42 millisecond pulsars (MSPs) observed with the European Pulsar Timing Array. For characterising the low-frequency, stochastic and achromatic noise component, or "timing noise", we employ two methods, based on Bayesian and frequentist statistics. For 25 MSPs, we achieve statistically significant measurements of their timing noise parameters and find that the two methods give consistent results. For the remaining 17 MSPs, we place upper limits on the timing noise amplitude at the 95% confidence level. We additionally place an upper limit on the contribution to the pulsar noise budget from errors in the reference terrestrial time standards (below 1%), and we find evidence for a noise component which is present only in the data of one of the four used telescopes. Finally, we estimate that the timing noise of individual pulsars reduces the sensitivity of this data set to an isotropic, stochastic GW background by a factor of >9.1 and by a factor of >2.3 for continuous GWs from resolvable, inspiralling supermassive black-hole binaries with circular orbits.Comment: Accepted for publication by the Monthly Notices of the Royal Astronomical Societ

Geodesic least squares regression on information manifolds

We present a novel regression method targeted at situations with significant uncertainty on both the dependent and independent variables or with non-Gaussian distribution models. Unlike the classic regression model, the conditional distribution of the response variable suggested by the data need not be the same as the modeled distribution. Instead they are matched by minimizing the Rao geodesic distance between them. This yields a more flexible regression method that is less constrained by the assumptions imposed through the regression model. As an example, we demonstrate the improved resistance of our method against some flawed model assumptions and we apply this to scaling laws in magnetic confinement fusion