Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory

The comparison of benchmark error sets is an essential tool for the evaluation of theories in computational chemistry. The standard ranking of methods by their Mean Unsigned Error is unsatisfactory for several reasons linked to the non-normality of the error distributions and the presence of underlying trends. Complementary statistics have recently been proposed to palliate such deficiencies, such as quantiles of the absolute errors distribution or the mean prediction uncertainty. We introduce here a new score, the systematic improvement probability (SIP), based on the direct system-wise comparison of absolute errors. Independently of the chosen scoring rule, the uncertainty of the statistics due to the incompleteness of the benchmark data sets is also generally overlooked. However, this uncertainty is essential to appreciate the robustness of rankings. In the present article, we develop two indicators based on robust statistics to address this problem: P_{inv}, the inversion probability between two values of a statistic, and \mathbf{P}_{r}, the ranking probability matrix. We demonstrate also the essential contribution of the correlations between error sets in these scores comparisons

Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors

Benchmarking studies in computational chemistry use reference datasets to assess the accuracy of a method through error statistics. The commonly used error statistics, such as the mean signed and mean unsigned errors, do not inform end-users on the expected amplitude of prediction errors attached to these methods. We show that, the distributions of model errors being neither normal nor zero-centered, these error statistics cannot be used to infer prediction error probabilities. To overcome this limitation, we advocate for the use of more informative statistics, based on the empirical cumulative distribution function of unsigned errors, namely (1) the probability for a new calculation to have an absolute error below a chosen threshold, and (2) the maximal amplitude of errors one can expect with a chosen high confidence level. Those statistics are also shown to be well suited for benchmarking and ranking studies. Moreover, the standard error on all benchmarking statistics depends on the size of the reference dataset. Systematic publication of these standard errors would be very helpful to assess the statistical reliability of benchmarking conclusions.Comment: Supplementary material: https://github.com/ppernot/ECDF

Range separation combined with the Overhauser model: Application to the H2_2 molecule along the dissociation curve

The combination of density-functional theory with other approaches to the many-electron problem through the separation of the electron-electron interaction into a short-range and a long-range contribution (range separation) is a successful strategy, which is raising more and more interest in recent years. We focus here on a range-separated method in which only the short-range correlation energy needs to be approximated, and we model it within the "extended Overhauser approach". We consider the paradigmatic case of the H$_2$ molecule along the dissociation curve, finding encouraging results. By means of very accurate variational wavefunctions, we also study how the effective electron-electron interaction appearing in the Overhauser model should be in order to yield the exact correlation energy for standard Kohn-Sham density functional theory.Comment: submitted to Int. J. Quantum Chem., special issue dedicated to Prof. Hira

Study of the discontinuity of the exchange-correlation potential in an exactly soluble case

It was found by Perdew, Parr, Levy, and Balduz [Phys. Rev. Lett. {\bf 49}, 1691 (1982)] and by Sham and Schl\"uter [Phys. Rev. Lett. {\bf 51}, 1884 (1983)] that the exact Kohn-Sham exchange-correlation potential of an open system may jump discontinuosly as the particle number crosses an integer, with important physical consequences. Recently, Sagvolden and Perdew [Phys. Rev. A {\bf 77}, 012517 (2008)] have analyzed the discontinuity of the exchange-correlation potential as the particle number crosses one, with an illustration that uses a model density for the H$^-$ ion. In this work, we extend their analysis to the case in which the external potential is the simple harmonic confinement, choosing spring-constant values for which the two-electron hamiltonian has an analytic solution. This way, we can obtain the exact, analytic, exchange and correlation potentials for particle number fluctuating between zero and two, illustrating the discontinuity as the particle number crosses one without introducing any model or approximation. We also discuss exchange and correlation separately.Comment: Submitted to Int. J. Quantum Chem., special issue honoring Prof. Mayer. New version, where an important error has been correcte