Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors

Benchmarking studies in computational chemistry use reference datasets to assess the accuracy of a method through error statistics. The commonly used error statistics, such as the mean signed and mean unsigned errors, do not inform end-users on the expected amplitude of prediction errors attached to these methods. We show that, the distributions of model errors being neither normal nor zero-centered, these error statistics cannot be used to infer prediction error probabilities. To overcome this limitation, we advocate for the use of more informative statistics, based on the empirical cumulative distribution function of unsigned errors, namely (1) the probability for a new calculation to have an absolute error below a chosen threshold, and (2) the maximal amplitude of errors one can expect with a chosen high confidence level. Those statistics are also shown to be well suited for benchmarking and ranking studies. Moreover, the standard error on all benchmarking statistics depends on the size of the reference dataset. Systematic publication of these standard errors would be very helpful to assess the statistical reliability of benchmarking conclusions.Comment: Supplementary material: https://github.com/ppernot/ECDF

RESOURCE ALLOCATION AND DECISION ANALYSIS FOR AGRICULTURAL RESEARCH

Research and Development/Tech Change/Emerging Technologies,

Transitivity and inter-definition consistency of neo neuroticism-domain ratings

It has been suggested that psychometrics is a pathological science on the basis that its normal processes prevent the attainment of its stated goal and conflicting interests prevent its constituents from acknowledging that this is the case. This study addressed basic measurement concerns associated with self- and other-report personality measures that are not addressed under the prevailing psychometric for establishing psychometric validity. Participants completed a series of pairwise comparisons (i.e., "more," "less," "equal") of themselves and known others with respect to 13 NEO Neuroticism-domain constructs. These judgments were examined for triplet transitivity; participants often made judgments that violated the condition of transitivity, suggesting that researchers are not justified in self- and other-ratings with respect to these attributes as ordinal or quantitative relations. Participants also compared themselves and known others in a complex ranking procedures with respect to three broad-level definitions of Neuroticism that appear in the NEO literature. Responses were examined for rank consistency across definitions. Low rates of rank consistency across definitions suggest that these definitions are not practically synonymous among lay persons. These studies do not prove that people cannot consistently order person with respect to Neuroticism-domain attributes, nor do they prove that multiple definitions of Neuroticism evoke different constructs for all people; however, in the absence of evidence that self- and other-ratings are ordinal and in the presence of evidence that they are often not ordinal, it is inappropriate to treat them as representations of rank or quantity. Likewise, in the absence of demonstrations that multiple definitions of Neuroticism are practically synonymous to individual raters and in the presence of evidence that they are often not practically synonymous, it is inappropriate to use them interchangeably. In brief, these are the kinds of measurement tasks that researchers need to engage in order to justify the measurement claims they are already making. These methods are recommended for use in conjunction with conversations with raters as a means to confront and correct the present state of psychometrics

Relationships between personal values, and depressed mood and subjective wellbeing.

The central objective of this thesis was to explore relationships between personal values, and depressed mood and subjective wellbeing, and to determine if the notion of values can be more useful in the fields of clinical and positive psychology. An initial literature review of values identified the potential importance of values in relation to mood and wellbeing, but also showed that more research was required to clearly establish such links. Two survey studies using Schwartz’s model of values (Schwartz, 1992), and one longitudinal study investigating relational aspects of values, were conducted to explore these relationships. Study 1 was a New Zealand paper-based study and investigated links between the importance of, and satisfaction with, values and depressed mood and subjective wellbeing. Study 2 was a larger international internet-based study which sought to replicate important findings from Study 1 and investigate links between people’s knowledge of their values and the extent to which they were living in alignment with values. Study 3 consisted of a sub-sample of participants from Study Two who completed a subset of Study 2 assessment measures six months later. This study explored how relational aspects of values (knowledge of values, living in alignment with values) related to changes in depressed mood and SWB over time. Cumulatively the results from these studies re-orientate our thinking towards an increased utility for the notion of values in the areas of clinical and positive psychology. Regarding depressed mood, these studies found links between greater depressed mood and lesser importance of Self-Direction, Stimulation and Hedonism value types. The importance of values as a whole was not associated with depressed mood; however being satisfied, knowing values, and living in alignment with values were associated with less depressed mood. Regarding subjective wellbeing, these studies found links between greater subjective wellbeing and greater importance of Self-Direction, Stimulation, Hedonism, and Benevolence value types. The importance of values as a whole was not associated with subjective wellbeing; however being satisfied, knowing values, and living in alignment with values were associated with greater subjective wellbeing. A causal relationship was found between living in alignment with values and latter subjective wellbeing, but not for knowledge of values and later subjective wellbeing. In addition, no major deviations in the coherence of values’ systems between individuals with and without depressed mood, or for individuals with and without high subjective wellbeing, were found. Strengths, implications, and limitations of the studies are noted for the fields of clinical and positive psychology, and suggestions for future research are made