DPRESS: Localizing estimates of predictive uncertainty

<p>Abstract</p> <p>Background</p> <p>The need to have a quantitative estimate of the uncertainty of prediction for QSAR models is steadily increasing, in part because such predictions are being widely distributed as tabulated values disconnected from the models used to generate them. Classical statistical theory assumes that the error in the population being modeled is independent and identically distributed (IID), but this is often not actually the case. Such inhomogeneous error (heteroskedasticity) can be addressed by providing an individualized estimate of predictive uncertainty for each particular new object <it>u</it>: the standard error of prediction <it>s</it>_ucan be estimated as the non-cross-validated error <it>s</it>_t*for the closest object <it>t</it>* in the training set adjusted for its separation <it>d </it>from <it>u </it>in the descriptor space relative to the size of the training set.</p> <p><display-formula><graphic file="1758-2946-1-11-i1.gif"/></display-formula></p> <p>The predictive uncertainty factor <it>γ</it>_t*is obtained by distributing the internal predictive error sum of squares across objects in the training set based on the distances between them, hence the acronym: <it>D</it>istributed <it>PR</it>edictive <it>E</it>rror <it>S</it>um of <it>S</it>quares (DPRESS). Note that <it>s</it>_t*and <it>γ</it>_t*are characteristic of each training set compound contributing to the model of interest.</p> <p>Results</p> <p>The method was applied to partial least-squares models built using 2D (molecular hologram) or 3D (molecular field) descriptors applied to mid-sized training sets (<it>N </it>= 75) drawn from a large (<it>N </it>= 304), well-characterized pool of cyclooxygenase inhibitors. The observed variation in predictive error for the external 229 compound test sets was compared with the uncertainty estimates from DPRESS. Good qualitative and quantitative agreement was seen between the distributions of predictive error observed and those predicted using DPRESS. Inclusion of the distance-dependent term was essential to getting good agreement between the estimated uncertainties and the observed distributions of predictive error. The uncertainty estimates derived by DPRESS were conservative even when the training set was biased, but not excessively so.</p> <p>Conclusion</p> <p>DPRESS is a straightforward and powerful way to reliably estimate individual predictive uncertainties for compounds outside the training set based on their distance to the training set and the internal predictive uncertainty associated with its nearest neighbor in that set. It represents a sample-based, <it>a posteriori </it>approach to defining applicability domains in terms of localized uncertainty.</p

The effects of activating a “baby brain” stereotype on pregnant women’s cognitive functioning

Throughout pregnancy and into the immediate postpartum period, women are generally perceived to be incompetent, stressed, and forgetful. However, the neuropsychological “baby brain” literature remains unclear and contradictory. Across two studies, we provide the first experimental tests of whether perceived cognitive impairment in pregnancy can be explained by stereotype threat theory, which proposes that awareness of negative stereotypes about one’s ingroup can harm performance. In Study 1 (N = 364), we tested stereotype threat effects in a 2 (stereotype threat versus no threat) × 3 (pregnant women versus new mothers versus never-pregnant female control) design. We observed a main effect of group on memory performance (pregnant women and new mothers performed worse than controls), but no other main or interactive effects. Study 2 (N = 409) aimed to extend these research questions with mathematics ability, memory, and attention as the dependent variables. Again, we found that a stereotype threat manipulation did not impair pregnant women and new mothers’ cognitive performance, nor was there any interactive effects. Groups also did not differ in their performance. We discuss these results in the context of stereotype threat mechanisms, calling into question whether a stereotype threat paradigm can be applied effectively to pregnancy-related stereotypes. This work has implications for the advancement of stereotype threat as a theory and contributes to the reappraisal of the utility of stereotype threat as a way of understanding how stereotypes affect performance