A statistical framework to evaluate virtual screening

<p>Abstract</p> <p>Background</p> <p>Receiver operating characteristic (ROC) curve is widely used to evaluate virtual screening (VS) studies. However, the method fails to address the "early recognition" problem specific to VS. Although many other metrics, such as RIE, BEDROC, and pROC that emphasize "early recognition" have been proposed, there are no rigorous statistical guidelines for determining the thresholds and performing significance tests. Also no comparisons have been made between these metrics under a statistical framework to better understand their performances.</p> <p>Results</p> <p>We have proposed a statistical framework to evaluate VS studies by which the threshold to determine whether a ranking method is better than random ranking can be derived by bootstrap simulations and 2 ranking methods can be compared by permutation test. We found that different metrics emphasize "early recognition" differently. BEDROC and RIE are 2 statistically equivalent metrics. Our newly proposed metric SLR is superior to pROC. Through extensive simulations, we observed a "seesaw effect" – overemphasizing early recognition reduces the statistical power of a metric to detect true early recognitions.</p> <p>Conclusion</p> <p>The statistical framework developed and tested by us is applicable to any other metric as well, even if their exact distribution is unknown. Under this framework, a threshold can be easily selected according to a pre-specified type I error rate and statistical comparisons between 2 ranking methods becomes possible. The theoretical null distribution of SLR metric is available so that the threshold of SLR can be exactly determined without resorting to bootstrap simulations, which makes it easy to use in practical virtual screening studies.</p

In Vitro Amplification of Misfolded Prion Protein Using Lysate of Cultured Cells

Protein misfolding cyclic amplification (PMCA) recapitulates the prion protein (PrP) conversion process under cell-free conditions. PMCA was initially established with brain material and then with further simplified constituents such as partially purified and recombinant PrP. However, availability of brain material from some species or brain material from animals with certain mutations or polymorphisms within the PrP gene is often limited. Moreover, preparation of native PrP from mammalian cells and tissues, as well as recombinant PrP from bacterial cells, involves time-consuming purification steps. To establish a convenient and versatile PMCA procedure unrestricted to the availability of substrate sources, we attempted to conduct PMCA with the lysate of cells that express cellular PrP (PrPC). PrPSc was efficiently amplified with lysate of rabbit kidney epithelial RK13 cells stably transfected with the mouse or Syrian hamster PrP gene. Furthermore, PMCA was also successful with lysate of other established cell lines of neuronal or non-neuronal origins. Together with the data showing that the abundance of PrPC in cell lysate was a critical factor to drive efficient PrPSc amplification, our results demonstrate that cell lysate in which PrPC is present abundantly serves as an excellent substrate source for PMCA

Pleiotropic Roles of a Ribosomal Protein in Dictyostelium discoideum

The cell cycle phase at starvation influences post-starvation differentiation and morphogenesis in Dictyostelium discoideum. We found that when expressed in Saccharomyces cerevisiae, a D. discoideum cDNA that encodes the ribosomal protein S4 (DdS4) rescues mutations in the cell cycle genes cdc24, cdc42 and bem1. The products of these genes affect morphogenesis in yeast via a coordinated moulding of the cytoskeleton during bud site selection. D. discoideum cells that over- or under-expressed DdS4 did not show detectable changes in protein synthesis but displayed similar developmental aberrations whose intensity was graded with the extent of over- or under-expression. This suggested that DdS4 might influence morphogenesis via a stoichiometric effect – specifically, by taking part in a multimeric complex similar to the one involving Cdc24p, Cdc42p and Bem1p in yeast. In support of the hypothesis, the S. cerevisiae proteins Cdc24p, Cdc42p and Bem1p as well as their D. discoideum cognates could be co-precipitated with antibodies to DdS4. Computational analysis and mutational studies explained these findings: a C-terminal domain of DdS4 is the functional equivalent of an SH3 domain in the yeast scaffold protein Bem1p that is central to constructing the bud site selection complex. Thus in addition to being part of the ribosome, DdS4 has a second function, also as part of a multi-protein complex. We speculate that the existence of the second role can act as a safeguard against perturbations to ribosome function caused by spontaneous variations in DdS4 levels