Confidence Intervals for the Area Under the Receiver Operating Characteristic Curve in the Presence of Ignorable Missing Data

Receiver operating characteristic curves are widely used as a measure of accuracy of diagnostic tests and can be summarised using the area under the receiver operating characteristic curve (AUC). Often, it is useful to construct a confidence interval for the AUC; however, because there are a number of different proposed methods to measure variance of the AUC, there are thus many different resulting methods for constructing these intervals. In this article, we compare different methods of constructing Wald‐type confidence interval in the presence of missing data where the missingness mechanism is ignorable. We find that constructing confidence intervals using multiple imputation based on logistic regression gives the most robust coverage probability and the choice of confidence interval method is less important. However, when missingness rate is less severe (e.g. less than 70%), we recommend using Newcombe\u27s Wald method for constructing confidence intervals along with multiple imputation using predictive mean matching

Evaluating the Quality of Research into a Single Prognostic Biomarker: A Systematic Review and Meta-analysis of 83 Studies of C-Reactive Protein in Stable Coronary Artery Disease

Background Systematic evaluations of the quality of research on a single prognostic biomarker are rare. We sought to evaluate the quality of prognostic research evidence for the association of C-reactive protein (CRP) with fatal and nonfatal events among patients with stable coronary disease. Methods and Findings We searched MEDLINE (1966 to 2009) and EMBASE (1980 to 2009) and selected prospective studies of patients with stable coronary disease, reporting a relative risk for the association of CRP with death and nonfatal cardiovascular events. We included 83 studies, reporting 61,684 patients and 6,485 outcome events. No study reported a prespecified statistical analysis protocol; only two studies reported the time elapsed (in months or years) between initial presentation of symptomatic coronary disease and inclusion in the study. Studies reported a median of seven items (of 17) from the REMARK reporting guidelines, with no evidence of change over time. The pooled relative risk for the top versus bottom third of CRP distribution was 1.97 (95% confidence interval [CI] 1.78–2.17), with substantial heterogeneity (I2 = 79.5). Only 13 studies adjusted for conventional risk factors (age, sex, smoking, obesity, diabetes, and low-density lipoprotein [LDL] cholesterol) and these had a relative risk of 1.65 (95% CI 1.39–1.96), I2 = 33.7. Studies reported ten different ways of comparing CRP values, with weaker relative risks for those based on continuous measures. Adjusting for publication bias (for which there was strong evidence, Egger's p<0.001) using a validated method reduced the relative risk to 1.19 (95% CI 1.13–1.25). Only two studies reported a measure of discrimination (c-statistic). In 20 studies the detection rate for subsequent events could be calculated and was 31% for a 10% false positive rate, and the calculated pooled c-statistic was 0.61 (0.57–0.66). Conclusion Multiple types of reporting bias, and publication bias, make the magnitude of any independent association between CRP and prognosis among patients with stable coronary disease sufficiently uncertain that no clinical practice recommendations can be made. Publication of prespecified statistical analytic protocols and prospective registration of studies, among other measures, might help improve the quality of prognostic biomarker research

What do we know and when do we know it?

Two essential aspects of virtual screening are considered: experimental design and performance metrics. In the design of any retrospective virtual screen, choices have to be made as to the purpose of the exercise. Is the goal to compare methods? Is the interest in a particular type of target or all targets? Are we simulating a ‘real-world’ setting, or teasing out distinguishing features of a method? What are the confidence limits for the results? What should be reported in a publication? In particular, what criteria should be used to decide between different performance metrics? Comparing the field of molecular modeling to other endeavors, such as medical statistics, criminology, or computer hardware evaluation indicates some clear directions. Taken together these suggest the modeling field has a long way to go to provide effective assessment of its approaches, either to itself or to a broader audience, but that there are no technical reasons why progress cannot be made

ATP and its N6-substituted analogues: parameterization, molecular dynamics simulation and conformational analysis

In this work we used a combination of classical molecular dynamics and simulated annealing techniques to shed more light on the conformational flexibility of 12 adenosine triphosphate (ATP) analogues in a water environment. We present simulations in AMBER force field for ATP and 12 published analogues [Shah et al. (1997) Proc Natl Acad Sci USA 94: 3565–3570]. The calculations were carried out using the generalized Born (GB) solvation model in the presence of the cation Mg2+. The ion was placed at a close distance (2 Å) from the charged oxygen atoms of the beta and gamma phosphate groups of the −3 negatively charged ATP analogue molecules. Analysis of the results revealed the distribution of inter-proton distances H8–H1′ and H8–H2′ versus the torsion angle ψ (C4–N9-C1′–O4′) for all conformations of ATP analogues. There are two gaps in the distribution of torsion angle ψ values: the first is between −30 and 30 degrees and is described by cis-conformation; and the second is between 90 and 175 degrees, which mostly covers a region of anti conformation. Our results compare favorably with results obtained in experimental assays [Jiang and Mao (2002) Polyhedron 21:435–438]

A tree-based decision model to support prediction of the severity of asthma exacerbations in children

This paper describes the development of a tree-based decision model to predict the severity of pediatric asthma exacerbations in the emergency department (ED) at 2 h following triage. The model was constructed from retrospective patient data abstracted from the ED charts. The original data was preprocessed to eliminate questionable patient records and to normalize values of age-dependent clinical attributes. The model uses attributes routinely collected in the ED and provides predictions even for incomplete observations. Its performance was verified on independent validating data (split-sample validation) where it demonstrated AUC (area under ROC curve) of 0.83, sensitivity of 84%, specificity of 71% and the Brier score of 0.18. The model is intended to supplement an asthma clinical practice guideline, however, it can be also used as a stand-alone decision tool

Poor reproducibility of compression elastography in the Achilles tendon: same day and consecutive day measurements.

OBJECTIVE To determine the reproducibility of compression elastography (CE) when measuring strain data, a measure of stiffness of the human Achilles tendon in vivo, over consecutive measures, consecutive days and when using different foot positions. MATERIALS AND METHODS Eight participants (4 males, 4 females; mean age 25.5 ± 2.51 years, range 21-30 years; height 173.6 ± 11.7 cm, range 156-189 cm) had five consecutive CE measurements taken on one day and a further five CE measures taken, one per day, at the same time of day, every day for a consecutive 5-day period. These 80 measurements were used to assess both the repeatability and reproducibility of the technique. Means, standard deviations, coefficient of variation (CV), Pearson correlation analysis (R) and intra-class correlation coefficients (ICC) were calculated. RESULTS For CE data, all CVs were above 53%, R values indicated no-to-weak correlations between measures at best (range 0.01-0.25), and ICC values were all classified in the poor category (range 0.00-0.11). CVs for length and diameter measures were acceptably low indicating a high level of reliability. CONCLUSIONS Given the wide variation obtained in the CE results, it was concluded that CE using this specific system has a low level of reproducibility for measuring the stiffness of the human Achilles tendon in vivo over consecutive days, consecutive measures and in different foot positions

Comparison of the diagnostic accuracy of commercial NS1-based diagnostic tests for early dengue infection

<p>Abstract</p> <p>Background</p> <p>We compared the diagnostic accuracy and reproducibility of commercially available NS1-based dengue tests and explored factors influencing their sensitivities.</p> <p>Methods</p> <p>Paired analysis of 310 samples previously characterized as positive (n = 218) and negative (n = 92) for viral isolation and/or RT-PCR and/or IgM seroconversion. Masked samples were tested by two observers with Platelia™ Dengue NS1 Ag, second generation Pan-E™ Dengue Early ELISA, SD Dengue NS1 Ag ELISA, Dengue NS1 Ag STRIP™, and SD BIOLINE™ Dengue Duo (NS1/IgM/IgG).</p> <p>Results</p> <p>SD BIOLINE™ NS1/IgM/IgG had the highest sensitivity (80.7% 95%CI 75-85.7) with likelihood ratios of 7.4 (95%CI 4.1-13.8) and 0.21 (95%CI 0.16-0.28). The ELISA-format tests showed comparable sensitivities; all below 75%. STRIP™ and SD NS1 had even lower sensitivities (<65%). The sensitivities significantly decreased in samples taken after 3 days of fever onset, in secondary infections, viral serotypes 2 and 4, and severe dengue. Adding IgM or IgG to SD NS1 increased its sensitivity in all these situations.</p> <p>Conclusions</p> <p>The simultaneous detection of NS1/IgM/IgG would be potentially useful for dengue diagnosis in both endemic and non endemic areas. A negative result does not rule out dengue. Further studies are required to assess the performance and impact of early laboratory diagnosis of dengue in the routine clinical setting.</p

Morphological analysis of the sheathed flagellum of Brucella melitensis

<p>Abstract</p> <p>Background</p> <p>It was recently shown that <it>B. melitensis </it>is flagellated. However, the flagellar structure remains poorly described.</p> <p>Findings</p> <p>We analyzed the structure of the polar sheathed flagellum of <it>B. melitensis </it>by TEM analysis and demonstrated that the Ryu staining is a good method to quickly visualize the flagellum by optical microscopy. The TEM analysis demonstrated that an extension of the outer membrane surrounds a filament ending by a club-like structure. The Δ<it>ftcR</it>, Δ<it>fliF</it>, Δ<it>flgE </it>and Δ<it>fliC </it>flagellar mutants still produce an empty sheath.</p> <p>Conclusions</p> <p>Our results demonstrate that the flagellum of <it>B. melitensis </it>has the characteristics of the sheathed flagella. Our results also suggest that the flagellar sheath production is not directly linked to the flagellar structure assembly and is not regulated by the FtcR master regulator.</p