Calibration : the Achilles heel of predictive analytics

Acknowledgements This work was developed as part of the international STRengthening Analytical Thinking for Observational Studies (STRATOS) initiative. The objective of STRATOS is to provide accessible and accurate guidance in the design and analysis of observational studies (http://stratos-initiative.org/). Members of the STRATOS Topic Group ‘Evaluating diagnostic tests and prediction models’ are (alphabetically) Patrick Bossuyt, Gary S. Collins, Petra Macaskill, David J. McLernon, Karel G.M. Moons, Ewout W. Steyerberg, Ben Van Calster, Maarten van Smeden, and Andrew Vickers. Funding This work was funded by the Research Foundation – Flanders (FWO; grant G0B4716N) and Internal Funds KU Leuven (grant C24/15/037). The funders had no role in study design, data collection, data analysis, interpretation of results, or writing of the manuscript. Contributions All authors conceived of the study. BVC drafted the manuscript. All authors reviewed and edited the manuscript and approved the final version.Peer reviewedPublisher PD

Development of a heptaplex PCR assay for identification of Staphylococcus aureus and CoNS with simultaneous detection of virulence and antibiotic resistance genes

Background Staphylococcal toxicity and antibiotic resistance (STAAR) have been menacing public health. Although vancomycin-resistant Staphylococcus aureus (VRSA) is currently not as widespread as methicillin-resistant S. aureus (MRSA), genome evolution of MRSA into VRSA, including strains engineered within the same patient under anti-staphylococcal therapy, may build up to future public health concern. To further complicate diagnosis, infection control and anti-microbial chemotherapy, non-sterile sites such as the nares and the skin could contain both S. aureus and coagulase-negative staphylococci (CoNS), either of which could harbour mecA the gene driving staphylococcal methicillin-resistance and required for MRSA-VRSA evolution. Results A new heptaplex PCR assay has been developed which simultaneously detects seven markers for: i) eubacteria (16S rRNA), ii) Staphylococcus genus (tuf), iii) Staphylococcus aureus (spa), iv) CoNS (cns), v) Panton-Valentine leukocidin (pvl), vi) methicillin resistance (mecA), and vii) vancomycin resistance (vanA). Following successful validation using 255 reference bacterial strains, applicability to analyse clinical samples was evaluated by direct amplification in spiked blood cultures (n = 89) which returned 100 % specificity, negative and positive predictive values. The new assay has LoD of 1.0x103 CFU/mL for the 16S rRNA marker and 1.0x104 CFU/mL for six other markers and completes cycling in less than one hour. Conclusion The speed, sensitivity (100 %), NPV (100 %) and PPV (100 %) suggest the new heptaplex PCR assay could be easily integrated into a routine diagnostic microbiology workflow. Detection of the cns marker allows for unique identification of CoNS in mono-microbial and in poly-microbial samples containing mixtures of CoNS and S. aureus without recourse to the conventional elimination approach which is ambiguous. In addition to the SA-CoNS differential diagnostic essence of the new assay, inclusion of vanA primers will allow microbiology laboratories to stay ahead of the emerging MRSA-VRSA evolution. To the best of our knowledge, the new heptaplex PCR assay is the most multiplexed among similar PCR-based assays for simultaneous detection of STAAR

Nucleic acid amplification tests in the diagnosis of tuberculous pleuritis: a systematic review and meta-analysis

BACKGROUND: Conventional tests for tuberculous pleuritis have several limitations. A variety of new, rapid tests such as nucleic acid amplification tests – including polymerase chain reaction – have been evaluated in recent times. We conducted a systematic review to determine the accuracy of nucleic acid amplification (NAA) tests in the diagnosis of tuberculous pleuritis. METHODS: A systematic review and meta-analysis of 38 English and Spanish articles (with 40 studies), identified via searches of six electronic databases, hand searching of selected journals, and contact with authors, experts, and test manufacturers. Sensitivity, specificity, and other measures of accuracy were pooled using random effects models. Summary receiver operating characteristic curves were used to summarize overall test performance. Heterogeneity in study results was formally explored using subgroup analyses. RESULTS: Of the 40 studies included, 26 used in-house ("home-brew") tests, and 14 used commercial tests. Commercial tests had a low overall sensitivity (0.62; 95% confidence interval [CI] 0.43, 0.77), and high specificity (0.98; 95% CI 0.96, 0.98). The positive and negative likelihood ratios for commercial tests were 25.4 (95% CI 16.2, 40.0) and 0.40 (95% CI 0.24, 0.67), respectively. All commercial tests had consistently high specificity estimates; the sensitivity estimates, however, were heterogeneous across studies. With the in-house tests, both sensitivity and specificity estimates were significantly heterogeneous. Clinically meaningful summary estimates could not be determined for in-house tests. CONCLUSIONS: Our results suggest that commercial NAA tests may have a potential role in confirming (ruling in) tuberculous pleuritis. However, these tests have low and variable sensitivity and, therefore, may not be useful in excluding (ruling out) the disease. NAA test results, therefore, cannot replace conventional tests; they need to be interpreted in parallel with clinical findings and results of conventional tests. The accuracy of in-house nucleic acid amplification tests is poorly defined because of heterogeneity in study results. The clinical applicability of in-house NAA tests remains unclear

Influence of the calcium concentration in the presence of organic phosphorus on the physicochemical compatibility and stability of all-in-one admixtures for neonatal use

<p>Abstract</p> <p>Background</p> <p>Preterm infants need high amounts of calcium and phosphorus for bone mineralization, which is difficult to obtain with parenteral feeding due to the low solubility of these salts. The objective of this study was to evaluate the physicochemical compatibility of high concentrations of calcium associated with organic phosphate and its influence on the stability of AIO admixtures for neonatal use.</p> <p>Methods</p> <p>Three TPN admixture formulas were prepared in multilayered bags. The calcium content of the admixtures was adjusted to 0, 46.5 or 93 mg/100 ml in the presence of a fixed organic phosphate concentration as well as lipids, amino acids, inorganic salts, glucose, vitamins and oligoelements at pH 5.5. Each admixture was stored at 4°C, 25°C or 37°C and evaluated over a period of 7 days. The physicochemical stability parameters evaluated were visual aspect, pH, sterility, osmolality, peroxide formation, precipitation, and the size of lipid globules.</p> <p>Results</p> <p>Color alterations occurred from the first day on, and reversible lipid film formation from the third day of study for the admixtures stored at 25°C and 37°C. According to the parameters evaluated, the admixtures were stable at 4°C; and none of them presented precipitated particles due to calcium/phosphate incompatibility or lipid globules larger than 5 μm, which is the main parameter currently used to evaluate lipid emulsion stability. The admixtures maintained low peroxide levels and osmolarity was appropriate for parenteral administration.</p> <p>Conclusion</p> <p>The total calcium and calcium/phosphorus ratios studied appeared not to influence the physicochemical compatibility and stability of AIO admixtures.</p

Serum screening with Down's syndrome markers to predict pre-eclampsia and small for gestational age: Systematic review and meta-analysis

<p>Abstract</p> <p>Background</p> <p>Reliable antenatal identification of pre-eclampsia and small for gestational age is crucial to judicious allocation of monitoring resources and use of preventative treatment with the prospect of improving maternal/perinatal outcome. The purpose of this systematic review was to determine the accuracy of five serum analytes used in Down's serum screening for prediction of pre-eclampsia and/or small for gestational age.</p> <p>Methods</p> <p>The data sources included Medline, Embase, Cochrane library, Medion (inception to February 2007), hand searching of relevant journals, reference list checking of included articles, contact with experts. Two reviewers independently selected the articles in which the accuracy of an analyte used in Downs's serum screening before the 25^thgestational week was associated with the occurrence of pre-eclampsia and/or small for gestational age without language restrictions. Two authors independently extracted data on study characteristics, quality and results.</p> <p>Results</p> <p>Five serum screening markers were evaluated. 44 studies, testing 169,637 pregnant women (4376 pre-eclampsia cases) and 86 studies, testing 382,005 women (20,339 fetal growth restriction cases) met the selection criteria. The results showed low predictive accuracy overall. For pre-eclampsia the best predictor was inhibin A>2.79MoM positive likelihood ratio 19.52 (8.33,45.79) and negative likelihood ratio 0.30 (0.13,0.68) (single study). For small for gestational age it was AFP>2.0MoM to predict birth weight < 10^thcentile with birth < 37 weeks positive likelihood ratio 27.96 (8.02,97.48) and negative likelihood ratio 0.78 (0.55,1.11) (single study). A potential clinical application using aspirin as a treatment is given as an example.</p> <p>There were methodological and reporting limitations in the included studies thus studies were heterogeneous giving pooled results with wide confidence intervals.</p> <p>Conclusion</p> <p>Down's serum screening analytes have low predictive accuracy for pre-eclampsia and small for gestational age. They may be a useful means of risk assessment or of use in prediction when combined with other tests.</p

Criteria for the use of omics-based predictors in clinical trials: Explanation and elaboration

High-throughput 'omics' technologies that generate molecular profiles for biospecimens have been extensively used in preclinical studies to reveal molecular subtypes and elucidate the biological mechanisms of disease, and in retrospective studies on clinical specimens to develop mathematical models to predict clinical endpoints. Nevertheless, the translation of these technologies into clinical tests that are useful for guiding management decisions for patients has been relatively slow. It can be difficult to determine when the body of evidence for an omics-based test is sufficiently comprehensive and reliable to support claims that it is ready for clinical use, or even that it is ready for definitive evaluation in a clinical trial in which it may be used to direct patient therapy. Reasons for this difficulty include the exploratory and retrospective nature of many of these studies, the complexity of these assays and their application to clinical specimens, and the many potential pitfalls inherent in the development of mathematical predictor models from the very high-dimensional data generated by these omics technologies. Here we present a checklist of criteria to consider when evaluating the body of evidence supporting the clinical use of a predictor to guide patient therapy. Included are issues pertaining to specimen and assay requirements, the soundness of the process for developing predictor models, expectations regarding clinical study design and conduct, and attention to regulatory, ethical, and legal issues. The proposed checklist should serve as a useful guide to investigators preparing proposals for studies involving the use of omics-based tests. The US National Cancer Institute plans to refer to these guidelines for review of proposals for studies involving omics tests, and it is hoped that other sponsors will adopt the checklist as well. © 2013 McShane et al.; licensee BioMed Central Ltd