Feature selection and validated predictive performance in the domain of Legionella pneumophila: a comparative study

textabstractBackground: Genetic comparisons of clinical and environmental Legionella strains form an essential part of outbreak investigations. DNA microarrays often comprise many DNA markers (features). Feature selection and the development of prediction models are particularly challenging in this domain with many variables and comparatively few subjects or data points. We aimed to compare modeling strategies to develop prediction models for classifying infections as clinical or environmental. Methods: We applied a bootstrap strategy for preselecting important features to a database containing 222 Legionella pneumophila strains with 448 continuous markers and a dichotomous outcome (clinical or environmental). Feature selection was done with 50 bootstrap samples resulting in a top 10 of most important features for each of four modeling techniques: classification and regression trees (CART), random forests (RF), support vector machines (SVM) and least absolute shrinkage and selection operator (LASSO). Validation was done in a second bootstrap resampling loop (200x) for evaluation of discriminatory model performance according to the AUC. Results: The top 5 of selected features differed considerably between the various modeling techniques, with only one common feature ("LePn.007B8"). The mean validated AUC-values of the SVM model and the CART model were 0.859 and 0.873 respectively. The LASSO and the RF model showed higher validated AUC-values (0.925 and 0.975 respectively). Conclusions: In the domain of Legionella pneumophila, which comprises many potential features for classifying of infections as clinical or environmental, the RF and LASSO techniques provide good prediction models. The identification of potentially biologically relevant features is highly dependent on the technique used, and should hence be interpreted with caution

Incorporating natural variation into IVF clinic league tables: The Expected Rank

Background. Rankings based on outcome are often used to present health care provider performance. These rankings do however not reflect that part of the variation in outcome between providers is caused by natural variation, and not by any differences in quality of care. The aim of this study is to compare standard methods for ranking with a novel method that takes into account natural variation. Methods. We used data on the number of treatment cycles and the number of pregnancies of 13 Dutch IVF clinics from 2004. We calculated the Expected Rank (ER), an estimate of the true rank of a provider, accounting for natural variation. We rescaled the ER to obtain the Percentile based on ER (PCER), that can be interpreted as the probability that a clinic is worse than a randomly selected other clinic. We also calculated a measure for rankability ρ, which is the part of variation between providers that is due to true differences (as opposed to natural variation). Results. The expected ranks ranged from 1.4 to 11.9 instead of the original ranks 1-13. The ER showed that some clinics performed very similar, which would be disregarded when using standard ranks. The PCER ranged from 7% to 88%. Rankability was substantial (ρ = 0.9). Conclusion. The Expected Rank provides a way to combine the attractiveness of a ranking, a single number and easy interpretation, with reliable analyses that does justice to the providers, and also allows individual comparisons

Sensitivity and specificity of single IgA and IgG antibody concentrations for early diagnosis of pertussis in adults: an evaluation for outbreak management in public health practice

Background: An accurate, practical laboratory test is needed to confirm clinical diagnosis of pertussis in adults during the first 3 symptomatic weeks, when treatment is effective and transmission can be interrupted. Methods: The sensitivity and specificity of single IgA and IgG levels were assessed in a cohort study of a pertussis epidemic in 99 adults in a closed community. Sensitivities were assessed in the sera of 46 laboratory confirmed clinical pertussis cases during the first 3 weeks. Specificities were calculated in sera of 35 asymptomatic controls without clinical symptoms or laboratory confirmed infections from the same community (internal contro

Sensitivity and specificity of single IgA and IgG antibody concentrations for early diagnosis of pertussis in adults: an evaluation for outbreak management in public health practice

Background: An accurate, practical laboratory test is needed to confirm clinical diagnosis of pertussis in adults during the first 3 symptomatic weeks, when treatment is effective and transmission can be interrupted. Methods: The sensitivity and specificity of single IgA and IgG levels were assessed in a cohort study of a pertussis epidemic in 99 adults in a closed community. Sensitivities were assessed in the sera of 46 laboratory confirmed clinical pertussis cases during the first 3 weeks. Specificities were calculated in sera of 35 asymptomatic controls without clinical symptoms or laboratory confirmed infections from the same community (internal contro

Predicting the chances of a live birth after one or more complete cycles of in vitro fertilisation : population based study of linked cycle data from 113 873 women

Funding: This work was supported by a Chief Scientist Office postdoctoral training fellowship in health services research and health of the public research (ref PDF/12/06). The views expressed here are those of the authors and not necessarily those of the Chief Scientist Office. The funder did not have any role in the study design; the collection, analysis, and interpretation of data; the writing of the report; nor the decision to submit the paper for publication. DJM had full access to all the data in the study and had final responsibility for the decision to submit for publication.Peer reviewedPublisher PD

An improvement in the method used to assess discriminatory ability when predicting the chances of a live birth after one or more complete cycles of in vitro fertilisation

Peer reviewedPostprin