Machine learning clinical decision support systems for surveillance: a case study on pertussis and RSV in children

We tested the performance of a machine learning (ML) algorithm based on signs and symptoms for the diagnosis of RSV infection or pertussis in the first year of age to support clinical decisions and provide timely data for public health surveillance. We used data from a retrospective case series of children in the first year of life investigated for acute respiratory infections in the emergency room from 2015 to 2020. We collected data from PCR laboratory tests for confirming pertussis or RSV infection, clinical symptoms, and routine blood testing results, which were used for the algorithm development. We used a LightGBM model to develop 2 sets of models for predicting pertussis and RSV infection: for each type of infection, we developed one model trained with the combination of clinical symptoms and results from routine blood test (white blood cell count, lymphocyte fraction and C-reactive protein), and one with symptoms only. All analyses were performed using Python 3.7.4 with Shapley values (Shap values) visualization package for predictor visualization. The performance of the models was assessed through confusion matrices. The models were developed on a dataset of 599 children. The recall for the pertussis model combining symptoms and routine laboratory tests was 0.72, and 0.74 with clinical symptoms only. For RSV infection, recall was 0.68 with clinical symptoms and laboratory tests and 0.71 with clinical symptoms only. The F1 score for the pertussis model was 0.72 in both models, and, for RSV infection, it was 0.69 and 0.75. ML models can support the diagnosis and surveillance of infectious diseases such as pertussis or RSV infection in children based on common symptoms and laboratory tests. ML-based clinical decision support systems may be developed in the future in large networks to create accurate tools for clinical support and public health surveillance

Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis

BACKGROUND We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs. METHODS AND FINDINGS We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations. CONCLUSIONS We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research

Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis.

BackgroundWe previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs.Methods and findingsWe included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations.ConclusionsWe have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research