Methods to account for measured and unmeasured confounders in influenza relative vaccine effectiveness studies:A brief review of the literature

Observational seasonal influenza relative vaccine effectiveness (rVE) studies employ a variety of statistical methods to account for confounding and biases. To better understand the range of methods employed and implications for policy, we conducted a brief literature review. Across 37 included rVE studies, 10 different types of statistical methods were identified, and only eight studies reported methods to detect residual confounding, highlighting the heterogeneous state of the literature. To improve the comparability and credibility of future rVE research, researchers should clearly explain methods and design choices and implement methods to detect and quantify residual confounding

Developing and validating a comprehensive implementation framework for reporting reproducible infectious disease computational modeling studies

In the wake of the coronavirus disease 2019 (COVID-19) pandemic, policymakers have relied heavily on computational models to inform decisions concerning public health interventions. Unfortunately, reproducibility of computational modeling studies is limited due to methodological complexity and lack of transparent reporting practices. We filled this critical gap in the literature by developing an implementation framework for representing infectious disease computational models in a reproducible way, grounded in previous research on reproducibility from a broad range of scientific disciplines. The implementation framework provides a foundation that can be further developed into tools, such as checklists or machine-interpretable metadata, for sharing computational models in a reproducible manner. We formatted the implementation framework into the Infectious Disease Modeling Reproducibility Checklist (IDMRC) and validated the checklist through an iterative process by evaluating random samples of infectious disease modeling studies. In addition to our framework and the IDMRC, we evaluated several workflow tools, for representing, evaluating, and reproducing models which may lead to useful insights for improving the coordination of modeling resources. We tested the feasibility of reproducing a COVID-19 model using the Open Curation for Computer Architecture Modeling (Occam), an open-source workflow platform that encapsulates and preserves the complete experimental workflow of a modeling study. For years, attempts have been made to develop a comprehensive tool that can be adopted by researchers, journal editors, and scientific organizations with minimal success in preventing irreproducible models from being published. Our implementation framework and the IDMRC are the first reproducibility tools that can be used by researchers to assess infectious disease computational modeling studies starting from the description of the model and ending with the obtainment of the results. By easily comparing models and their output, researchers will be able to efficiently identify the best models to inform life-saving interventions

An implementation framework to improve the transparency and reproducibility of computational models of infectious diseases.

Computational models of infectious diseases have become valuable tools for research and the public health response against epidemic threats. The reproducibility of computational models has been limited, undermining the scientific process and possibly trust in modeling results and related response strategies, such as vaccination. We translated published reproducibility guidelines from a wide range of scientific disciplines into an implementation framework for improving reproducibility of infectious disease computational models. The framework comprises 22 elements that should be described, grouped into 6 categories: computational environment, analytical software, model description, model implementation, data, and experimental protocol. The framework can be used by scientific communities to develop actionable tools for sharing computational models in a reproducible way

Inter-rater reliability of the infectious disease modeling reproducibility checklist (IDMRC) as applied to COVID-19 computational modeling research

Abstract Background Infectious disease computational modeling studies have been widely published during the coronavirus disease 2019 (COVID-19) pandemic, yet they have limited reproducibility. Developed through an iterative testing process with multiple reviewers, the Infectious Disease Modeling Reproducibility Checklist (IDMRC) enumerates the minimal elements necessary to support reproducible infectious disease computational modeling publications. The primary objective of this study was to assess the reliability of the IDMRC and to identify which reproducibility elements were unreported in a sample of COVID-19 computational modeling publications. Methods Four reviewers used the IDMRC to assess 46 preprint and peer reviewed COVID-19 modeling studies published between March 13th, 2020, and July 30th, 2020. The inter-rater reliability was evaluated by mean percent agreement and Fleiss’ kappa coefficients (κ). Papers were ranked based on the average number of reported reproducibility elements, and average proportion of papers that reported each checklist item were tabulated. Results Questions related to the computational environment (mean κ = 0.90, range = 0.90–0.90), analytical software (mean κ = 0.74, range = 0.68–0.82), model description (mean κ = 0.71, range = 0.58–0.84), model implementation (mean κ = 0.68, range = 0.39–0.86), and experimental protocol (mean κ = 0.63, range = 0.58–0.69) had moderate or greater (κ > 0.41) inter-rater reliability. Questions related to data had the lowest values (mean κ = 0.37, range = 0.23–0.59). Reviewers ranked similar papers in the upper and lower quartiles based on the proportion of reproducibility elements each paper reported. While over 70% of the publications provided data used in their models, less than 30% provided the model implementation. Conclusions: The IDMRC is the first comprehensive, quality-assessed tool for guiding researchers in reporting reproducible infectious disease computational modeling studies. The inter-rater reliability assessment found that most scores were characterized by moderate or greater agreement. These results suggest that the IDMRC might be used to provide reliable assessments of the potential for reproducibility of published infectious disease modeling publications. Results of this evaluation identified opportunities for improvement to the model implementation and data questions that can further improve the reliability of the checklist

Associations between sleep health and obesity and weight change in adults: The Daily24 Multisite Cohort Study.

OBJECTIVES: To examine cross-sectional and longitudinal associations of individual sleep domains and multidimensional sleep health with current overweight or obesity and 5-year weight change in adults. METHODS: We estimated sleep regularity, quality, timing, onset latency, sleep interruptions, duration, and napping using validated questionnaires. We calculated multidimensional sleep health using a composite score (total number of good sleep health indicators) and sleep phenotypes derived from latent class analysis. Logistic regression was used to examine associations between sleep and overweight or obesity. Multinomial regression was used to examine associations between sleep and weight change (gain, loss, or maintenance) over a median of 1.66 years. RESULTS: The sample included 1016 participants with a median age of 52 (IQR = 37-65), who primarily identified as female (78%), White (79%), and college-educated (74%). We identified 3 phenotypes: good, moderate, and poor sleep. More regularity of sleep, sleep quality, and shorter sleep onset latency were associated with 37%, 38%, and 45% lower odds of overweight or obesity, respectively. The addition of each good sleep health dimension was associated with 16% lower adjusted odds of having overweight or obesity. The adjusted odds of overweight or obesity were similar between sleep phenotypes. Sleep, individual or multidimensional sleep health, was not associated with weight change. CONCLUSIONS: Multidimensional sleep health showed cross-sectional, but not longitudinal, associations with overweight or obesity. Future research should advance our understanding of how to assess multidimensional sleep health to understand the relationship between all aspects of sleep health and weight over time