Utility of intermittent online quizzes as an early warning for residents at risk of failing the pediatric board certification examination

Abstract Background Traditionally, quizzes have been applied as a tool for summative assessment, though literature suggests their use as a formative assessment can improve motivation and content retention. With this premise, we implemented a series of intermittent, online quizzes known as the Board Examination Simulation Exercise (BESE). We sought to demonstrate an association between BESE participation and scores and performance on the American Board of Pediatrics (ABP) Certifying Examination (CE). Methods Residents were assigned online quizzes on a single topic at 2 week intervals that consisted of 20 multiple choice questions written by the study authors. This analysis includes graduates of 3 Pediatric and Internal Medicine-Pediatrics residency programs. Results Data were available for 329 residents. The overall BESE score weakly correlated with ABP CE score (n = 287; r = 0.39, p < 0.0001). ABP CE pass rates increased from 2009 to 2016 at all programs combined (p = 0.0001). A composite BESE score ≤ 11 had sensitivity of 54% and specificity of 80% for predicting ABP CE failure on the first attempt. There was no difference in ABP CE failure rates or scores by number of completed quizzes. Conclusion Intermittent online quizzes implemented at three pediatric residency programs were associated with overall increasing ABP CE pass rates. BESE increased program emphasis on board preparation. Residents with lower BESE scores more often failed ABP CE. Though additional data are needed, BESE is a promising tool for pediatric resident learning and board preparation. It may also aid in earlier identification of residents at higher risk of failing the ABP CE and facilitate targeted interventions

Molecular Distance to Health Transcriptional Score and Disease Severity in Children Hospitalized With Community-Acquired Pneumonia

Background: Community-acquired pneumonia (CAP) is a leading cause of hospitalization and mortality in children. Diagnosis remains challenging and there are no reliable tools to objectively risk stratify patients or predict clinical outcomes. Molecular distance to health (MDTH) is a genomic score that measures the global perturbation of the transcriptional profile and may help classify patients by disease severity. We evaluated the value of MDTH to assess disease severity in children hospitalized with CAP.Methods: Children hospitalized with CAP and matched healthy controls were enrolled in a prospective observational study. Blood samples were obtained for transcriptome analyses within 24 h of hospitalization. MDTH scores were calculated to assess disease severity and correlated with laboratory markers, such as white blood cell count, c-reactive protein (CRP), and procalcitonin (PCT), and clinical outcomes, including duration of fever and duration of hospitalization (LOS). Univariate and multivariable logistic regression were applied to assess factors associated with LOS and duration of fever after hospitalization.Results: Among children hospitalized with CAP (n = 152), pyogenic bacteria (PB) were detected in 16 (11%), Mycoplasma pneumoniae was detected in 41 (28%), respiratory viruses (RV) alone were detected in 78 (51%), and no pathogen was detected in 17 (11%) children. Statistical group comparisons identified 6,726 genes differentially expressed in patients with CAP vs. healthy controls (n = 39). Children with confirmed PB had higher MDTH scores than those with RV (p &lt; 0.05) or M. pneumoniae (p &lt; 0.01) detected alone. CRP (r = 0.39, p &lt; 0.0001), PCT (r = 0.39, p &lt; 0.0001), and MDTHs (r = 0.24, p &lt; 0.01) correlated with duration of fever, while only MDTHs correlated with LOS (r = 0.33, p &lt; 0.0001). Unadjusted analyses showed that both higher CRP and MDTHs were associated with longer LOS (OR 1.04 [1–1.07] and 1.12 [1.04–1.20], respectively), however, only MDTH remained significant when adjusting for other covariates (aOR 1.11 [1.01–1.22]).Conclusions: In children hospitalized with CAP MDTH score measured within 24 h of admission was independently associated with longer duration of hospitalization, regardless of the pathogen detected. This suggests that transcriptional biomarkers may represent a promising approach to assess disease severity in children with CAP

Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice

Abstract Background As a community of practice (CoP), medical education depends on its research literature to communicate new knowledge, examine alternative perspectives, and share methodological innovations. As a key route of communication, the medical education CoP must be concerned about the rigor and validity of its research literature, but prior studies have suggested the need to improve medical education research quality. Of concern in the present study is the question of how responsive the medical education research literature is to changes in the CoP. We examine the nature and extent of changes in the quality of medical education research over a decade, using a widely cited study of research quality in the medical education research literature as a benchmark to compare more recent quality indicators. Methods A bibliometric analysis was conducted to examine the methodologic quality of quantitative medical education research studies published in 13 selected journals from September 2013 to December 2014. Quality scores were calculated for 482 medical education studies using a 10-item Medical Education Research Study Quality Instrument (MERSQI) that has demonstrated strong validity evidence. These data were compared with data from the original study for the same journals in the period September 2002 to December 2003. Eleven investigators representing 6 academic medical centers reviewed and scored the research studies that met inclusion and exclusion criteria. Primary outcome measures include MERSQI quality indicators for 6 domains: study design, sampling, type of data, validity, data analysis, and outcomes. Results There were statistically significant improvements in four sub-domain measures: study design, type of data, validity and outcomes. There were no changes in sampling quality or the appropriateness of data analysis methods. There was a small but significant increase in the use of patient outcomes in these studies. Conclusions Overall, we judge this as equivocal evidence for the responsiveness of the research literature to changes in the medical education CoP. This study identified areas of strength as well as opportunities for continued development of medical education research.http://deepblue.lib.umich.edu/bitstream/2027.42/173627/1/12909_2022_Article_3301.pd

Assessing changes in the quality of quantitative health educations research: a perspective from communities of practice.

BackgroundAs a community of practice (CoP), medical education depends on its research literature to communicate new knowledge, examine alternative perspectives, and share methodological innovations. As a key route of communication, the medical education CoP must be concerned about the rigor and validity of its research literature, but prior studies have suggested the need to improve medical education research quality. Of concern in the present study is the question of how responsive the medical education research literature is to changes in the CoP. We examine the nature and extent of changes in the quality of medical education research over a decade, using a widely cited study of research quality in the medical education research literature as a benchmark to compare more recent quality indicators.MethodsA bibliometric analysis was conducted to examine the methodologic quality of quantitative medical education research studies published in 13 selected journals from September 2013 to December 2014. Quality scores were calculated for 482 medical education studies using a 10-item Medical Education Research Study Quality Instrument (MERSQI) that has demonstrated strong validity evidence. These data were compared with data from the original study for the same journals in the period September 2002 to December 2003. Eleven investigators representing 6 academic medical centers reviewed and scored the research studies that met inclusion and exclusion criteria. Primary outcome measures include MERSQI quality indicators for 6 domains: study design, sampling, type of data, validity, data analysis, and outcomes.ResultsThere were statistically significant improvements in four sub-domain measures: study design, type of data, validity and outcomes. There were no changes in sampling quality or the appropriateness of data analysis methods. There was a small but significant increase in the use of patient outcomes in these studies.ConclusionsOverall, we judge this as equivocal evidence for the responsiveness of the research literature to changes in the medical education CoP. This study identified areas of strength as well as opportunities for continued development of medical education research