Development and validation of the ACE tool: Assessing medical trainees' competency in evidence based medicine

BACKGROUND: While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude. METHODS: The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed. RESULTS: We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20). CONCLUSION: The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis

Evidence-based practice educational intervention studies: A systematic review of what is taught and how it is measured

Abstract Background Despite the established interest in evidence-based practice (EBP) as a core competence for clinicians, evidence for how best to teach and evaluate EBP remains weak. We sought to systematically assess coverage of the five EBP steps, review the outcome domains measured, and assess the properties of the instruments used in studies evaluating EBP educational interventions. Methods We conducted a systematic review of controlled studies (i.e. studies with a separate control group) which had investigated the effect of EBP educational interventions. We used citation analysis technique and tracked the forward and backward citations of the index articles (i.e. the systematic reviews and primary studies included in an overview of the effect of EBP teaching) using Web of Science until May 2017. We extracted information on intervention content (grouped into the five EBP steps), and the outcome domains assessed. We also searched the literature for published reliability and validity data of the EBP instruments used. Results Of 1831 records identified, 302 full-text articles were screened, and 85 included. Of these, 46 (54%) studies were randomised trials, 51 (60%) included postgraduate level participants, and 63 (75%) taught medical professionals. EBP Step 3 (critical appraisal) was the most frequently taught step (63 studies; 74%). Only 10 (12%) of the studies taught content which addressed all five EBP steps. Of the 85 studies, 52 (61%) evaluated EBP skills, 39 (46%) knowledge, 35 (41%) attitudes, 19 (22%) behaviours, 15 (18%) self-efficacy, and 7 (8%) measured reactions to EBP teaching delivery. Of the 24 instruments used in the included studies, 6 were high-quality (achieved ≥3 types of established validity evidence) and these were used in 14 (29%) of the 52 studies that measured EBP skills; 14 (41%) of the 39 studies that measured EBP knowledge; and 8 (26%) of the 35 studies that measured EBP attitude. Conclusions Most EBP educational interventions which have been evaluated in controlled studies focus on teaching only some of the EBP steps (predominantly critically appraisal of evidence) and did not use high-quality instruments to measure outcomes. Educational packages and instruments which address all EBP steps are needed to improve EBP teaching

Molecular Surveillance of True Nontypeable Haemophilus influenzae: An Evaluation of PCR Screening Assays

BackgroundUnambiguous identification of nontypeable Haemophilus influenzae (NTHi) is not possible by conventional microbiology. Molecular characterisation of phenotypically defined NTHi isolates suggests that up to 40% are Haemophilus haemolyticus (Hh); however, the genetic similarity of NTHi and Hh limits the power of simple molecular techniques such as PCR for species discrimination.Methodology/Principal FindingsHere we assess the ability of previously published and novel PCR-based assays to identify true NTHi. Sixty phenotypic NTHi isolates, classified by a dual 16S rRNA gene PCR algorithm as NTHi (n = 22), Hh (n = 27) or equivocal (n = 11), were further characterised by sequencing of the 16S rRNA and recA genes then interrogated by PCR-based assays targeting the omp P2, omp P6, lgtC, hpd, 16S rRNA, fucK and iga genes. The sequencing data and PCR results were used to define NTHi for this study. Two hpd real time PCR assays (hpd#1 and hpd#3) and the conventional iga PCR assay were equally efficient at differentiating study-defined NTHi from Hh, each with a receiver operator characteristic curve area of 0.90 [0.83; 0.98]. The hpd#1 and hpd#3 assays were completely specific against a panel of common respiratory bacteria, unlike the iga PCR, and the hpd#3 assay was able to detect below 10 copies per reaction.Conclusions/SignificanceOur data suggest an evolutionary continuum between NTHi and Hh and therefore no single gene target could completely differentiate NTHi from Hh. The hpd#3 real time PCR assay proved to be the superior method for discrimination of NTHi from closely related Haemophilus species with the added potential for quantification of H. influenzae directly from specimens. We suggest the hpd#3 assay would be suitable for routine NTHi surveillance and to assess the impact of antibiotics and vaccines, on H. influenzae carriage rates, carriage density, and disease

Impact of Capsular Switch on Invasive Pneumococcal Disease Incidence in a Vaccinated Population

BACKGROUND: Despite the dramatic decline in the incidence of invasive pneumococcal disease (IPD) observed since the introduction of conjugate vaccination, it is feared that several factors may undermine the future effectiveness of the vaccines. In particular, pathogenic pneumococci may switch their capsular types and evade vaccine-conferred immunity. METHODOLOGY/PRINCIPAL FINDINGS: Here, we first review the literature and summarize the available epidemiological data on capsular switch for S. pneumoniae. We estimate the weekly probability that a persistently carried strain may switch its capsule from four studies, totalling 516 children and 6 years of follow-up, at 1.5x10(-3)/week [4.6x10(-5)-4.8x10(-3)/week]. There is not enough power to assess an increase in this frequency in vaccinated individuals. Then, we use a mathematical model of pneumococcal transmission to quantify the impact of capsular switch on the incidence of IPD in a vaccinated population. In this model, we investigate a wide range of values for the frequency of vaccine-selected capsular switch. Predictions show that, with vaccine-independent switching only, IPD incidence in children should be down by 48% 5 years after the introduction of the vaccine with high coverage. Introducing vaccine-selected capsular switch at a frequency up to 0.01/week shows little effect on this decrease; yearly, at most 3 excess cases of IPD per 10(6) children might occur due to switched pneumococcal strains. CONCLUSIONS: Based on all available data and model predictions, the existence of capsular switch by itself should not impact significantly the efficacy of pneumococcal conjugate vaccination on IPD incidence. This optimistic result should be tempered by the fact that the selective pressure induced by the vaccine is currently increasing along with vaccine coverage worldwide; continued surveillance of pneumococcal populations remains of the utmost importance, in particular during clinical trials of the new conjugate vaccines

The Population Genetics of dN/dS

Evolutionary pressures on proteins are often quantified by the ratio of substitution rates at non-synonymous and synonymous sites. The dN/dS ratio was originally developed for application to distantly diverged sequences, the differences among which represent substitutions that have fixed along independent lineages. Nevertheless, the dN/dS measure is often applied to sequences sampled from a single population, the differences among which represent segregating polymorphisms. Here, we study the expected dN/dS ratio for samples drawn from a single population under selection, and we find that in this context, dN/dS is relatively insensitive to the selection coefficient. Moreover, the hallmark signature of positive selection over divergent lineages, dN/dS>1, is violated within a population. For population samples, the relationship between selection and dN/dS does not follow a monotonic function, and so it may be impossible to infer selection pressures from dN/dS. These results have significant implications for the interpretation of dN/dS measurements among population-genetic samples

The effects of selection for early (day) and late (dusk) mating lines of hybrids of Bactrocera tryoni and Bactrocera neohumeralis

Bactrocera neohumeralis and Bactrocera tryoni are closely related tephritid fruit fly species. B. neohumeralis mates throughout the day ( in bright light) and B. tryoni mates at dusk. The two species can also be distinguished by the colour of their calli ( prothoracic sclerites) which are brown and yellow, respectively. The F-1 hybrids can mate both in bright light just before dusk and during dusk and have calli that are partly brown and partly yellow. The F-2 hybrids have a wider range of callus patterns and mating occurs more widely in the day as well as at dusk. We directly selected hybrid stocks for mating time, creating 'early' (day-mating) and 'late' (dusk-mating) lines. As an apparently inadvertent consequence, the two types of line respectively had predominantly brown and predominantly yellow calli and thus came to closely resemble the original two species in both behaviour and appearance. Lines that were evenly selected (half for day and half for dusk) essentially retained the mating pattern of F-2 hybrids. Selection for callus colour alone also affected the distribution of mating times in a predictable way. We propose a genetical model to account for the results and discuss them in the light of the apparent maintenance of species integrity in nature