Explaining computation of predictive values: 2 × 2 table versus frequency tree. A randomized controlled trial [ISRCTN74278823]

BACKGROUND: Involving patients in decision making on diagnostic procedures requires a basic level of statistical thinking. However, innumeracy is prevalent even among physicians. In medical teaching the 2 × 2 table is widely used as a visual help for computations whereas in psychology the frequency tree is favoured. We assumed that the 2 × 2 table is more suitable to support computations of predictive values. METHODS: 184 students without prior statistical training were randomised either to a step-by-step self-learning tutorial using the 2 × 2 table (n = 94) or the frequency tree (n = 90). During the training session students were instructed by two sample tasks and a total of five positive predictive values had to be computed. During a follow-up session 4 weeks later participants had to compute 5 different tasks of comparable degree of difficulty without having the tutorial instructions at their disposal. The primary outcome was the correct solution of the tasks. RESULTS: There were no statistically significant differences between the two groups. About 58% achieved correct solutions in 4–5 tasks following the training session and 26% in the follow-up examination. CONCLUSIONS: These findings do not support the hypothesis that the 2 × 2 table is more valuable to facilitate the calculation of positive predictive values than the frequency tree

Appraising and applying evidence about a diagnostic test during a performance-based assessment

BACKGROUND: The practice of Evidence-based Medicine requires that clinicians assess the validity of published research and then apply the results to patient care. We wanted to assess whether our soon-to-graduate medical students could appraise and apply research about a diagnostic test within a clinical context and to compare our students with peers trained at other institutions. METHODS: 4(th )year medical students who previously had demonstrated competency at probability revision and just starting first-year Internal Medicine residents were used for this research. Following an encounter with a simulated patient, subjects critically appraised a paper about an applicable diagnostic test and revised the patient's pretest probability given the test result. RESULTS: The medical students and residents demonstrated similar skills at critical appraisal, correctly answering 4.7 and 4.9, respectively, of 6 questions (p = 0.67). Only one out of 28 (3%) medical students and none of the 15 residents were able to correctly complete the probability revision task (p = 1.00). CONCLUSIONS: This study found that most students completing medical school are able to appraise an article about a diagnostic test but few are able to apply the information from the article to a patient. These findings raise questions about the clinical usefulness of the EBM skills possessed by graduating medical students within the area of diagnostic testing