Validity and responsiveness of the Clubfoot Assessment Protocol (CAP). A methodological study

BACKGROUND: The Clubfoot Assessment Protocol (CAP) is a multi dimensional instrument designed for longitudinal follow up of the clubfoot deformity during growth. Item reliability has shown to be sufficient. In this article the CAP's validity and responsiveness is studied using the Dimeglio classification scoring as a gold standard. METHODS: Thirty-two children with 45 congenital clubfeet were assessed prospectively and consecutively at ages of new-born, one, two, four months and two years of age. For convergent/divergent construct validity the Spearman's correlation coefficients were calculated. Discriminate validity was evaluated by studying the scores in bilateral clubfeet. The floor-ceiling effects at baseline (untreated clubfeet) and at two years of age (treated clubfeet) were evaluated. Responsiveness was evaluated by using effect sizes (ES) and by calculating if significant changes (Wilcoxons signed test) had occurred between the different measurement occasions. RESULTS: High to moderate significant correlation were found between CAP mobility I and morphology and the Dimeglio scores (r(s )= 0.77 and 0.44 respectively). Low correlation was found between CAP muscle function, mobility II and motion quality and the Dimeglio scoring system (r(s )= 0.20, 0.09 and 0.06 respectively). Of 13 children with bilateral clubfeet, 11 showed different CAP mobility I scores between right and left foot at baseline (untreated) compared with 5 with the Dimeglio score. At the other assessment occasions the CAP mobility I continued to show higher discrimination ability than the Dimeglio. No floor effects and low ceiling effects were found in the untreated clubfeet for both instruments. High ceiling effects were found in the CAP for the treated children and low for the Dimeglio. Responsiveness was good. ES from untreated to treated ranged from 0.80 to 4.35 for the CAP subgroups and was 4.68 for the Dimeglio. The first four treatment months, the CAP mobility I had generally higher ES compared with the Dimeglio. CONCLUSION: The Clubfoot Assessment Protocol shows in this study good validity and responsiveness. The CAP is more responsive when severity ranges between mild – moderate to severe, while the Dimeglio focuses more on the extremes. The ability to discriminate between different mobility status of the right and left foot in bilaterally affected children in this population was higher compared with the Dimeglio score implicating a better sensitivity for the CAP

Progress testing in postgraduate medical education

Item does not contain fulltextBACKGROUND: The role of knowledge in postgraduate medical education has often been discussed. However, recent insights from cognitive psychology and the study of deliberate practice recognize that expert problem solving requires a well-organized knowledge database. This implies that postgraduate assessment should include knowledge testing. Longitudinal assessment, like progress testing, seems a promising approach for postgraduate progress knowledge assessment. AIMS: To evaluate the validity and reliability of a national progress test in postgraduate Obstetrics and Gynaecology training. METHODS: Data of 10 years of postgraduate progress testing were analyzed on reliability with Cronbach's alpha and on construct validity using one-way ANOVA with a post hoc Scheffe test. RESULTS: Average reliability with true-false questions was 0.50, which is moderate at best. After the introduction of multiple-choice questions average reliability improved to 0.65. Construct validity or discriminative power could only be demonstrated with some certainty between training year 1 and training year 2 and higher training years. CONCLUSION: Validity and reliability of the current progress test in postgraduate Obstetrics and Gynaecology training is unsatisfactory. Suggestions for improvement of both test construct and test content are provided