Direct observation of procedural skills (DOPS) assessment in diagnostic gastroscopy: nationwide evidence of validity and competency development during training.

Background: Validated competency assessment tools and the data supporting milestone development during gastroscopy training are lacking. We aimed to assess the validity of the formative direct observation of procedural skills (DOPS) assessment tool in diagnostic gastroscopy and study competency development using DOPS. Methods: This was a prospective multicentre (N = 275) analysis of formative gastroscopy DOPS assessments. Internal structure validity was tested using exploratory factor analysis and reliability estimated using generalisability theory. Item and global DOPS scores were stratified by lifetime procedure count to define learning curves, using a threshold determined from receiver operator characteristics (ROC) analysis. Multivariable binary logistic regression analysis was performed to identify independent predictors of DOPS competence. Results: In total, 10086 DOPS were submitted for 987 trainees. Exploratory factor analysis identified three distinct item groupings, representing 'pre-procedure', 'technical', and 'post-procedure non-technical' skills. From generalisability analyses, sources of variance in overall DOPS scores included trainee ability (31%), assessor stringency (8%), assessor subjectivity (18%), and trainee case-to-case variation (43%). The combination of three assessments from three assessors was sufficient to achieve the reliability threshold of 0.70. On ROC analysis, a mean score of 3.9 provided optimal sensitivity and specificity for determining competency. This threshold was attained in the order of 'pre-procedure' (100-124 procedures), 'technical' (150-174 procedures), 'post-procedure non-technical' skills (200-224 procedures), and global competency (225-249 procedures). Higher lifetime procedure count, DOPS count, surgical trainees and assessors, higher trainee seniority, and lower case difficulty were significant multivariable predictors of DOPS competence. Conclusion: This study establishes milestones for competency acquisition during gastroscopy training and provides validity and reliability evidence to support gastroscopy DOPS as a competency assessment tool

Design of a multicentre randomized trial to evaluate CT colonography versus colonoscopy or barium enema for diagnosis of colonic cancer in older symptomatic patients: The SIGGAR study

Background and Aims: The standard whole-colon tests used to investigate patients with symptoms of colorectal cancer are barium enema and colonoscopy. Colonoscopy is the reference test but is technically difficult, resource intensive, and associated with adverse events, especially in the elderly. Barium enema is safer but has reduced sensitivity for cancer. CT colonography ("virtual colonoscopy") is a newer alternative that may combine high sensitivity for cancer with safety and patient acceptability. The SIGGAR trial aims to determine the diagnostic efficacy, acceptability, and economic costs associated with this new technology.Methods: The SIGGAR trial is a multi-centre randomised comparison of CT colonography versus standard investigation ( barium enema or colonoscopy), the latter determined by individual clinician preference. Diagnostic efficacy for colorectal cancer and colonic polyps measuring 1 cm or larger will be determined, as will the physical and psychological morbidity associated with each diagnostic test, the latter via questionnaires developed from qualitative interviews. The economic costs of making or excluding a diagnosis will be determined for each diagnostic test and information from the trial and other data from the literature will be used to populate models framed to summarise the health effects and costs of alternative approaches to detection of significant colonic neoplasia in patients of different ages, prior risks and preferences. This analysis will focus particularly on the frequency, clinical relevance, costs, and psychological and physical morbidity associated with detection of extracolonic lesions by CT colonography.Results: Recruitment commenced in March 2004 and at the time of writing ( July 2007) 5025 patients have been randomised. A lower than expected prevalence of end-points in the barium enema sub-trial has caused an increase in sample size. In addition to the study protocol, we describe our approach to recruitment, notably the benefits of extensive piloting, the use of a sham-randomisation procedure, which was employed to determine whether centres interested in participating were likely to be effective in practice, and the provision of funding for dedicated sessions for a research nurse at each centre to devote specifically to this trial