Online clinical reasoning assessment with the Script Concordance test: a feasibility study

BACKGROUND: The script concordance (SC) test is an assessment tool that measures capacity to solve ill-defined problems, that is, reasoning in context of uncertainty. This tool has been used up to now mainly in medicine. The purpose of this pilot study is to assess the feasibility of the test delivered on the Web to French urologists. METHODS: The principle of SC test construction and the development of the Web site are described. A secure Web site was created with two sequential modules: (a) The first one for the reference panel (n = 26) with two sub-tasks: to validate the content of the test and to elaborate the scoring system; (b) The second for candidates with different levels of experience in Urology: Board certified urologists, residents, medical students (5 or 6(th )year). Minimum expected number of participants is 150 for urologists, 100 for residents and 50 for medical students. Each candidate is provided with an individual access code to this Web site. He/she may complete the Script Concordance test several times during his/her curriculum. RESULTS: The Web site has been operational since April 2004. The reference panel validated the test in June of the same year during the annual seminar of the French Society of Urology. The Web site is available for the candidates since September 2004. In six months, 80% of the target figure for the urologists, 68% of the target figure for the residents and 20% of the target figure for the student passed the test online. During these six months, no technical problem was encountered. CONCLUSION: The feasibility of the web-based SC test is successful as two-thirds of the expected number of participants was included within six months. Psychometric properties (validity, reliability) of the test will be evaluated on a large scale (N = 300). If positive, educational impact of this assessment tool will be useful to help urologists during their curriculum for the acquisition of clinical reasoning skills, which is crucial for professional competence

On line clinical reasoning assessment with Script Concordance test in urology: results of a French pilot study

BACKGROUND: The Script Concordance test (SC) test is an assessment tool that measures the capacity to solve ill-defined problems, that is, reasoning in a context of uncertainty. This study assesses the feasibility, reliability and validity of the SC test made available on the Web to French urologists. METHODS: A 97 items SC test was developed based on major educational objectives of French urology training programmes. A secure Web site was created with two sequential modules: a) The first one for the reference panel to elaborate the scoring system; b) The second for candidates with different levels of experience in urology: Board certified urologists, chief-residents, residents, medical students. All participants were recruited on a voluntary basis. Statistical analysis included descriptive statistics of the participants' scores and factorial analysis of variance (ANOVA) to study differences between groups' means. Reliability was evaluated with Cronbach's alpha coefficient. RESULTS: The on line SC test has been operational since June 2004. Twenty-six faculty members constituted the reference panel. During the following 10 months, 207 participants took the test online (124 urologists, 29 chief-residents, 38 residents, 16 students). No technical problem was encountered. Forty-five percent of the participants completed the test partially only. Differences between the means scores for the 4 groups were statistically significant (P = 0.0123). The Bonferroni post-hoc correction indicated that significant differences were present between students and chief-residents, between students and urologists. There were no differences between chief-residents and urologists. Reliability coefficient was 0.734 for the total group of participants. CONCLUSION: Feasibility of Web-based SC test was proved successful by the large number of participants who participated in a few months. This Web site has permitted to quickly confirm reliability of the SC test and develop strategy to improve construct validity of the test when applied in the field of urology. Nevertheless, optimisation of the SC test content, with a smaller number of items will be necessary. Virtual medical education initiative such as this SC test delivered on the Internet warrants consideration in the current context of national pre-residency certification examination in France

Summative assessment of 5th year medical students' clinical reasoning by script concordance test: requirements and challenges

Background: The Script Concordance Test (SCT) has not been reported in summative assessment of students across the multiple domains of a medical curriculum. We report the steps used to build a test for summative assessment in a medical curriculum. Methods: A 51 case, 158-question, multidisciplinary paper was constructed to assess clinical reasoning in 5th-year. 10–16 experts in each of 7 discipline-based reference panels answered questions on-line. A multidisciplinary group considered reference panel data and data from a volunteer group of 6th Years, who sat the same test, to determine the passing score for the 5th Years. Results: The mean (SD) scores were 63.6 (7.6) and 68.6 (4.8) for the 6th Year (n = 23, alpha = 0.78) and and 5th Year (n = 132, alpha =0.62) groups (p < 0.05), respectively. The passing score was set at 4 SD from the expert mean. Four students failed. Conclusions: The SCT may be a useful method to assess clinical reasoning in medical students in multidisciplinary summative assessments. Substantial investment in training of faculty and students and in the development of questions is required.Paul Duggan and Bernard Charli

Professional approaches in clinical judgements among senior and junior doctors: implications for medical education

<p>Abstract</p> <p>Background</p> <p>Clinical experience has traditionally been highly valued in medical education and clinical healthcare. On account of its multi-faceted nature, clinical experience is mostly difficult to articulate, and is mainly expressed in clinical situations as professional approaches. Due to retirement, hospitals in Scandinavia will soon face a substantial decrease in the number of senior specialist doctors, and it has been discussed whether healthcare will suffer an immense loss of experienced-based knowledge when this senior group leaves the organization. Both senior specialists and junior colleagues are often involved in clinical education, but the way in which these two groups vary in professional approaches and contributions to clinical education has not been so well described. Cognitive psychology has contributed to the understanding of how experience may influence professional approaches, but such studies have not included the effect of differences in position and responsibilities that junior and senior doctors hold in clinical healthcare. In the light of the discussion above, it is essential to describe the professional approaches of senior doctors in relation to those of their junior colleagues. This study therefore aims to describe and compare the professional approaches of junior and senior doctors when making clinical judgements.</p> <p>Methods</p> <p>Critical incident technique was used in interviews with nine senior doctors and nine junior doctors in internal medicine. The interviews were subjected to qualitative content analysis.</p> <p>Result</p> <p>Senior and junior doctors expressed a variety of professional approaches in clinical judgement as follows: use of theoretical knowledge, use of prior experience of cases and courses of events, use of ethical and moral values, meeting and communicating with the patient, focusing on available information, relying on their own ability, getting support and guidance from others and being directed by the organization.</p> <p>Conclusion</p> <p>The most prominent varieties of professional approaches were seen in use of knowledge and work-related experience. Senior doctors know how the organization has worked in the past and have acquired techniques with respect to long-term decisions and their consequences. Junior doctors, on the other hand, have developed techniques and expertise for making decisions based on a restricted amount of information, in relation to patients' wellbeing as well as organizational opportunities and constraints. This study contributes to medical education by elucidating the variation in professional approaches among junior and senior doctors, which can be used as a basis for discussion about clinical judgement, in both pre-clinical and clinical education. Further research is required to explain how these professional approaches are expressed and used in clinical education.</p

Molecular screening of 980 cases of suspected hereditary optic neuropathy with a report on 77 novel OPA1 mutations

We report the results of molecular screening in 980 patients carried out as part of their work-up for suspected hereditary optic neuropathies. All the patients were investigated for Leber\u27s hereditary optic neuropathy (LHON) and autosomal dominant optic atrophy (ADOA), by searching for the ten primary LHON-causing mtDNA mutations and examining the entire coding sequences of the OPA1 and OPA3 genes, the two genes currently identified in ADOA. Molecular defects were identified in 440 patients (45% of screened patients). Among these, 295 patients (67%) had an OPA1 mutation, 131 patients (30%) had an mtDNA mutation, and 14 patients (3%), belonging to three unrelated families, had an OPA3 mutation. Interestingly, OPA1 mutations were found in 157 (40%) of the 392 apparently sporadic cases of optic atrophy. The eOPA1 locus-specific database now contains a total of 204 OPA1 mutations, including 77 novel OPA1 mutations reported here. The statistical analysis of this large set of mutations has led us to propose a diagnostic strategy that should help with the molecular work-up of optic neuropathies. Our results highlight the importance of investigating LHON-causing mtDNA mutations as well as OPA1 and OPA3 mutations in cases of suspected hereditary optic neuropathy, even in absence of a family history of the disease. © 2009 Wiley-Liss, Inc

What Was I Thinking? Eye-Tracking Experiments Underscore the Bias that Architecture Exerts on Nuclear Grading in Prostate Cancer

We previously reported that nuclear grade assignment of prostate carcinomas is subject to a cognitive bias induced by the tumor architecture. Here, we asked whether this bias is mediated by the non-conscious selection of nuclei that “match the expectation” induced by the inadvertent glance at the tumor architecture. 20 pathologists were asked to grade nuclei in high power fields of 20 prostate carcinomas displayed on a computer screen. Unknown to the pathologists, each carcinoma was shown twice, once before a background of a low grade, tubule-rich carcinoma and once before the background of a high grade, solid carcinoma. Eye tracking allowed to identify which nuclei the pathologists fixated during the 8 second projection period. For all 20 pathologists, nuclear grade assignment was significantly biased by tumor architecture. Pathologists tended to fixate on bigger, darker, and more irregular nuclei when those were projected before kigh grade, solid carcinomas than before low grade, tubule-rich carcinomas (and vice versa). However, the morphometric differences of the selected nuclei accounted for only 11% of the architecture-induced bias, suggesting that it can only to a small part be explained by the unconscious fixation on nuclei that “match the expectation”. In conclusion, selection of « matching nuclei » represents an unconscious effort to vindicate the gravitation of nuclear grades towards the tumor architecture