Effect of response format for clinical vignettes on reporting quality of physician practice

<p>Abstract</p> <p>Background</p> <p>Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats – open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) – in rheumatologists' pre-treatment assessment for tumor-necrosis-factor (TNF) blocker therapy.</p> <p>Methods</p> <p><b><it>Study design</it></b>: Prospective randomized study. <b><it>Setting</it></b>: Rheumatologists attending the 2004 French Society of Rheumatology meeting. Physicians were given a vignette describing the history of a fictitious woman with active rheumatoid arthritis, who was a candidate for therapy with TNF blocking agents, and then were randomized to receive questionnaire A, B, or C, each containing the same four questions but with different response formats, that asked about their pretreatment assessment. <b><it>Measurements</it></b>: Long (recommended items) and short (mandatory items) checklists were developed for pretreatment assessment for TNF-blocker therapy, and scores were expressed on the basis of responses to questionnaires A, B, and C as the percentage of respondents correctly choosing explicit items on these checklists. <b><it>Statistical analysis</it></b>: Comparison of the selected items using pairwise Chi-square tests with Bonferonni correction for variables with statistically significant differences.</p> <p>Results</p> <p>Data for all surveys distributed (114 As, 118 Bs, and 118 Cs) were complete and available for analysis. The percentage of questionnaire A, B, and C respondents for whom data was correctly complete for the short checklist was 50.4%, 84.0% and 95.0%, respectively, and was 0%, 5.0% and 5.9%, respectively, for the long version. As an example, 65.8%, 85.7% and 95.8% of the respondents of A, B, and C questionnaires, respectively, correctly identified the need for tuberculin skin test (p < 0.0001).</p> <p>Conclusion</p> <p>In evaluating clinical practice with use of a clinical vignette, a multiple-choice format rather than an open-ended format overestimates physician performance. The insertion of deceptive response items mixed with correct items in closed-ended (multiple-choice) questionnaire failed to avoid this overestimation.</p

Are there valid proxy measures of clinical behaviour?

Background: Accurate measures of health professionals' clinical practice are critically important to guide health policy decisions, as well as for professional self-evaluation and for research-based investigation of clinical practice and process of care. It is often not feasible or ethical to measure behaviour through direct observation, and rigorous behavioural measures are difficult and costly to use. The aim of this review was to identify the current evidence relating to the relationships between proxy measures and direct measures of clinical behaviour. In particular, the accuracy of medical record review, clinician self-reported and patient-reported behaviour was assessed relative to directly observed behaviour. Methods: We searched: PsycINFO; MEDLINE; EMBASE; CINAHL; Cochrane Central Register of Controlled Trials; science/social science citation index; Current contents (social & behavioural med/clinical med); ISI conference proceedings; and Index to Theses. Inclusion criteria: empirical, quantitative studies; and examining clinical behaviours. An independent, direct measure of behaviour (by standardised patient, other trained observer or by video/audio recording) was considered the 'gold standard' for comparison. Proxy measures of behaviour included: retrospective self-report; patient-report; or chart-review. All titles, abstracts, and full text articles retrieved by electronic searching were screened for inclusion and abstracted independently by two reviewers. Disagreements were resolved by discussion with a third reviewer where necessary. Results: Fifteen reports originating from 11 studies met the inclusion criteria. The method of direct measurement was by standardised patient in six reports, trained observer in three reports, and audio/video recording in six reports. Multiple proxy measures of behaviour were compared in five of 15 reports. Only four of 15 reports used appropriate statistical methods to compare measures. Some direct measures failed to meet our validity criteria. The accuracy of patient report and chart review as proxy measures varied considerably across a wide range of clinical actions. The evidence for clinician self-report was inconclusive. Conclusion: Valid measures of clinical behaviour are of fundamental importance to accurately identify gaps in care delivery, improve quality of care, and ultimately to improve patient care. However, the evidence base for three commonly used proxy measures of clinicians' behaviour is very limited. Further research is needed to better establish the methods of development, application, and analysis for a range of both direct and proxy measures of behaviour

Low agreement for assessing the risk of postoperative deep venous thrombosis when deciding prophylaxis strategies: a study using clinical vignettes

BACKGROUND: Several clinical practice guidelines (CPG) on antithrombotic prophylaxis in surgical patients help to decide about the prophylaxis strategy based on the patient risk of deep venous thrombosis (DVT). However, the physician risk estimates of DVT could have little inter-observer reproducibility, which could lead to different individual prophylaxis practices. METHODS: Physicians were asked to evaluate DVT risk in eight clinical vignettes, describing actual patients cared for in our hospital. The vignettes included all possible levels of DVT risk. RESULTS: The degree of prophylaxis strategies accuracy was 63% (95% CI 523–75%). Overall agreement was 0.32 (z = 7.61, p < 0.001) and for each level of risk kappa was 0.38 (z = 6.50, p < 0.001); 0.1 (z = 1.65, p < 0.049) and 0.5 (z = 8.45, p < 0.001) for small, moderate and high risk group respectively CONCLUSIONS: Our results showed that there is poor agreement when physicians have to evaluate the risk for postoperative DVT, and in the cases of low and moderate risks of DVT there is the smallest agreement. In addition, the data also showed that the overall accuracy of DVT prophylaxis strategy was only moderate and the risk evaluation did not correlate to the selection of the strategy. The issue of inter-observers variability should be taken into account when CPG performance are analysed, especially when considering the risk-evaluation to choose the appropriate actions

A survey of primary care physician practices in antibiotic prescribing for the treatment of uncomplicated male gonoccocal urethritis

<p>Abstract</p> <p>Background</p> <p>The development of resistance to antimicrobial therapy by <it>Neisseria gonorrhoeae </it>causes on-going problems for individual case management of gonorrhoea. Surveillance data about <it>N. gonorrhoeae </it>have indicated an increase in the incidence of gonorrhoea in France in 2006. As a consequence of the development of antibiotic resistance in <it>N. gonorrhoeae</it>, French guidelines excluded fluoroquinolones as a standard treatment for <it>N. gonorrhoeae</it>. Ceftriaxone became the recommended treatment, associated with azithromycin for <it>Clamydia trachomatis </it>infection. Our aim was to describe the practice patterns of general practitioners (GPs) in managing the antibiotic treatment of patients with symptoms suggestive of uncomplicated male urethritis.</p> <p>Methods</p> <p>We developed a clinical vignette describing a man with typical gonococcal urethritis symptoms to elicit questions about antibiotic treatment. We mailed the electronic questionnaire to a random sample of 1000 French GPs belonging to the <it>Sentinelles </it>Network.</p> <p>Results</p> <p>By the end of the survey period, 350 vignettes were received, yielding a response rate of 35%. Sixty-six GPs (20.2%) prescribed the recommended antibiotics for the simultaneous treatment of <it>N</it>. <it>gonorrhoeae </it>and <it>C. trachomatis </it>infections, while 132 GPs (40.4%) prescribed only non-recommended antibiotics, including ciprofloxacin in 69 cases (21.1%). General practitioners with less than 10 years in practice showed better compliance to guidelines than those with more years in practice (p < 0.05).</p> <p>Conclusions</p> <p>The results suggest a mismatch between the guidelines and the antibiotic treatment of male uncomplicated urethritis by French GPs, mostly among the subgroup of physicians who have been in practice longer. Educational approaches based on practice feedback need to be developed to improve these deficits in the quality of care.</p

Multifaceted contributions : health workers and smallpox eradication in India

Smallpox eradication in South Asia was a result of the efforts of many grades of health-workers. Working from within the confines of international organisations and government structures, the role of the field officials, who were of various nationalities and also drawn from the cities and rural enclaves of the countries in these regions, was crucial to the development and deployment of policies. However, the role of these personnel is often downplayed in official histories and academic histories, which highlight instead the roles played by a handful of senior officials within the World Health Organization and the federal governments in the sub-continent. This article attempts to provide a more rounded assessment of the complex situation in the field. In this regard, an effort is made to underline the great usefulness of the operational flexibility displayed by field officers, wherein lessons learnt in the field were made an integral part of deploying local campaigns; careful engagement with the communities being targeted, as well as the employment of short term workers from amongst them, was an important feature of this work

Accuracy of the discharge destination field in administrative data for identifying transfer to a long-term acute care hospital

<p>Abstract</p> <p>Background</p> <p>Long-term acute care hospitals (LTACs) provide specialized care for patients recovering from severe acute illness. In order to facilitate research into LTAC utilization and outcomes, we studied whether or not the discharge destination field in administrative data accurately identifies patients transferred to an LTAC following acute care hospitalization.</p> <p>Findings</p> <p>We used the 2006 hospitalization claims for United States Medicare beneficiaries to examine the performance characteristics of the discharge destination field in the administrative record, compared to the reference standard of directly observing LTAC transfers in the claims. We found that the discharge destination field was highly specific (99.7%, 95 percent CI: 99.7% - 99.8%) but modestly sensitive (77.3%, 95 percent CI: 77.0% - 77.6%), with corresponding low positive predictive value (72.6%, 95 percent CI: 72.3% - 72.9%) and high negative predictive value (99.8%, 95 percent CI: 99.8% - 99.8%). Sensitivity and specificity were similar when limiting the analysis to only intensive care unit patients and mechanically ventilated patients, two groups with higher rates of LTAC utilization. Performance characteristics were slightly better when limiting the analysis to Pennsylvania, a state with relatively high LTAC penetration.</p> <p>Conclusions</p> <p>The discharge destination field in administrative data can result in misclassification when used to identify patients transferred to long-term acute care hospitals. Directly observing transfers in the claims is the preferable method, although this approach is only feasible in identified data.</p

Country differences in the diagnosis and management of coronary heart disease : a comparison between the US, the UK and Germany

Background The way patients with coronary heart disease (CHD) are treated is partly determined by non-medical factors. There is a solid body of evidence that patient and physician characteristics influence doctors' management decisions. Relatively little is known about the role of structural issues in the decision making process. This study focuses on the question whether doctors' diagnostic and therapeutic decisions are influenced by the health care system in which they take place. This non-medical determinant of medical decision-making was investigated in an international research project in the US, the UK and Germany. Methods Videotaped patients within an experimental study design were used. Experienced actors played the role of patients with symptoms of CHD. Several alternative versions were taped featuring the same script with patients of different sex, age and social status. The videotapes were shown to 384 randomly selected primary care physicians in the three countries under study. The sample was stratified on gender and duration of professional experience. Physicians were asked how they would diagnose and manage the patient after watching the video vignette using a questionnaire with standardised and open-ended questions. Results Results show only small differences in decision making between British and American physicians in essential aspects of care. About 90% of the UK and US doctors identified CHD as one of the possible diagnoses. Further similarities were found in test ordering and lifestyle advice. Some differences between the US and UK were found in the certainty of the diagnoses, prescribed medications and referral behaviour. There are numerous significant differences between Germany and the other two countries. German physicians would ask fewer questions, they would order fewer tests, prescribe fewer medications and give less lifestyle advice. Conclusion Although all physicians in the three countries under study were presented exactly the same patient, some disparities in the diagnostic and patient management decisions were evident. Since other possible influences on doctors treatment decisions are controlled within the experimental design, characteristics of the health care system seem to be a crucial factor within the decision making process

Impact of optical coherence tomography on diagnostic decision-making by UK community optometrists: a clinical vignette study.

PURPOSE: In recent years, there has been widespread investment in imaging technologies by community optometrists in the UK, most notably optical coherence tomography (OCT). The aim of the current study was to determine the value of OCT in the diagnosis of posterior segment diseases in a representative sample of community optometrists using a clinical vignette methodology. METHODS: A group of community optometrists (n = 50) initially completed a standardised training package on OCT interpretation followed by a computer-based assessment featuring 52 clinical vignettes, containing images of healthy (n = 8) or glaucomatous (n = 18) discs or healthy (n = 8) or diseased (n = 18) fundi. Each vignette featured either a single fundus/disc photographic image, or a combination of a fundus/disc image with the corresponding OCT scan. An expert panel confirmed that the fundus images presented alone and those in combination with OCT data were of a similar level of difficulty and that the cases were typical of those seen in primary care. For each case, the optometrist selected their diagnosis from a pull-down list and reported their confidence in their decision using a 10-point Likert scale. Pairwise comparisons of the fundus image alone and fundus image/OCT combination were made for both diagnostic performance and confidence. RESULTS: The mean percentage of correct diagnoses using fundus imaging alone was 62% (95% CI 59-64%) and for the combination of fundus image/OCT was 80% (95% CI 77-82%). The mean false negative rate with fundus alone was 27% reducing to 13% with the OCT combination. Median confidence scores for fundus imaging alone was 8.0 (IQR 7.0-8.0) and 8.3 (IQR 8.0-9.0) for the combination. Improvements in performance and confidence were statistically significant (p < 0.001). CONCLUSION: The results from this vignette study suggests that OCT improves optometrists' diagnostic performance compared to fundus observation alone. These initial results suggest that OCT provides valuable additional data that could augment case-finding for glaucoma and retinal disease; however, further research is needed to assess its diagnostic performance in a routine clinical practice setting

The missions of medical schools: the pursuit of health in the service of society

Mission statements and role documents of medical schools in the United Kingdom, United States, Canada and Australia have been examined on their Internet Web sites and categorised in purpose, content and presentation. The format and content are highly variable, but there is a common vision of three integral roles, namely, education, advancement of knowledge and service to society. Other frequent themes include tradition and historical perspective, service for designated communities, and benchmarking to accreditation standards. Differences in content reflect variable interpretation of the notion of "mission", and local or national characteristics such as institutional affiliations, the types, levels and organisation of medical education, relationships with health systems, and extent of multi-professional education. Outcomes data and measures of medical school performance referenced to the institution's stated missions are rarely encountered. Mission documents placed on the Internet are in the public domain. These Web sites and documents and linked information constitute a valuable new resource for international exchange of approaches and ideas in medical education and generally in academic medicine. Routine inclusion of outcome or performance data could help to demonstrate the community roles and social accountability of medical schools This paper proposes that partial standardisation of these Web documents could enhance their value both internally and for external readers. A generic descriptive statement template is offered