Do physician outcome judgments and judgment biases contribute to inappropriate use of treatments? Study protocol

<p>Abstract</p> <p>Background</p> <p>There are many examples of physicians using treatments inappropriately, despite clear evidence about the circumstances under which the benefits of such treatments outweigh their harms. When such over- or under- use of treatments occurs for common diseases, the burden to the healthcare system and risks to patients can be substantial. We propose that a major contributor to inappropriate treatment may be how clinicians judge the likelihood of important treatment outcomes, and how these judgments influence their treatment decisions. The current study will examine the role of judged outcome probabilities and other cognitive factors in the context of two clinical treatment decisions: 1) prescription of antibiotics for sore throat, where we hypothesize overestimation of benefit and underestimation of harm leads to over-prescription of antibiotics; and 2) initiation of anticoagulation for patients with atrial fibrillation (AF), where we hypothesize that underestimation of benefit and overestimation of harm leads to under-prescription of warfarin.</p> <p>Methods</p> <p>For each of the two conditions, we will administer surveys of two types (Type 1 and Type 2) to different samples of Canadian physicians. The primary goal of the Type 1 survey is to assess physicians' perceived outcome probabilities (both good and bad outcomes) for the target treatment. Type 1 surveys will assess judged outcome probabilities in the context of a representative patient, and include questions about how physicians currently treat such cases, the recollection of rare or vivid outcomes, as well as practice and demographic details. The primary goal of the Type 2 surveys is to measure the specific factors that drive individual clinical judgments and treatment decisions, using a 'clinical judgment analysis' or 'lens modeling' approach. This survey will manipulate eight clinical variables across a series of sixteen realistic case vignettes. Based on the survey responses, we will be able to identify which variables have the greatest effect on physician judgments, and whether judgments are affected by inappropriate cues or incorrect weighting of appropriate cues. We will send antibiotics surveys to family physicians (300 per survey), and warfarin surveys to both family physicians and internal medicine specialists (300 per group per survey), for a total of 1,800 physicians. Each Type 1 survey will be two to four pages in length and take about fifteen minutes to complete, while each Type 2 survey will be eight to ten pages in length and take about thirty minutes to complete.</p> <p>Discussion</p> <p>This work will provide insight into the extent to which clinicians' judgments about the likelihood of important treatment outcomes explain inappropriate treatment decisions. This work will also provide information necessary for the development of an individualized feedback tool designed to improve treatment decisions. The techniques developed here have the potential to be applicable to a wide range of clinical areas where inappropriate utilization stems from biased judgments.</p

A new method for determining physician decision thresholds using empiric, uncertain recommendations

<p>Abstract</p> <p>Background</p> <p>The concept of risk thresholds has been studied in medical decision making for over 30 years. During that time, physicians have been shown to be poor at estimating the probabilities required to use this method. To better assess physician risk thresholds and to more closely model medical decision making, we set out to design and test a method that derives thresholds from actual physician treatment recommendations. Such an approach would avoid the need to ask physicians for estimates of patient risk when trying to determine individual thresholds for treatment. Assessments of physician decision making are increasingly relevant as new data are generated from clinical research. For example, recommendations made in the setting of ocular hypertension are of interest as a large clinical trial has identified new risk factors that should be considered by physicians. Precisely how physicians use this new information when making treatment recommendations has not yet been determined.</p> <p>Results</p> <p>We derived a new method for estimating treatment thresholds using ordinal logistic regression and tested it by asking ophthalmologists to review cases of ocular hypertension before expressing how likely they would be to recommend treatment. Fifty-eight physicians were recruited from the American Glaucoma Society. Demographic information was collected from the participating physicians and the treatment threshold for each physician was estimated. The method was validated by showing that while treatment thresholds varied over a wide range, the most common values were consistent with the 10-15% 5-year risk of glaucoma suggested by expert opinion and decision analysis.</p> <p>Conclusions</p> <p>This method has advantages over prior means of assessing treatment thresholds. It does not require physicians to explicitly estimate patient risk and it allows for uncertainty in the recommendations. These advantages will make it possible to use this method when assessing interventions intended to alter clinical decision making.</p

Unanswered ethical and scientific questions for trials of invasive interventions for coronary disease: The case of single vessel disease

Trials in the 1990s demonstrated that medical therapy is as effective as invasive therapies for treating single-vessel coronary disease. Yet more recent studies enrolling patients with this condition have focused on evaluating only invasive approaches, namely, stenting versus coronary artery bypass surgery. Several ethical and scientific questions remain unanswered regarding the conduct of these later trials. Were they justified? Why wasn't a medical therapy arm included? Were subjects informed about the availability of medical therapy as an equivalent option? Was optimized medical therapy given prior to randomization? The absence of clear answers to these questions raises the possibility of serious bias in favor of invasive interventions. Considering that medical therapy is underutilized in patients with coronary disease, efforts should focus more on increasing utilization of medical therapy and proper selection of noninvasive interventions

Results from the national sepsis practice survey: predictions about mortality and morbidity and recommendations for limitation of care orders

Introduction: Critically ill patients and families rely upon physicians to provide estimates of prognosis and recommendations for care. Little is known about patient and clinician factors which influence these predictions. The association between these predictions and recommendations for continued aggressive care is also understudied. Methods: We administered a mail-based survey with simulated clinical vignettes to a random sample of the Critical Care Assembly of the American Thoracic Society. Vignettes represented a patient with septic shock with multi-organ failure with identical APACHE II scores and sepsis-associated organ failures. Vignettes varied by age (50 or 70 years old), body mass index (BMI) (normal or obese) and co-morbidities (none or recently diagnosed stage IIA lung cancer). All subjects received the vignettes with the highest and lowest mortality predictions from pilot testing and two additional, randomly selected vignettes. Respondents estimated outcomes and selected care for each hypothetical patient. Results: Despite identical severity of illness, the range of estimates for hospital mortality (5th to 95th percentile range, 17% to 78%) and for problems with self-care (5th to 95th percentile range, 2% to 74%) was wide. Similar variation was observed when clinical factors (age, BMI, and co-morbidities) were identical. Estimates of hospital mortality and problems with self-care among survivors were significantly higher in vignettes with obese BMIs (4.3% and 5.3% higher, respectively), older age (8.2% and 11.6% higher, respectively), and cancer diagnosis (5.9% and 6.9% higher, respectively). Higher estimates of mortality (adjusted odds ratio 1.29 per 10% increase in predicted mortality), perceived problems with self-care (adjusted odds ratio 1.26 per 10% increase in predicted problems with self-care), and early-stage lung cancer (adjusted odds ratio 5.82) were independently associated with recommendations to limit care. Conclusions: The studied clinical factors were consistently associated with poorer outcome predictions but did not explain the variation in prognoses offered by experienced physicians. These observations raise concern that provided information and the resulting decisions about continued aggressive care may be influenced by individual physician perception. To provide more reliable and accurate estimates of outcomes, tools are needed which incorporate patient characteristics and preferences with physician predictions and practices

Changing treatment patterns for coronary artery revascularization in Canada: the projected impact of drug eluting stents

BACKGROUND: To evaluate current treatment patterns for coronary artery revascularization in Canada and explore the potential impact of drug eluting stents (DES) on these treatment patterns. METHODS: Eleven cardiologists at multiple Canadian academic centers completed a questionnaire on coronary artery revascularization rates and treatment patterns. RESULTS: Participating physicians indicated slightly higher rates of PTCA, CABG, and stent implantation than reported in CCN publications. Participants estimated that 24% of all patients currently receiving bare metal stents (BMS) would receive DES in the first year following DES approval in Canada, although there was a large range of estimates around this value (5% to 65%). By the fifth year following DES approval, it was estimated that 85% of BMS patients would instead receive DES. Among diabetic patients, estimates ranged from 43% in the first year following approval to 91% in the fifth year. For all patients currently receiving CABG, mean use of DES instead was estimated at 12% in the first year to 42% at five years; rates among diabetic patients currently undergoing CABG were 17% in the first year to 49% in the fifth year. CONCLUSIONS: These results suggest a continued increase in revascularization procedures in Canada. Based on the panel's responses, it is likely that a trend away from CABG towards PTCA will continue in Canada, and will be augmented by the availability of DES as a treatment option. The availability of DES as a treatment option in Canada may change the threshold at which revascularization procedures are considered

A comparison between the APACHE II and Charlson Index Score for predicting hospital mortality in critically ill patients

<p>Abstract</p> <p>Background</p> <p>Risk adjustment and mortality prediction in studies of critical care are usually performed using acuity of illness scores, such as Acute Physiology and Chronic Health Evaluation II (APACHE II), which emphasize physiological derangement. Common risk adjustment systems used in administrative datasets, like the Charlson index, are entirely based on the presence of co-morbid illnesses. The purpose of this study was to compare the discriminative ability of the Charlson index to the APACHE II in predicting hospital mortality in adult multisystem ICU patients.</p> <p>Methods</p> <p>This was a population-based cohort design. The study sample consisted of adult (>17 years of age) residents of the Calgary Health Region admitted to a multisystem ICU between April 2002 and March 2004. Clinical data were collected prospectively and linked to hospital outcome data. Multiple regression analyses were used to compare the performance of APACHE II and the Charlson index.</p> <p>Results</p> <p>The Charlson index was a poor predictor of mortality (C = 0.626). There was minimal difference between a baseline model containing age, sex and acute physiology score (C = 0.74) and models containing either chronic health points (C = 0.76) or Charlson index variations (C = 0.75, 0.76, 0.77). No important improvement in prediction occurred when the Charlson index was added to the full APACHE II model (C = 0.808 to C = 0.813).</p> <p>Conclusion</p> <p>The Charlson index does not perform as well as the APACHE II in predicting hospital mortality in ICU patients. However, when acuity of illness scores are unavailable or are not recorded in a standard way, the Charlson index might be considered as an alternative method of risk adjustment and therefore facilitate comparisons between intensive care units.</p

Evaluation of the safety of C-spine clearance by paramedics: design and methodology

<p>Abstract</p> <p>Background</p> <p>Canadian Emergency Medical Services annually transport 1.3 million patients with potential neck injuries to local emergency departments. Less than 1% of those patients have a c-spine fracture and even less (0.5%) have a spinal cord injury. Most injuries occur before the arrival of paramedics, not during transport to the hospital, yet most patients are transported in ambulances immobilized. They stay fully immobilized until a bed is available, or until physician assessment and/or X-rays are complete. The prolonged immobilization is often unnecessary and adds to the burden of already overtaxed emergency medical services systems and crowded emergency departments.</p> <p>Methods/Design</p> <p>The goal of this study is to evaluate the safety and potential impact of an active strategy that allows paramedics to assess very low-risk trauma patients using a validated clinical decision rule, the Canadian C-Spine Rule, in order to determine the need for immobilization during transport to the emergency department.</p> <p>This cohort study will be conducted in Ottawa, Canada with one emergency medical service. Paramedics with this service participated in an earlier validation study of the Canadian C-Spine Rule. Three thousand consecutive, alert, stable adult trauma patients with a potential c-spine injury will be enrolled in the study and evaluated using the Canadian C-Spine Rule to determine the need for immobilization. The outcomes that will be assessed include measures of safety (numbers of missed fractures and serious adverse outcomes), measures of clinical impact (proportion of patients transported without immobilization, key time intervals) and performance of the Rule.</p> <p>Discussion</p> <p>Approximately 40% of all very low-risk trauma patients could be transported safely, without c-spine immobilization, if paramedics were empowered to make clinical decisions using the Canadian C-Spine Rule. This safety study is an essential step before allowing all paramedics across Canada to selectively immobilize trauma victims before transport. Once safety and potential impact are established, we intend to implement a multi-centre study to study actual impact.</p> <p>Trial Registration</p> <p>ClinicalTrials.gov <a href="http://www.clinicaltrials.gov/ct2/show/NCT01188447">NCT01188447</a></p

Implementation science: a role for parallel dual processing models of reasoning?

BACKGROUND: A better theoretical base for understanding professional behaviour change is needed to support evidence-based changes in medical practice. Traditionally strategies to encourage changes in clinical practices have been guided empirically, without explicit consideration of underlying theoretical rationales for such strategies. This paper considers a theoretical framework for reasoning from within psychology for identifying individual differences in cognitive processing between doctors that could moderate the decision to incorporate new evidence into their clinical decision-making. DISCUSSION: Parallel dual processing models of reasoning posit two cognitive modes of information processing that are in constant operation as humans reason. One mode has been described as experiential, fast and heuristic; the other as rational, conscious and rule based. Within such models, the uptake of new research evidence can be represented by the latter mode; it is reflective, explicit and intentional. On the other hand, well practiced clinical judgments can be positioned in the experiential mode, being automatic, reflexive and swift. Research suggests that individual differences between people in both cognitive capacity (e.g., intelligence) and cognitive processing (e.g., thinking styles) influence how both reasoning modes interact. This being so, it is proposed that these same differences between doctors may moderate the uptake of new research evidence. Such dispositional characteristics have largely been ignored in research investigating effective strategies in implementing research evidence. Whilst medical decision-making occurs in a complex social environment with multiple influences and decision makers, it remains true that an individual doctor's judgment still retains a key position in terms of diagnostic and treatment decisions for individual patients. This paper argues therefore, that individual differences between doctors in terms of reasoning are important considerations in any discussion relating to changing clinical practice. SUMMARY: It is imperative that change strategies in healthcare consider relevant theoretical frameworks from other disciplines such as psychology. Generic dual processing models of reasoning are proposed as potentially useful in identifying factors within doctors that may moderate their individual uptake of evidence into clinical decision-making. Such factors can then inform strategies to change practice

Practice Inquiry: Clinical Uncertainty as a Focus for Small-Group Learning and Practice Improvement

PROBLEM: Many primary care physicians in nonacademic settings lack a collegial forum for engaging the clinical uncertainties inherent in their work. PROGRAM DESCRIPTION: “Practice Inquiry” is proposed as a set of small-group, practice-based learning and improvement (PBLI) methods designed to help clinicians better manage case-based clinical uncertainty. Clinicians meet regularly at their offices/clinics to present dilemma cases, share clinical experience, review evidence for blending with experience, and draw implications for practice improvement. From 2001 through 2005, Practice Inquiry was introduced to sites in the San Francisco Bay Area as a demonstration effort. Meeting rosters, case logs, a feedback survey, and meeting field notes documented implementation and provided data for a formative, qualitative evaluation. PROGRAM EVALUATION: Of the 30 sites approached, 14 held introductory meetings. As of summer 2006, 98 clinicians in 11 sites continue to hold regularly scheduled group meetings. Of the 118 patient cases presented in the seven oldest groups, clinician–patient relationship and treatment dilemmas were most common. Clinician feedback and meeting transcript data provided insights into how busy practitioners shared cases, developed trust, and learned new knowledge/skills for moving forward with patients. DISCUSSION: Ongoing clinician involvement suggests that Practice Inquiry is a feasible, acceptable, and potentially useful set of PBLI methods. Two of the Practice Inquiry’s group learning tasks received comparatively less focus: integrating research evidence with clinical experience and tracking dilemma case outcomes. Future work should focus on reducing the methodological limitations of a demonstration effort and examining factors affecting clinician participation. Set-aside work time for clinicians, or other equally potent incentives, will be necessary for the further elaboration of these PBLI methods aimed at managing uncertainty

Vignette studies of medical choice and judgement to study caregivers' medical decision behaviour: systematic review

BACKGROUND: Vignette studies of medical choice and judgement have gained popularity in the medical literature. Originally developed in mathematical psychology they can be used to evaluate physicians' behaviour in the setting of diagnostic testing or treatment decisions. We provide an overview of the use, objectives and methodology of these studies in the medical field. METHODS: Systematic review. We searched in electronic databases; reference lists of included studies. We included studies that examined medical decisions of physicians, nurses or medical students using cue weightings from answers to structured vignettes. Two reviewers scrutinized abstracts and examined full text copies of potentially eligible studies. The aim of the included studies, the type of clinical decision, the number of participants, some technical aspects, and the type of statistical analysis were extracted in duplicate and discrepancies were resolved by consensus. RESULTS: 30 reports published between 1983 and 2005 fulfilled the inclusion criteria. 22 studies (73%) reported on treatment decisions and 27 (90%) explored the variation of decisions among experts. Nine studies (30%) described differences in decisions between groups of caregivers and ten studies (33%) described the decision behaviour of only one group. Only six studies (20%) compared decision behaviour against an empirical reference of a correct decision. The median number of considered attributes was 6.5 (IQR 4-9), the median number of vignettes was 27 (IQR 16-40). In 17 studies, decision makers had to rate the relative importance of a given vignette; in six studies they had to assign a probability to each vignette. Only ten studies (33%) applied a statistical procedure to account for correlated data. CONCLUSION: Various studies of medical choice and judgement have been performed to depict weightings of the value of clinical information from answers to structured vignettes of care givers. We found that the design and analysis methods used in current applications vary considerably and could be improved in a large number of cases