Establishing the effectiveness of patient decision aids: key constructs and measurement instruments

Background: Establishing the effectiveness of patient decision aids (PtDA) requires evidence that PtDAs improve the quality of the decision-making process and the quality of the choice made, or decision quality. The aim of this paper is to review the theoretical and empirical evidence for PtDA effectiveness and discuss emerging practical and research issues in the measurement of effectiveness. Methods: This updated overview incorporates: a) an examination of the instruments used to measure five key decision-making process constructs (i.e., recognize decision, feel informed about options and outcomes, feel clear about goals and preferences, discuss goals and preferences with health care provider, and be involved in decisions) and decision quality constructs (i.e., knowledge, realistic expectations, values-choice agreement) within the 86 trials in the Cochrane review; and b) a summary of the 2011 Cochrane Collaboration’s review of PtDAs for these key constructs. Data on the constructs and instruments used were extracted independently by two authors from the 86 trials and any disagreements were resolved by discussion, with adjudication by a third party where required. Results: The 86 studies provide considerable evidence that PtDAs improve the decision-making process and decision quality. A majority of the studies (76/86; 88%) measured at least one of the key decision-making process or decision quality constructs. Seventeen different measurement instruments were used to measure decision-making process constructs, but no single instrument covered all five constructs. The Decisional Conflict Scale was most commonly used (n = 47), followed by the Control Preference Scale (n = 9). Many studies reported one or more constructs of decision quality, including knowledge (n = 59), realistic expectation of risks and benefits (n = 21), and values-choice agreement (n = 13). There was considerable variability in how values-choice agreement was defined and determined. No study reported on all key decision-making process and decision quality constructs. Conclusions: Evidence of PtDA effectiveness in improving the quality of the decision-making process and decision quality is strong and growing. There is not, however, consensus or standardization of measurement for either the decision-making process or decision quality. Additional work is needed to develop and evaluate measurement instruments and further explore theoretical issues to advance future research on PtDA effectiveness

Treatment decision-making and the form of risk communication: results of a factorial survey

BACKGROUND: Prospective users of preventive therapies often must evaluate complex information about therapeutic risks and benefits. The purpose of this study was to evaluate the effect of relative and absolute risk information on patient decision-making in scenarios typical of health information for patients. METHODS: Factorial experiments within a telephone survey of the Michigan adult, non-institutionalized, English-speaking population. Average interview lasted 23 minutes. Subjects and sample design: 952 randomly selected adults within a random-digit dial sample of Michigan households. Completion rate was 54.3%. RESULTS: When presented hypothetical information regarding additional risks of breast cancer from a medication to prevent a bone disease, respondents reduced their willingness to recommend a female friend take the medication compared to the baseline rate (66.8% = yes). The decrease was significantly greater with relative risk information. Additional benefit information regarding preventing heart disease from the medication increased willingness to recommend the medication to a female friend relative to the baseline scenario, but did not differ between absolute and relative risk formats. When information about both increased risk of breast cancer and reduced risk of heart disease were provided, typical respondents appeared to make rational decisions consistent with Expected Utility Theory, but the information presentation format affected choices. Those 11% – 33% making decisions contrary to the medical indications were more likely to be Hispanic, older, more educated, smokers, and to have children in the home. CONCLUSIONS: In scenarios typical of health risk information, relative risk information led respondents to make non-normative decisions that were "corrected" when the frame used absolute risk information. This population sample made generally rational decisions when presented with absolute risk information, even in the context of a telephone interview requiring remembering rates given. The lack of effect of gender and race suggests that a standard strategy of presenting absolute risk information may improve patient decision-making

Decision Aids for Benign Prostatic Hyperplasia: Applicability across Race and Education

Background/Method. Decision aids have not been widely tested in diverse audiences. The authors conducted interviews in a 2 x 2 race by education design with participants who were 50 years old (n = 188). The decision aid was a benign prostatic hyperplasia videotape. Results. There was an increase in knowledge equal in all groups, with baseline knowledge higher in whites. The decision stage increased in all groups and was equivalent in the marginal-illiterate subgroup (n = 0.15). Conclusion. Contrary to expectations, results show no difference by race or college education in knowledge gain or increase in reported readiness to decide. The video appeared to produce change across race and education. The end decision stage was high, especially in less educated men. Results suggest that decision aids may be effective without tailoring, as suggested previously to enhance health communication in diverse audiences. Research should test findings in representative samples and in clinical encounters and identify types of knowledge absorbed from decision aids and whether the shift to decision reflects data/knowledge or shared decision-making message.https://journals.sagepub.com/doi/abs/10.1177/0272989x0426701

“It’s Valid and Reliable” Is Not Enough

Background. This review systematically appraises the quality of reporting of measures used in trials to evaluate the effectiveness of patient decision aids (PtDAs) and presents recommendations for minimum reporting standards. Methods. We reviewed measures of decision quality and decision process in 86 randomized controlled trials (RCTs) from the 2011 Cochrane Collaboration systematic review of PtDAs. Data on development of the measures, reliability, validity, responsiveness, precision, interpretability, feasibility, and acceptability were independently abstracted by 2 reviewers. Results. Information from 178 instances of use of measures was abstracted. Very few studies reported data on the performance of measures, with reliability (21%) and validity (16%) being the most common. Studies using new measures were less likely to include information about their psychometric performance. The review was limited to reporting of measures in studies included in the Cochrane review and did not consult prior publications. Conclusions. Very little is reported about the development or performance of measures used to evaluate the effectiveness of PtDAs in published trials. Minimum reporting standards are proposed to enable authors to prepare study reports, editors and reviewers to evaluate submitted papers, and readers to appraise published studies