A conceptual framework and protocol for defining clinical decision support objectives applicable to medical specialties.

BackgroundThe U.S. Centers for Medicare and Medicaid Services established the Electronic Health Record (EHR) Incentive Program in 2009 to stimulate the adoption of EHRs. One component of the program requires eligible providers to implement clinical decision support (CDS) interventions that can improve performance on one or more quality measures pre-selected for each specialty. Because the unique decision-making challenges and existing HIT capabilities vary widely across specialties, the development of meaningful objectives for CDS within such programs must be supported by deliberative analysis.DesignWe developed a conceptual framework and protocol that combines evidence review with expert opinion to elicit clinically meaningful objectives for CDS directly from specialists. The framework links objectives for CDS to specialty-specific performance gaps while ensuring that a workable set of CDS opportunities are available to providers to address each performance gap. Performance gaps may include those with well-established quality measures but also priorities identified by specialists based on their clinical experience. Moreover, objectives are not constrained to performance gaps with existing CDS technologies, but rather may include those for which CDS tools might reasonably be expected to be developed in the near term, for example, by the beginning of Stage 3 of the EHR Incentive program. The protocol uses a modified Delphi expert panel process to elicit and prioritize CDS meaningful use objectives. Experts first rate the importance of performance gaps, beginning with a candidate list generated through an environmental scan and supplemented through nominations by panelists. For the highest priority performance gaps, panelists then rate the extent to which existing or future CDS interventions, characterized jointly as "CDS opportunities," might impact each performance gap and the extent to which each CDS opportunity is compatible with specialists' clinical workflows. The protocol was tested by expert panels representing four clinical specialties: oncology, orthopedic surgery, interventional cardiology, and pediatrics

How are medical groups identified as high-performing? The effect of different approaches to classification of performance

BackgroundPayers and policy makers across the international healthcare market are increasingly using publicly available summary measures to designate providers as "high-performing", but no consistently-applied approach exists to identifying high performers. This paper uses publicly available data to examine how different classification approaches influence which providers are designated as "high-performers".MethodsWe conducted a quantitative analysis of cross-sectional publicly-available performance data in the U.S. We used 2014 Minnesota Community Measurement data from 58 medical groups to classify performance across 4 domains: quality (two process measures of cancer screening and 2 composite measures of chronic disease management), total cost of care, access (a composite CAHPS measure), and patient experience (3 CAHPS measures). We classified medical groups based on performance using either relative thresholds or absolute values of performance on all included measures.ResultsUsing relative thresholds, none of the 58 medical groups achieved performance in the top 25% or 35% in all 4 performance domains. A relative threshold of 40% was needed before one group was classified as high-performing in all 4 domains. Using absolute threshold values, two medical groups were classified as high-performing across all 4 domains. In both approaches, designating "high performance" using fewer domains led to more groups designated as high-performers, though there was little to moderate concordance across identified "high-performing" groups.ConclusionsClassification of medical groups as high performing is sensitive to the domains of performance included, the classification approach, and choice of threshold. With increasing focus on achieving high performance in healthcare delivery, the absence of a consistently-applied approach to identify high performers impedes efforts to reliably compare, select and reward high-performing providers

What Role Does Efficiency Play in Understanding the Relationship Between Cost and Quality in Physician Organizations?

BackgroundThe belief that there is inefficiency, or the potential to improve patient health at current levels of spending, is driving the push for greater value in health care. Previous studies demonstrate overuse of a narrow set of services, suggesting provider inefficiency, but existing studies neither quantify inefficiency more broadly nor assess its variation across physician organizations (POs).Data and methodsWe used data on quality of care and total cost of care from 129 California POs participating in a statewide value-based pay-for-performance program. We estimated a production function with quality as the output and cost as the input, using a stochastic frontier model, to develop a measure of relative efficiency for each PO. To validate the efficiency measure, we examined correlations of PO efficiency estimates with indicators representing overuse of services.ResultsThe estimated production function showed that PO quality was positively associated with costs, although there were diminishing marginal returns to spending. A certain minimum level of spending was associated with high quality even among efficient POs. Most strikingly, however, POs had substantial variation in efficiency, producing widely differing levels of quality for the same cost.ConclusionsDifferences among POs in the efficiency with which they produce quality suggest opportunities for improvements in care delivery that increase quality without increasing spending

Challenges in assessing the process-outcome link in practice.

The expanded use of clinical process-of-care measures to assess the quality of health care in the context of public reporting and pay-for-performance applications has led to a desire to demonstrate the value of such efforts in terms of improved patient outcomes. The inability to observe associations between improved delivery of clinical processes and improved clinical outcomes in practice has raised concerns about the value of holding providers accountable for delivery of clinical processes of care. Analyses that attempt to investigate this relationship are fraught with many challenges, including selection of an appropriate outcome, the proximity of the outcome to the receipt of the clinical process, limited power to detect an effect, small expected effect sizes in practice, potential bias due to unmeasured confounding factors, and difficulties due to changes in measure specification over time. To avoid potentially misleading conclusions about an observed or lack of observed association between a clinical process of care and an outcome in the context of observational studies, individuals conducting and interpreting such studies should carefully consider, evaluate, and acknowledge these types of challenges

Imputation of SF-12 health scores for respondents with partially missing data.

ObjectiveTo create an efficient imputation algorithm for imputing the SF-12 physical component summary (PCS) and mental component summary (MCS) scores when patients have one to eleven SF-12 items missing.Study settingPrimary data collection was performed between 1996 and 1998.Study designMulti-pattern regression was conducted to impute the scores using only available SF-12 items (simple model), and then supplemented by demographics, smoking status and comorbidity (enhanced model) to increase the accuracy. A cut point of missing SF-12 items was determined for using the simple or the enhanced model. The algorithm was validated through simulation.Data collectionThirty-thousand-three-hundred and eight patients from 63 physician groups were surveyed for a quality of care study in 1996, which collected the SF-12 and other information. The patients were classified as "chronic" patients if they reported that they had diabetes, heart disease, asthma/chronic obstructive pulmonary disease, or low back pain. A follow-up survey was conducted in 1998.Principal findingsThirty-one percent of the patients missed at least one SF-12 item. Means of variance of prediction and standard errors of the mean imputed scores increased with the number of missing SF-12 items. Correlations between the observed and the imputed scores derived from the enhanced models were consistently higher than those derived from the simple model and the increments were significant for patients with > or =6 missing SF-12 items (p<.03).ConclusionMissing SF-12 items are prevalent and lead to reduced analytical power. Regression-based multi-pattern imputation using the available SF-12 items is efficient and can produce good estimates of the scores. The enhancement from the additional patient information can significantly improve the accuracy of the imputed scores for patients with > or =6 items missing, leading to estimated scores that are as accurate as that of patients with <6 missing items