Patient- and surgeon-adjusted control charts for monitoring performance

Objectives: To determine whether an innovative graphical tool for accurate measurement of individual surgeon performance metrics, adjusted for both surgeon-specific and patient-specific factors, significantly alters interpretation of performance data. Design: Retrospective analysis of all total knee replacements (TKRs) conducted at the host institution between 1996 and 2009. The database was randomly divided into training and testing datasets. Using multivariate generalised estimating equation regression models, the training dataset enabled generation of patient-risk and surgeon-experience adjustment factors. To simulate prospective monitoring of individual surgeon outcomes, the testing dataset was mapped on control charts. Weighted κ statistics were calculated to measure the agreement between patient-risk adjusted and fully adjusted control charts. Setting: Tertiary care academic hospital. Participants: All patients undergoing TKR at the host institution 1996–2009. Main outcome measure Operative efficiency. Results: 5313 procedures were analysed. Adjusted control charts were generated using a training dataset comprising 3756 procedures performed by 13 surgeons. The operative time gradually declined by 121 min with 25 years of experience (p<0.0001). Charts were tested by monitoring four other surgeons, performing an average of 389 procedures each. Adjustment for surgeon experience significantly altered the interpretation of operative efficiency (κ=0.29 (95% CI 0.11 to 0.47)), and enhanced assessment of a surgeon's improvement or diminishment in efficiency over time. Specifically, experience adjustment inverted the interpretation of surgeon efficiency from above average to below average, or from improving to declining performance. Conclusions: Adjustment for surgeon experience is necessary for accurate interpretation of metrics over the course of a surgeon's career. Patient-adjusted and surgeon-adjusted control charts provide an accurate method of monitoring individual operative efficiency

Hospital Surgical Volume, Utilization, Costs and Outcomes of Retroperitoneal Lymph Node Dissection for Testis Cancer

Objectives. Retroperitoneal lymph node dissection (RPLND) outcomes for testis cancer originate mostly from single-center series. We characterized population-based utilization, costs, and outcomes and assessed whether higher volume affects outcomes. Methods and Materials. Using the US Nationwide Inpatient Sample from 2001–2008, we identified 993 RPLND and used propensity score methods to assess utilization, costs, and inpatient outcomes based on hospital surgical volume. Results. 51.6% of RPLND were performed at hospitals where there were two or fewer cases per year. RPLND was more commonly performed at large urban teaching hospitals, where men were younger, more likely to be white and earning incomes exceeding the 50th percentile (all P ≤ .05). Higher hospital volumes were associated with fewer complications and more routine home discharges (all P ≤ .047). However, higher volume hospitals had more transfusions (P = .004) and incurred $1,435 more in median costs (P < .001). Limitations include inability to adjust for tumor characteristics and absence of outpatient outcomes. Conclusions. Sociodemographic differences exist between high versus low volume RPLND hospitals. Although higher volume hospitals had more transfusions and higher costs, perhaps due to more complex cases, they experienced fewer complications. However, most RPLND are performed at hospitals where there were two or fewer cases per year

Geographic Variation Within the Military Health System

Background: This study seeks to quantify variation in healthcare utilization and per capita costs using system-defined geographic regions based on enrollee residence within the Military Health System (MHS). Methods: Data for fiscal years 2007 – 2010 were obtained from the Military Health System under a data sharing agreement with the Defense Health Agency (DHA). DHA manages all aspects of the Department of Defense Military Health System, including TRICARE. Adjusted rates were calculated for per capita costs and for two procedures with high interest to the MHS- back surgery and Cesarean sections for TRICARE Prime and Plus enrollees. Coefficients of variation (CoV) and interquartile ranges (IQR) were calculated and analyzed using residence catchment area as the geographic unit. Catchment areas anchored by a Military Treatment Facility (MTF) were compared to catchment areas not anchored by a MTF. Results: Variation, as measured by CoV, was 0.37 for back surgery and 0.13 for C-sections in FY 2010- comparable to rates documented in other healthcare systems. The 2010 CoV (and average cost) for per capita costs was 0.26 ($3,479.51). Procedure rates were generally lower and CoVs higher in regions anchored by a MTF compared with regions not anchored by a MTF, based on both system-wide comparisons and comparisons of neighboring areas. Conclusions: In spite of its centrally managed system and relatively healthy beneficiaries with very robust health benefits, the MHS is not immune to unexplained variation in utilization and cost of healthcare

Joint generalized estimating equations for multivariate longitudinal binary outcomes with missing data: an application to acquired immune deficiency syndrome data

In a large, prospective longitudinal study designed to monitor cardiac abnormalities in children born to HIV-infected women, instead of a single outcome variable, there are multiple binary outcomes (e.g., abnormal heart rate, abnormal blood pressure, abnormal heart wall thickness) considered as joint measures of heart function over time. In the presence of missing responses at some time points, longitudinal marginal models for these multiple outcomes can be estimated using generalized estimating equations (GEE) (Liang and Zeger, 1986), and consistent estimates can be obtained under the assumption of a missing completely at random (MCAR) mechanism. When the missing data mechanism is missing at random (MAR), that is the probability of missing a particular outcome at a time-point depends on observed values of that outcome and the remaining outcomes at other time points, we propose joint estimation of the marginal models using a single modified GEE based on an EM-type algorithm. The proposed method is motivated by the longitudinal study of cardiac abnormalities in children born to HIV-infected women and analyses of these data are presented to illustrate the application of the method. Further, in an asymptotic study of bias, we show that under an MAR mechanism in which missingness depends on all observed outcome variables, our joint estimation via the modified GEE produces almost unbiased estimates, provided the correlation model has been correctly specified, whereas estimates from standard GEE can lead to substantial bias

Testing for independence in J × K contingency tables with complex sample survey data

Summary: The test of independence of row and column variables in a (J × K) contingency table is a widely used statistical test in many areas of application. For complex survey samples, use of the standard Pearson chi-squared test is inappropriate due to correlation among units within the same cluster

A weighted combination of pseudo-likelihood estimators for longitudinal binary data subject to non-ignorable non-monotone missingness

For longitudinal binary data with non-monotone non-ignorably missing outcomes over time, a full likelihood approach is complicated algebraically, and with many follow-up times, maximum likelihood estimation can be computationally prohibitive. As alternatives, two pseudo-likelihood approaches have been proposed that use minimal parametric assumptions. One formulation requires specification of the marginal distributions of the outcome and missing data mechanism at each time point, but uses an “independence working assumption,” i.e., an assumption that observations are independent over time. Another method avoids having to estimate the missing data mechanism by formulating a “protective estimator.” In simulations, these two estimators can be very inefficient, both for estimating time trends in the first case and for estimating both time-varying and time-stationary effects in the second. In this paper, we propose use of the optimal weighted combination of these two estimators, and in simulations we show that the optimal weighted combination can be much more efficient than either estimator alone. Finally, the proposed method is used to analyze data from two longitudinal clinical trials of HIV-infected patients

An extension of the Wilcoxon rank sum test for complex sample survey data: Extension of Wilcoxon Rank Sum Test

In complex survey sampling, a fraction of a finite population is sampled. Often, the survey is conducted so that each subject in the population has a different probability of being selected into the sample. Further, many complex surveys involve stratification and clustering. For generalizability of the sample to the finite population, these features of the design are usually incorporated in the analysis. While the Wilcoxon rank sum test is commonly used to compare an ordinal variable in bivariate analyses, no simple extension of the Wilcoxon rank sum test has been proposed for complex survey data. With multinomial sampling of independent subjects, the Wilcoxon rank-sum test statistic equals the score test statistic for the group effect from a proportional odds cumulative logistic regression model for an ordinal outcome. Using this regression framework, for complex survey data, we formulate a similar proportional odds cumulative logistic regression model for the ordinal variable, and use an estimating equations score statistic for no group effect as an extension of the Wilcoxon test. The proposed method is applied to a complex survey designed to produce national estimates of the health care use, expenditures, sources of payment, and insurance coverage