Joint Indirect Standardization When Only Marginal Distributions are Observed in the Index Population

<p>It is a common interest in medicine to determine whether a hospital meets a benchmark created from an aggregate reference population, after accounting for differences in distributions of multiple covariates. Due to the difficulties of collecting individual-level data, however, it is often the case that only marginal distributions of the covariates are available, making covariate-adjusted comparison challenging. We propose and evaluate a novel approach for conducting indirect standardization when only marginal covariate distributions of the studied hospital are known, but complete information is available for the reference hospitals. We do this with the aid of two existing methods: iterative proportional fit, which estimates the cells of a contingency table when only marginal sums are known, and synthetic control methods, which create a counterfactual control group using a weighted combination of potential control groups. The proper application of these existing methods for indirect standardization would require accounting for the statistical uncertainties induced by a situation where no individual-level data are collected from the studied population. We address this need with a novel method which uses a random Dirichlet parameterization of the synthetic control weights to estimate uncertainty intervals for the standard incidence ratio. We demonstrate our novel methods by estimating hospital-level standardized incidence ratios for comparing the adjusted probability of computed tomography examinations with high radiations doses, relative to a reference standard and we evaluate out methods in a simulation study. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.</p

Relationship of Predicted Risk of Developing Invasive Breast Cancer, as Assessed with Three Models, and Breast Cancer Mortality among Breast Cancer Patients

<div><p>Purpose</p><p>Breast cancer risk prediction models are used to plan clinical trials and counsel women; however, relationships of predicted risks of breast cancer incidence and prognosis after breast cancer diagnosis are unknown.</p><p>Methods</p><p>Using largely pre-diagnostic information from the Breast Cancer Surveillance Consortium (BCSC) for 37,939 invasive breast cancers (1996–2007), we estimated 5-year breast cancer risk (<1%; 1–1.66%; ≥1.67%) with three models: BCSC 1-year risk model (BCSC-1; adapted to 5-year predictions); Breast Cancer Risk Assessment Tool (BCRAT); and BCSC 5-year risk model (BCSC-5). Breast cancer-specific mortality post-diagnosis (range: 1–13 years; median: 5.4–5.6 years) was related to predicted risk of developing breast cancer using unadjusted Cox proportional hazards models, and in age-stratified (35–44; 45–54; 55–69; 70–89 years) models adjusted for continuous age, BCSC registry, calendar period, income, mode of presentation, stage and treatment. Mean age at diagnosis was 60 years.</p><p>Results</p><p>Of 6,021 deaths, 2,993 (49.7%) were ascribed to breast cancer. In unadjusted case-only analyses, predicted breast cancer risk ≥1.67% versus <1.0% was associated with lower risk of breast cancer death; BCSC-1: hazard ratio (HR) = 0.82 (95% CI = 0.75–0.90); BCRAT: HR = 0.72 (95% CI = 0.65–0.81) and BCSC-5: HR = 0.84 (95% CI = 0.75–0.94). Age-stratified, adjusted models showed similar, although mostly non-significant HRs. Among women ages 55–69 years, HRs approximated 1.0. Generally, higher predicted risk was inversely related to percentages of cancers with unfavorable prognostic characteristics, especially among women 35–44 years.</p><p>Conclusions</p><p>Among cases assessed with three models, higher predicted risk of developing breast cancer was not associated with greater risk of breast cancer death; thus, these models would have limited utility in planning studies to evaluate breast cancer mortality reduction strategies. Further, when offering women counseling, it may be useful to note that high predicted risk of developing breast cancer does not imply that if cancer develops it will behave aggressively.</p></div

Predicted risk of developing breast cancer versus risk of death (age-stratified, adjusted models).

<p>Age-stratified models, adjusted for age in single years, registry, year of diagnosis, mode of detection, AJCC stage, treatment (surgery and chemotherapy: yes/no) and income (zip code of residence)</p

Characteristics of 37,939 Women with Breast Cancer.

<p>Characteristics of 37,939 Women with Breast Cancer.</p

Predicted 5-year risk of developing breast cancer versus clinicopathologic characteristics.

<p>Predicted 5-year risk of developing breast cancer versus clinicopathologic characteristics.</p

Addressing the Challenge of Assessing Physician-Level Screening Performance: Mammography as an Example

<div><p>Background</p><p>Motivated by the challenges in assessing physician-level cancer screening performance and the negative impact of misclassification, we propose a method (using mammography as an example) that enables confident assertion of adequate or inadequate performance or alternatively recognizes when more data is required.</p><p>Methods</p><p>Using established metrics for mammography screening performance–cancer detection rate (CDR) and recall rate (RR)–and observed benchmarks from the Breast Cancer Surveillance Consortium (BCSC), we calculate the minimum volume required to be 95% confident that a physician is performing at or above benchmark thresholds. We graphically display the minimum observed CDR and RR values required to confidently assert adequate performance over a range of interpretive volumes. We use a prospectively collected database of consecutive mammograms from a clinical screening program outside the BCSC to illustrate how this method classifies individual physician performance as volume accrues.</p><p>Results</p><p>Our analysis reveals that an annual interpretive volume of 2770 screening mammograms, above the United States’ (US) mandatory (480) and average (1777) annual volumes but below England’s mandatory (5000) annual volume is necessary to confidently assert that a physician performed adequately. In our analyzed US practice, a single year of data uniformly allowed confident assertion of adequate performance in terms of RR but not CDR, which required aggregation of data across more than one year.</p><p>Conclusion</p><p>For individual physician quality assessment in cancer screening programs that target low incidence populations, considering imprecision in observed performance metrics due to small numbers of patients with cancer is important.</p></div

Distribution of study population.

<p>*According to Rosenberg, et al. <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0089418#pone.0089418-Rosenberg1" target="_blank">[19]</a>.</p

Individual physician performance assessment based on volume.

<p>Plots of (A) CDR and (B) RR for the 4 included radiologists at 6 volumes from 500 examinations (then at 1000 and subsequently 1000 exam increments) to the maximum volume read over the 3 years or 5000 total (whichever was least).</p

Defining adequate performance based on volume.

<p>Plots demonstrate our method for constructing curves by using the benchmark threshold as the limit of 95% confidence based on volume: (A) CDR performance levels are established using 2.4 as the lower boundary for 95% CI of adequate performance (CIs shown) and the upper boundary for inadequate performance (CIs not shown). This methodology shows (indicated with a black dot) that a volume of 2770 is required to confidently assert the CDR benchmark median of 4.4/1000 is adequate; (B) RR performance levels are established using 16.8 as the upper boundary for 95% CI of adequate (CI shown) and inadequate (CI not shown) performance. A volume of 120 (indicated with a black dot) is required to confidently assert the RR benchmark median of 9.7% is adequate. Plots define regions of adequate, uncertain, and inadequate performance for (B) CDR and (D) RR.</p

BI-RADS^* final assessment categories with associated recommendation.

<p>*BI-RADS Version 4 <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0089418#pone.0089418-Blanks3" target="_blank">[22]</a>.</p