The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial.

ObjectivesThe diagnosis of surgical site infection following endoprosthetic reconstruction for bone tumours is frequently a subjective diagnosis. Large clinical trials use blinded Central Adjudication Committees (CACs) to minimise the variability and bias associated with assessing a clinical outcome. The aim of this study was to determine the level of inter-rater and intra-rater agreement in the diagnosis of surgical site infection in the context of a clinical trial.Materials and methodsThe Prophylactic Antibiotic Regimens in Tumour Surgery (PARITY) trial CAC adjudicated 29 non-PARITY cases of lower extremity endoprosthetic reconstruction. The CAC members classified each case according to the Centers for Disease Control (CDC) criteria for surgical site infection (superficial, deep, or organ space). Combinatorial analysis was used to calculate the smallest CAC panel size required to maximise agreement. A final meeting was held to establish a consensus.ResultsFull or near consensus was reached in 20 of the 29 cases. The Fleiss kappa value was calculated as 0.44 (95% confidence interval (CI) 0.35 to 0.53), or moderate agreement. The greatest statistical agreement was observed in the outcome of no infection, 0.61 (95% CI 0.49 to 0.72, substantial agreement). Panelists reached a full consensus in 12 of 29 cases and near consensus in five of 29 cases when CDC criteria were used (superficial, deep or organ space). A stable maximum Fleiss kappa of 0.46 (95% CI 0.50 to 0.35) at CAC sizes greater than three members was obtained.ConclusionsThere is substantial agreement among the members of the PARITY CAC regarding the presence or absence of surgical site infection. Agreement on the level of infection, however, is more challenging. Additional clinical information routinely collected by the prospective PARITY trial may improve the discriminatory capacity of the CAC in the parent study for the diagnosis of infection.Cite this article: J. Nuttall, N. Evaniew, P. Thornley, A. Griffin, B. Deheshi, T. O'Shea, J. Wunder, P. Ferguson, R. L. Randall, R. Turcotte, P. Schneider, P. McKay, M. Bhandari, M. Ghert. The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial. Bone Joint Res 2016;5:347-352. DOI: 10.1302/2046-3758.58.BJR-2016-0036.R1

The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial.

ObjectivesThe diagnosis of surgical site infection following endoprosthetic reconstruction for bone tumours is frequently a subjective diagnosis. Large clinical trials use blinded Central Adjudication Committees (CACs) to minimise the variability and bias associated with assessing a clinical outcome. The aim of this study was to determine the level of inter-rater and intra-rater agreement in the diagnosis of surgical site infection in the context of a clinical trial.Materials and methodsThe Prophylactic Antibiotic Regimens in Tumour Surgery (PARITY) trial CAC adjudicated 29 non-PARITY cases of lower extremity endoprosthetic reconstruction. The CAC members classified each case according to the Centers for Disease Control (CDC) criteria for surgical site infection (superficial, deep, or organ space). Combinatorial analysis was used to calculate the smallest CAC panel size required to maximise agreement. A final meeting was held to establish a consensus.ResultsFull or near consensus was reached in 20 of the 29 cases. The Fleiss kappa value was calculated as 0.44 (95% confidence interval (CI) 0.35 to 0.53), or moderate agreement. The greatest statistical agreement was observed in the outcome of no infection, 0.61 (95% CI 0.49 to 0.72, substantial agreement). Panelists reached a full consensus in 12 of 29 cases and near consensus in five of 29 cases when CDC criteria were used (superficial, deep or organ space). A stable maximum Fleiss kappa of 0.46 (95% CI 0.50 to 0.35) at CAC sizes greater than three members was obtained.ConclusionsThere is substantial agreement among the members of the PARITY CAC regarding the presence or absence of surgical site infection. Agreement on the level of infection, however, is more challenging. Additional clinical information routinely collected by the prospective PARITY trial may improve the discriminatory capacity of the CAC in the parent study for the diagnosis of infection.Cite this article: J. Nuttall, N. Evaniew, P. Thornley, A. Griffin, B. Deheshi, T. O'Shea, J. Wunder, P. Ferguson, R. L. Randall, R. Turcotte, P. Schneider, P. McKay, M. Bhandari, M. Ghert. The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial. Bone Joint Res 2016;5:347-352. DOI: 10.1302/2046-3758.58.BJR-2016-0036.R1

Decipher correlation patterns post prostatectomy: initial experience from 2 342 prospective patients.

BackgroundCurrently, there are multiple commercially available RNA-based biomarkers that are Medicare approved and suggested for use by the National Comprehensive Cancer Network guidelines. There is uncertainty as to which patients benefit from genomic testing and for whom these tests should be ordered. Here, we examined the correlation patterns of Decipher assay to understand the relationship between the Decipher and patient tumor characteristics.MethodsDe-identified Decipher test results (including Decipher risk scores and clinicopathologic data) from 2 342 consecutive radical prostatectomy (RP) patients tested between January and September 2015 were analyzed. For clinical testing, tumor specimen from the highest Gleason grade was sampled using a 1.5 mm tissue punch. Decipher scores were calculated based on a previously locked model. Correlations between Decipher score and clinicopathologic variables were computed using Spearman's rank correlation. Mixed-effect linear models were used to study the association of practice type and Decipher score. The significance level was 0.05 for all tests.ResultsDecipher score had a positive correlation with pathologic Gleason score (PGS; r=0.37, 95% confidence interval (CI) 0.34-0.41), pathologic T-stage (r=0.31, 95% CI 0.28-0.35), CAPRA-S (r=0.32, 95% CI 0.28-0.37) and patient age (r=0.09, 95% CI 0.05-0.13). Decipher reclassified 52%, 76% and 40% of patients in CAPRA-S low-, intermediate- and high-risk groups, respectively. We detected a 28% incidence of high-risk disease through the Decipher score in pT2 patients and 7% low risk in pT3b/pT4, PGS 8-10 patients. There was no significant difference in the Decipher score between patients from community centers and those from academic centers (P=0.82).ConclusionsAlthough Decipher correlated with baseline tumor characteristics for over 2 000 patients, there was significant reclassification of tumor aggressiveness as compared to clinical parameters alone. Utilization of the Decipher genomic classifier can have major implications in assessment of postoperative risk that may impact physician-patient decision making and ultimately patient management

Individual Patient-Level Meta-Analysis of the Performance of the Decipher Genomic Classifier in High-Risk Men After Prostatectomy to Predict Development of Metastatic Disease

Purpose To perform the first meta-analysis of the performance of the genomic classifier test, Decipher, in men with prostate cancer postprostatectomy. Methods MEDLINE, EMBASE, and the Decipher genomic resource information database were searched for published reports between 2011 and 2016 of men treated by prostatectomy that assessed the benefit of the Decipher test. Multivariable Cox proportional hazards models fit to individual patient data were performed; meta-analyses were conducted by pooling the study-specific hazard ratios (HRs) using random-effects modeling. Extent of heterogeneity between studies was determined with the I2 test. Results Five studies (975 total patients, and 855 patients with individual patient-level data) were eligible for analysis, with a median follow-up of 8 years. Of the total cohort, 60.9%, 22.6%, and 16.5% of patients were classified by Decipher as low, intermediate, and high risk, respectively. The 10-year cumulative incidence metastases rates were 5.5%, 15.0%, and 26.7% ( P < .001), respectively, for the three risk classifications. Pooling the study-specific Decipher HRs across the five studies resulted in an HR of 1.52 (95% CI, 1.39 to 1.67; I2 = 0%) per 0.1 unit. In multivariable analysis of individual patient data, adjusting for clinicopathologic variables, Decipher remained a statistically significant predictor of metastasis (HR, 1.30; 95% CI, 1.14 to 1.47; P < .001) per 0.1 unit. The C-index for 10-year distant metastasis of the clinical model alone was 0.76; this increased to 0.81 with inclusion of Decipher. Conclusion The genomic classifier test, Decipher, can independently improve prognostication of patients postprostatectomy, as well as within nearly all clinicopathologic, demographic, and treatment subgroups. Future study of how to best incorporate genomic testing in clinical decision-making and subsequent treatment recommendations is warranted

Individual Patient-Level Meta-Analysis of the Performance of the Decipher Genomic Classifier in High-Risk Men After Prostatectomy to Predict Development of Metastatic Disease

Purpose To perform the first meta-analysis of the performance of the genomic classifier test, Decipher, in men with prostate cancer postprostatectomy. Methods MEDLINE, EMBASE, and the Decipher genomic resource information database were searched for published reports between 2011 and 2016 of men treated by prostatectomy that assessed the benefit of the Decipher test. Multivariable Cox proportional hazards models fit to individual patient data were performed; meta-analyses were conducted by pooling the study-specific hazard ratios (HRs) using random-effects modeling. Extent of heterogeneity between studies was determined with the I2 test. Results Five studies (975 total patients, and 855 patients with individual patient-level data) were eligible for analysis, with a median follow-up of 8 years. Of the total cohort, 60.9%, 22.6%, and 16.5% of patients were classified by Decipher as low, intermediate, and high risk, respectively. The 10-year cumulative incidence metastases rates were 5.5%, 15.0%, and 26.7% ( P < .001), respectively, for the three risk classifications. Pooling the study-specific Decipher HRs across the five studies resulted in an HR of 1.52 (95% CI, 1.39 to 1.67; I2 = 0%) per 0.1 unit. In multivariable analysis of individual patient data, adjusting for clinicopathologic variables, Decipher remained a statistically significant predictor of metastasis (HR, 1.30; 95% CI, 1.14 to 1.47; P < .001) per 0.1 unit. The C-index for 10-year distant metastasis of the clinical model alone was 0.76; this increased to 0.81 with inclusion of Decipher. Conclusion The genomic classifier test, Decipher, can independently improve prognostication of patients postprostatectomy, as well as within nearly all clinicopathologic, demographic, and treatment subgroups. Future study of how to best incorporate genomic testing in clinical decision-making and subsequent treatment recommendations is warranted