From the Trenches: A Cross-Sectional Study Applying the GRADE Tool in Systematic Reviews of Healthcare Interventions

Background: GRADE was developed to address shortcomings of tools to rate the quality of a body of evidence. While much has been published about GRADE, there are few empirical and systematic evaluations. Objective: To assess GRADE for systematic reviews (SRs) in terms of inter-rater agreement and identify areas of uncertainty. Design: Cross-sectional, descriptive study. Methods: We applied GRADE to three SRs (n = 48, 66, and 75 studies, respectively) with 29 comparisons and 12 outcomes overall. Two reviewers graded evidence independently for outcomes deemed clinically important a priori. Inter-rater reliability was assessed using kappas for four main domains (risk of bias, consistency, directness, and precision) and overall quality of evidence. Results: For the first review, reliability was: k = 0.41 for risk of bias; 0.84 consistency; 0.18 precision; and 0.44 overall quality. Kappa could not be calculated for directness as one rater assessed all items as direct; assessors agreed in 41 % of cases. For the second review reliability was: 0.37 consistency and 0.19 precision. Kappa could not be assessed for other items; assessors agreed in 33 % of cases for risk of bias; 100 % directness; and 58 % overall quality. For the third review, reliability was: 0.06 risk of bias; 0.79 consistency; 0.21 precision; and 0.18 overall quality. Assessors agreed in 100 % of cases for directness. Precision created the most uncertainty due to difficulties in identifying ‘‘optimal’ ’ information size and ‘‘clinica

Diabetes and Clinical Outcome in Patients With Metastatic Colorectal Cancer: CALGB 80405 (Alliance)

Background Diabetes is a prognostic factor for some malignancies, but its association with outcome in patients with advanced or metastatic colorectal cancer (CRC) is less clear. Methods This cohort study was nested within a randomized trial of first-line chemotherapy and bevacizumab and/or cetuximab for advanced or metastatic CRC. Patients were enrolled at 508 community and academic centers throughout the National Clinical Trials Network. The primary exposure was physician-documented diabetes at the time of enrollment. The primary endpoint was overall survival (OS); secondary endpoints were progression-free survival (PFS) and adverse events. Tests of statistical significance were two-sided. Results Among 2326 patients, 378 (16.3%) had diabetes. The median follow-up time was 6.0 years. We observed 1973 OS events and 2173 PFS events. The median time to an OS event was 22.7 months among those with diabetes and 27.1 months among those without diabetes (HR = 1.27, 95% CI = 1.13 to 1.44; P < .001). The median time to a PFS event was 9.7 months among those with diabetes and 10.8 months among those without diabetes (HR = 1.16, 95% CI = 1.03 to 1.30; P = .02). Patients with diabetes were more likely to experience no less than grade 3 hypertension (8.1% vs 4.4%; P = .054) but were not more likely to experience other adverse events, including neuropathy. Conclusions Diabetes is associated with an increased risk of mortality and tumor progression in patients with advanced or metastatic CRC. Patients with diabetes tolerate first-line treatment with chemotherapy and monoclonal antibodies similarly to patients without diabetes

Post-Acquisition Processing Confounds in Brain Volumetric Quantification of White Matter Hyperintensities

BACKGROUND: Disparate research sites using identical or near-identical magnetic resonance imaging (MRI) acquisition techniques often produce results that demonstrate significant variability regarding volumetric quantification of white matter hyperintensities (WMH) in the aging population. The sources of such variability have not previously been fully explored. NEW METHOD: 3D FLAIR sequences from a group of randomly selected aged subjects were analyzed to identify sources-of-variability in post-acquisition processing that can be problematic when comparing WMH volumetric data across disparate sites. The methods developed focused on standardizing post-acquisition protocol processing methods to develop a protocol with less than 0.5% inter-rater variance. RESULTS: A series of experiments using standard MRI acquisition sequences explored post-acquisition sources-of-variability in the quantification of WMH volumetric data. Sources-of-variability included: the choice of image center, software suite and version, thresholding selection, and manual editing procedures (when used). Controlling for the identified sources-of-variability led to a protocol with less than 0.5% variability between independent raters in post-acquisition WMH volumetric quantification. COMPARISON WITH EXISTING METHOD(S): Post-acquisition processing techniques can introduce an average variance approaching 15% in WMH volume quantification despite identical scan acquisitions. Understanding and controlling for such sources-of-variability can reduce post-acquisition quantitative image processing variance to less than 0.5%. DISCUSSION: Considerations of potential sources-of-variability in MRI volume quantification techniques and reduction in such variability is imperative to allow for reliable cross-site and cross-study comparisons

Effect of First-Line Chemotherapy Combined With Cetuximab or Bevacizumab on Overall Survival in Patients With KRAS Wild-Type Advanced or Metastatic Colorectal Cancer: A Randomized Clinical Trial

Combining biologic monoclonal antibodies with chemotherapeutic cytotoxic drugs provides clinical benefit to patients with advanced or metastatic colorectal cancer, but the optimal choice of the initial biologic therapy in previously untreated patients is unknown

Phase II study of the farnesyltransferase inhibitor R115777 in advanced melanoma (CALGB 500104)

BACKGROUND: Multiple farnesylated proteins are involved in signal transduction in cancer. Farnesyltransferase inhibitors (FTIs) have been developed as a strategy to inhibit the function of these proteins. As FTIs inhibit proliferation of melanoma cell lines, we undertook a study to assess the impact of a FTI in advanced melanoma. As farnesylated proteins are also important for T cell activation, measurement of effects on T cell function was also pursued. METHODS: A 3-stage trial design was developed with a maximum of 40 patients and early stopping if there were no responders in the first 14, or fewer than 2 responders in the first 28 patients. Eligibility included performance status of 0–1, no prior chemotherapy, at most 1 prior immunotherapy, no brain metastases, and presence of at least 2 cutaneous lesions amenable to biopsy. R115777 was administered twice per day for 21 days of a 28-day cycle. Patients were evaluated every 2 cycles by RECIST. Blood and tumor were analyzed pre-treatment and during week 7. RESULTS: Fourteen patients were enrolled. Two patients had grade 3 toxicities, which included myelosuppression, nausea/vomiting, elevated BUN, and anorexia. There were no clinical responses. All patients analyzed showed potent inhibition of FT activity (85-98%) in tumor tissue; inhibition of phosphorylated ERK and Akt was also observed. T cells showed evidence of FT inhibition and diminished IFN-γ production. CONCLUSIONS: Despite potent target inhibition, R115777 showed no evidence of clinical activity in this cohort of melanoma patients. Inhibition of T cell function by FTIs has potential clinical implications. Clinicaltrials.gov number NCT0006012

Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada

<p>Abstract</p> <p>Background</p> <p>Consistent healthcare decisionmaking requires systematic consideration of decision criteria and evidence available to inform them. This can be tackled by combining multicriteria decision analysis (MCDA) and Health Technology Assessment (HTA). The objective of this study was to field-test a decision support framework (EVIDEM), explore its utility to a drug advisory committee and test its reliability over time.</p> <p>Methods</p> <p>Tramadol for chronic non-cancer pain was selected by the health plan as a case study relevant to their context. Based on extensive literature review, a by-criterion HTA report was developed to provide synthesized evidence for each criterion of the framework (14 criteria for the MCDA Core Model and 6 qualitative criteria for the Contextual Tool). During workshop sessions, committee members tested the framework in three steps by assigning: 1) weights to each criterion of the MCDA Core Model representing individual perspective; 2) scores for tramadol for each criterion of the MCDA Core Model using synthesized data; and 3) qualitative impacts of criteria of the Contextual Tool on the appraisal. Utility and reliability of the approach were explored through discussion, survey and test-retest. Agreement between test and retest data was analyzed by calculating intra-rater correlation coefficients (ICCs) for weights, scores and MCDA value estimates.</p> <p>Results</p> <p>The framework was found useful by the drug advisory committee in supporting systematic consideration of a broad range of criteria to promote a consistent approach to appraising healthcare interventions. Directly integrated in the framework as a "by-criterion" HTA report, synthesized evidence for each criterion facilitated its consideration, although this was sometimes limited by lack of relevant data. Test-retest analysis showed fair to good consistency of weights, scores and MCDA value estimates at the individual level (ICC ranging from 0.676 to 0.698), thus lending some support for the reliability of the approach. Overall, committee members endorsed the inclusion of most framework criteria and revealed important areas of discussion, clarification and adaptation of the framework to the needs of the committee.</p> <p>Conclusions</p> <p>By promoting systematic consideration of all decision criteria and the underlying evidence, the framework allows a consistent approach to appraising healthcare interventions. Further testing and validation are needed to advance MCDA approaches in healthcare decisionmaking.</p

Documenting the Natural History of Patients With Resected Stage II Adenocarcinoma of the Colon After Random Assignment to Adjuvant Treatment With Edrecolomab or Observation: Results From CALGB 9581

We conducted a randomized trial comparing adjuvant treatment with edrecolomab versus observation in patients with resected, low-risk, stage II colon cancer. This study also prospectively studied patient- and tumor-specific markers of treatment outcome

IGF-Binding Proteins, Adiponectin, and Survival in Metastatic Colorectal Cancer: Results From CALGB (Alliance)/SWOG 80405

Background: Energy balance-related biomarkers are associated with risk and prognosis of various malignancies. Their relationship to survival in metastatic colorectal cancer (mCRC) requires further study. Methods: Baseline plasma insulin-like growth factor (IGF)-1, IGF-binding protein (IGFBP)-3, IGFBP-7, C-peptide, and adiponectin were measured at time of trial registration in a prospective cohort of patients with mCRC participating in a National Cancer Institute-sponsored trial of first-line systemic therapy. We used Cox proportional hazards regression to adjust for confounders and examine associations of each biomarker with overall survival (OS) and progression-free survival (PFS). P values are 2-sided. Results: Median follow-up for 1086 patients was 6.2 years. Compared with patients in the lowest IGFBP-3 quintile, patients in the highest IGFBP-3 quintile experienced an adjusted hazard ratio (HR) for OS of 0.57 (95% confidence interval [CI] = 0.42 to 0.78; P nonlinearity < .001) and for PFS of 0.61 (95% CI = 0.45 to 0.82; P trend = .003). Compared with patients in the lowest IGFBP-7 quintile, patients in the highest IGFBP-7 quintile experienced an adjusted hazard ratio for OS of 1.60 (95% CI = 1.30 to 1.97; P trend < .001) and for PFS of 1.38 (95% CI = 1.13 to 1.69; P trend < .001). Plasma C-peptide and IGF-1 were not associated with patient outcomes. Adiponectin was not associated with OS; there was a nonlinear U-shaped association between adiponectin and PFS (P nonlinearity = .03). Conclusions: Among patients with mCRC, high plasma IGFBP-3 and low IGFBP-7 were associated with longer OS and PFS. Extreme levels of adiponectin were associated with shorter PFS. These findings suggest potential avenues for prognostic and therapeutic innovation

The genetic architecture of the human cerebral cortex

The cerebral cortex underlies our complex cognitive capabilities, yet little is known about the specific genetic loci that influence human cortical structure. To identify genetic variants that affect cortical structure, we conducted a genome-wide association meta-analysis of brain magnetic resonance imaging data from 51,665 individuals. We analyzed the surface area and average thickness of the whole cortex and 34 regions with known functional specializations. We identified 199 significant loci and found significant enrichment for loci influencing total surface area within regulatory elements that are active during prenatal cortical development, supporting the radial unit hypothesis. Loci that affect regional surface area cluster near genes in Wnt signaling pathways, which influence progenitor expansion and areal identity. Variation in cortical structure is genetically correlated with cognitive function, Parkinson's disease, insomnia, depression, neuroticism, and attention deficit hyperactivity disorder

Regulatory T Cell Responses in Participants with Type 1 Diabetes after a Single Dose of Interleukin-2: A Non-Randomised, Open Label, Adaptive Dose-Finding Trial

BACKGROUND: Interleukin-2 (IL-2) has an essential role in the expansion and function of CD4+ regulatory T cells (Tregs). Tregs reduce tissue damage by limiting the immune response following infection and regulate autoreactive CD4+ effector T cells (Teffs) to prevent autoimmune diseases, such as type 1 diabetes (T1D). Genetic susceptibility to T1D causes alterations in the IL-2 pathway, a finding that supports Tregs as a cellular therapeutic target. Aldesleukin (Proleukin; recombinant human IL-2), which is administered at high doses to activate the immune system in cancer immunotherapy, is now being repositioned to treat inflammatory and autoimmune disorders at lower doses by targeting Tregs. METHODS AND FINDINGS: To define the aldesleukin dose response for Tregs and to find doses that increase Tregs physiologically for treatment of T1D, a statistical and systematic approach was taken by analysing the pharmacokinetics and pharmacodynamics of single doses of subcutaneous aldesleukin in the Adaptive Study of IL-2 Dose on Regulatory T Cells in Type 1 Diabetes (DILT1D), a single centre, non-randomised, open label, adaptive dose-finding trial with 40 adult participants with recently diagnosed T1D. The primary endpoint was the maximum percentage increase in Tregs (defined as CD3+CD4+CD25highCD127low) from the baseline frequency in each participant measured over the 7 d following treatment. There was an initial learning phase with five pairs of participants, each pair receiving one of five pre-assigned single doses from 0.04 × 106 to 1.5 × 106 IU/m2, in order to model the dose-response curve. Results from each participant were then incorporated into interim statistical modelling to target the two doses most likely to induce 10% and 20% increases in Treg frequencies. Primary analysis of the evaluable population (n = 39) found that the optimal doses of aldesleukin to induce 10% and 20% increases in Tregs were 0.101 × 106 IU/m2 (standard error [SE] = 0.078, 95% CI = -0.052, 0.254) and 0.497 × 106 IU/m2 (SE = 0.092, 95% CI = 0.316, 0.678), respectively. On analysis of secondary outcomes, using a highly sensitive IL-2 assay, the observed plasma concentrations of the drug at 90 min exceeded the hypothetical Treg-specific therapeutic window determined in vitro (0.015-0.24 IU/ml), even at the lowest doses (0.040 × 106 and 0.045 × 106 IU/m2) administered. A rapid decrease in Treg frequency in the circulation was observed at 90 min and at day 1, which was dose dependent (mean decrease 11.6%, SE = 2.3%, range 10.0%-48.2%, n = 37), rebounding at day 2 and increasing to frequencies above baseline over 7 d. Teffs, natural killer cells, and eosinophils also responded, with their frequencies rapidly and dose-dependently decreased in the blood, then returning to, or exceeding, pretreatment levels. Furthermore, there was a dose-dependent down modulation of one of the two signalling subunits of the IL-2 receptor, the β chain (CD122) (mean decrease = 58.0%, SE = 2.8%, range 9.8%-85.5%, n = 33), on Tregs and a reduction in their sensitivity to aldesleukin at 90 min and day 1 and 2 post-treatment. Due to blood volume requirements as well as ethical and practical considerations, the study was limited to adults and to analysis of peripheral blood only. CONCLUSIONS: The DILT1D trial results, most notably the early altered trafficking and desensitisation of Tregs induced by a single ultra-low dose of aldesleukin that resolves within 2-3 d, inform the design of the next trial to determine a repeat dosing regimen aimed at establishing a steady-state Treg frequency increase of 20%-50%, with the eventual goal of preventing T1D. TRIAL REGISTRATION: ISRCTN Registry ISRCTN27852285; ClinicalTrials.gov NCT01827735.This is the final version of the article. It first appeared from the Public Library of Science via http://dx.doi.org/10.1371/journal.pmed.100213