Harmonization guidelines for HLA-peptide multimer assays derived from results of a large scale international proficiency panel of the Cancer Vaccine Consortium

Purpose: The Cancer Vaccine Consortium of the Cancer Research Institute (CVC-CRI) conducted a multicenter HLA-peptide multimer proficiency panel (MPP) with a group of 27 laboratories to assess the performance of the assay. Experimental design: Participants used commercially available HLA-peptide multimers and a well characterized common source of peripheral blood mononuclear cells (PBMC). The frequency of CD8+ T cells specific for two HLA-A2-restricted model antigens was measured by flow cytometry. The panel design allowed for participants to use their preferred staining reagents and locally established protocols for both cell labeling, data acquisition and analysis. Results: We observed significant differences in both the performance characteristics of the assay and the reported frequencies of specific T cells across laboratories. These results emphasize the need to identify the critical variables important for the observed variability to allow for harmonization of the technique across institutions. Conclusions: Three key recommendations emerged that would likely reduce assay variability and thus move toward harmonizing of this assay. (1) Use of more than two colors for the staining (2) collect at least 100,000 CD8 T cells, and (3) use of a background control sample to appropriately set the analytical gates. We also provide more insight into the limitations of the assay and identified additional protocol steps that potentially impact the quality of data generated and therefore should serve as primary targets for systematic analysis in future panels. Finally, we propose initial guidelines for harmonizing assay performance which include the introduction of standard operating protocols to allow for adequate training of technical staff and auditing of test analysis procedure

A critical assessment for the value of markers to gate-out undesired events in HLA-peptide multimer staining protocols

Background: The introduction of antibody markers to identify undesired cell populations in flow-cytometry based assays, so called DUMP channel markers, has become a practice in an increasing number of labs performing HLA-peptide multimer assays. However, the impact of the introduction of a DUMP channel in multimer assays has so far not been systematically investigated across a broad variety of protocols. Methods: The Cancer Research Institute%27s Cancer Immunotherapy Consortium (CRI-CIC) conducted a multimer proficiency panel with a specific focus on the impact of DUMP channel use. The panel design allowed individual laboratories to use their own protocol for thawing, staining, gating, and data analysis. Each experiment was performed twice and in parallel, with and without the application of a dump channel strategy. Results: The introduction of a DUMP channel is an effective measure to reduce the amount of non-specific MULTIMER binding to T cells. Beneficial effects for the use of a DUMP channel were observed across a wide range of individual laboratories and for all tested donor-antigen combinations. In 48% of experiments we observed a reduction of the background MULTIMER-binding. In this subgroup of experiments the median background reduction observed after introduction of a DUMP channel was 0.053%. Conclusions: We conclude that appropriate use of a DUMP channel can significantly reduce background staining across a large fraction of protocols and improve the ability to accurately detect and quantify the frequency of antigen-specific T cells by multimer reagents. Thus, use of a DUMP channel may become crucial for detecting low frequency antigen-specific immune responses. Further recommendations on assay performance and data presentation guidelines for publication of MULTIMER experimental data are provided

Harmonization guidelines for HLA-peptide multimer assays derived from results of a large scale international proficiency panel of the Cancer Vaccine Consortium

PURPOSE: The Cancer Vaccine Consortium of the Cancer Research Institute (CVC-CRI) conducted a multicenter HLA-peptide multimer proficiency panel (MPP) with a group of 27 laboratories to assess the performance of the assay. EXPERIMENTAL DESIGN: Participants used commercially available HLA-peptide multimers and a well characterized common source of peripheral blood mononuclear cells (PBMC). The frequency of CD8+ T cells specific for two HLA-A2-restricted model antigens was measured by flow cytometry. The panel design allowed for participants to use their preferred staining reagents and locally established protocols for both cell labeling, data acquisition and analysis. RESULTS: We observed significant differences in both the performance characteristics of the assay and the reported frequencies of specific T cells across laboratories. These results emphasize the need to identify the critical variables important for the observed variability to allow for harmonization of the technique across institutions. CONCLUSIONS: Three key recommendations emerged that would likely reduce assay variability and thus move toward harmonizing of this assay. (1) Use of more than two colors for the staining (2) collect at least 100,000 CD8 T cells, and (3) use of a background control sample to appropriately set the analytical gates. We also provide more insight into the limitations of the assay and identified additional protocol steps that potentially impact the quality of data generated and therefore should serve as primary targets for systematic analysis in future panels. Finally, we propose initial guidelines for harmonizing assay performance which include the introduction of standard operating protocols to allow for adequate training of technical staff and auditing of test analysis procedures

Results and harmonization guidelines from two large-scale international Elispot proficiency panels conducted by the Cancer Vaccine Consortium (CVC/SVI)

The Cancer Vaccine Consortium of the Sabin Vaccine Institute (CVC/SVI) is conducting an ongoing large-scale immune monitoring harmonization program through its members and affiliated associations. This effort was brought to life as an external validation program by conducting an international Elispot proficiency panel with 36 laboratories in 2005, and was followed by a second panel with 29 participating laboratories in 2006 allowing for application of learnings from the first panel. Critical protocol choices, as well as standardization and validation practices among laboratories were assessed through detailed surveys. Although panel participants had to follow general guidelines in order to allow comparison of results, each laboratory was able to use its own protocols, materials and reagents. The second panel recorded an overall significantly improved performance, as measured by the ability to detect all predefined responses correctly. Protocol choices and laboratory practices, which can have a dramatic effect on the overall assay outcome, were identified and lead to the following recommendations: (A) Establish a laboratory SOP for Elispot testing procedures including (A1) a counting method for apoptotic cells for determining adequate cell dilution for plating, and (A2) overnight rest of cells prior to plating and incubation, (B) Use only pre-tested serum optimized for low background: high signal ratio, (C) Establish a laboratory SOP for plate reading including (C1) human auditing during the reading process and (C2) adequate adjustments for technical artifacts, and (D) Only allow trained personnel, which is certified per laboratory SOPs to conduct assays. Recommendations described under (A) were found to make a statistically significant difference in assay performance, while the remaining recommendations are based on practical experiences confirmed by the panel results, which could not be statistically tested. These results provide initial harmonization guidelines to optimize Elispot assay performance to the immunotherapy community. Further optimization is in process with ongoing panels

Defining the critical hurdles in cancer immunotherapy

Scientific discoveries that provide strong evidence of antitumor effects in preclinical models often encounter significant delays before being tested in patients with cancer. While some of these delays have a scientific basis, others do not. We need to do better. Innovative strategies need to move into early stage clinical trials as quickly as it is safe, and if successful, these therapies should efficiently obtain regulatory approval and widespread clinical application. In late 2009 and 2010 the Society for Immunotherapy of Cancer (SITC), convened an "Immunotherapy Summit" with representatives from immunotherapy organizations representing Europe, Japan, China and North America to discuss collaborations to improve development and delivery of cancer immunotherapy. One of the concepts raised by SITC and defined as critical by all parties was the need to identify hurdles that impede effective translation of cancer immunotherapy. With consensus on these hurdles, international working groups could be developed to make recommendations vetted by the participating organizations. These recommendations could then be considered by regulatory bodies, governmental and private funding agencies, pharmaceutical companies and academic institutions to facilitate changes necessary to accelerate clinical translation of novel immune-based cancer therapies. The critical hurdles identified by representatives of the collaborating organizations, now organized as the World Immunotherapy Council, are presented and discussed in this report. Some of the identified hurdles impede all investigators; others hinder investigators only in certain regions or institutions or are more relevant to specific types of immunotherapy or first-in-humans studies. Each of these hurdles can significantly delay clinical translation of promising advances in immunotherapy yet if overcome, have the potential to improve outcomes of patients with cancer

Defining the Critical Hurdles in Cancer Immunotherapy

ABSTRACT: Scientific discoveries that provide strong evidence of antitumor effects in preclinical models often encounter significant delays before being tested in patients with cancer. While some of these delays have a scientific basis, others do not. We need to do better. Innovative strategies need to move into early stage clinical trials as quickly as it is safe, and if successful, these therapies should efficiently obtain regulatory approval and widespread clinical application. In late 2009 and 2010 the Society for Immunotherapy of Cancer (SITC), convened an "Immunotherapy Summit" with representatives from immunotherapy organizations representing Europe, Japan, China and North America to discuss collaborations to improve development and delivery of cancer immunotherapy. One of the concepts raised by SITC and defined as critical by all parties was the need to identify hurdles that impede effective translation of cancer immunotherapy. With consensus on these hurdles, international working groups could be developed to make recommendations vetted by the participating organizations. These recommendations could then be considered by regulatory bodies, governmental and private funding agencies, pharmaceutical companies and academic institutions to facilitate changes necessary to accelerate clinical translation of novel immune-based cancer therapies. The critical hurdles identified by representatives of the collaborating organizations, now organized as the World Immunotherapy Council, are presented and discussed in this report. Some of the identified hurdles impede all investigators, others hinder investigators only in certain regions or institutions or are more relevant to specific types of immunotherapy or first-in-humans studies. Each of these hurdles can significantly delay clinical translation of promising advances in immunotherapy yet be overcome to improve outcomes of patients with cancer

Immunological Monitoring to Rationally Guide AAV Gene Therapy

Recent successes with adeno-associated virus (AAV)-based gene therapies fuel the hope for new treatments for hereditary diseases. Pre-existing as well as therapy-induced immune responses against both AAV and the encoded transgenes have been described and may impact on safety and efficacy of gene-therapy approaches. Consequently, monitoring of vector- and transgene-specific immunity is mandated and may rationally guide clinical development. Next to the humoral immune response, the cellular response is central in our understanding of the host reaction in gene therapy. But in contrast to the monitoring of antibodies, which has matured over many decades, sensitive and robust monitoring of T cells is a relatively new development. To make cellular immune assessments fit for purpose, investigators need to know, control and report the critical assay variables that influence the results. In addition, the quality of immune assays needs to be continuously adjusted to allow for exploratory hypothesis generation in early stages and confirmatory hypothesis validation in later stages of clinical development. The concept of immune assay harmonization which includes use of field-wide benchmarks, harmonization guidelines, and external quality control can support the context-specific evolution of immune assays. Multi-center studies pose particular challenges to sample logistics and quality control of sample specimens. Cooperative groups need to define if immune assessments should be performed in one central facility, in peripheral labs or including a combination of both. Finally, engineered reference samples that contain a defined number of antigen-specific T cells may become broadly applicable tools to control assay performance over time or across institutions

"MIATA"-minimal information about T cell assays.

Immunotherapy, especially therapeutic vaccination, has a great deal of potential in the treatment of cancer and certain infectious diseases such as HIV (Allison et al., 2006; Fauci et al., 2008; Feldmann and Steinman, 2005). Numerous vaccine candidates have been tested in patients with a variety of tumor types and chronic viral diseases. Often, the best way to assess the clinical potential of these vaccines is to monitor the induced T cell response, and yet there are currently no standards for reporting these results. This letter is an effort to address this problem

Confidence-based Somatic Mutation Evaluation and Prioritization

Next generation sequencing (NGS) has enabled high throughput discovery of somatic mutations. Detection depends on experimental design, lab platforms, parameters and analysis algorithms. However, NGS-based somatic mutation detection is prone to erroneous calls, with reported validation rates near 54% and congruence between algorithms less than 50%. Here, we developed an algorithm to assign a single statistic, a false discovery rate (FDR), to each somatic mutation identified by NGS. This FDR confidence value accurately discriminates true mutations from erroneous calls. Using sequencing data generated from triplicate exome profiling of C57BL/6 mice and B16-F10 melanoma cells, we used the existing algorithms GATK, SAMtools and SomaticSNiPer to identify somatic mutations. For each identified mutation, our algorithm assigned an FDR. We selected 139 mutations for validation, including 50 somatic mutations assigned a low FDR (high confidence) and 44 mutations assigned a high FDR (low confidence). All of the high confidence somatic mutations validated (50 of 50), none of the 44 low confidence somatic mutations validated, and 15 of 45 mutations with an intermediate FDR validated. Furthermore, the assignment of a single FDR to individual mutations enables statistical comparisons of lab and computation methodologies, including ROC curves and AUC metrics. Using the HiSeq 2000, single end 50 nt reads from replicates generate the highest confidence somatic mutation call set

A critical assessment for the value of markers to gate-out undesired events in HLA-peptide multimer staining protocols

Abstract Background The introduction of antibody markers to identify undesired cell populations in flow-cytometry based assays, so called DUMP channel markers, has become a practice in an increasing number of labs performing HLA-peptide multimer assays. However, the impact of the introduction of a DUMP channel in multimer assays has so far not been systematically investigated across a broad variety of protocols. Methods The Cancer Research Institute's Cancer Immunotherapy Consortium (CRI-CIC) conducted a multimer proficiency panel with a specific focus on the impact of DUMP channel use. The panel design allowed individual laboratories to use their own protocol for thawing, staining, gating, and data analysis. Each experiment was performed twice and in parallel, with and without the application of a dump channel strategy. Results The introduction of a DUMP channel is an effective measure to reduce the amount of non-specific MULTIMER binding to T cells. Beneficial effects for the use of a DUMP channel were observed across a wide range of individual laboratories and for all tested donor-antigen combinations. In 48% of experiments we observed a reduction of the background MULTIMER-binding. In this subgroup of experiments the median background reduction observed after introduction of a DUMP channel was 0.053%. Conclusions We conclude that appropriate use of a DUMP channel can significantly reduce background staining across a large fraction of protocols and improve the ability to accurately detect and quantify the frequency of antigen-specific T cells by multimer reagents. Thus, use of a DUMP channel may become crucial for detecting low frequency antigen-specific immune responses. Further recommendations on assay performance and data presentation guidelines for publication of MULTIMER experimental data are provided.</p