ViZDoom: DRQN with Prioritized Experience Replay, Double-Q Learning, & Snapshot Ensembling

ViZDoom is a robust, first-person shooter reinforcement learning environment, characterized by a significant degree of latent state information. In this paper, double-Q learning and prioritized experience replay methods are tested under a certain ViZDoom combat scenario using a competitive deep recurrent Q-network (DRQN) architecture. In addition, an ensembling technique known as snapshot ensembling is employed using a specific annealed learning rate to observe differences in ensembling efficacy under these two methods. Annealed learning rates are important in general to the training of deep neural network models, as they shake up the status-quo and counter a model's tending towards local optima. While both variants show performance exceeding those of built-in AI agents of the game, the known stabilizing effects of double-Q learning are illustrated, and priority experience replay is again validated in its usefulness by showing immediate results early on in agent development, with the caveat that value overestimation is accelerated in this case. In addition, some unique behaviors are observed to develop for priority experience replay (PER) and double-Q (DDQ) variants, and snapshot ensembling of both PER and DDQ proves a valuable method for improving performance of the ViZDoom Marine.Comment: 9 pages, 5 figure

Automated pattern-guided principal component analysis vs expert-based immunophenotypic classification of B-cell chronic lymphoproliferative disorders: a step forward in the standardization of clinical immunophenotyping

Immunophenotypic characterization of B-cell chronic lymphoproliferative disorders (B-CLPD) is becoming increasingly complex due to usage of progressively larger panels of reagents and a high number of World Health Organization (WHO) entities. Typically, data analysis is performed separately for each stained aliquot of a sample; subsequently, an expert interprets the overall immunophenotypic profile (IP) of neoplastic B-cells and assigns it to specific diagnostic categories. We constructed a principal component analysis (PCA)-based tool to guide immunophenotypic classification of B-CLPD. Three reference groups of immunophenotypic data files—B-cell chronic lymphocytic leukemias (B-CLL; n=10), mantle cell (MCL; n=10) and follicular lymphomas (FL; n=10)—were built. Subsequently, each of the 175 cases studied was evaluated and assigned to either one of the three reference groups or to none of them (other B-CLPD). Most cases (89%) were correctly assigned to their corresponding WHO diagnostic group with overall positive and negative predictive values of 89 and 96%, respectively. The efficiency of the PCA-based approach was particularly high among typical B-CLL, MCL and FL vs other B-CLPD cases. In summary, PCA-guided immunophenotypic classification of B-CLPD is a promising tool for standardized interpretation of tumor IP, their classification into well-defined entities and comprehensive evaluation of antibody panels

Optimal number of reagents required to evaluate hematolymphoid neoplasias. Results of an International Consensus meeting

At the ISAC 2000 Congress, the Clinical Cytometry Society organized a meeting of international experts to reach consensus on the minimum number of antibodies required for a full evaluation of hematologic and lymphoid neoplasias. A questionnaire was distributed prior to the meeting to numerous experts from US and European institutions and 13 responses were received. At the meeting, 25 individuals, including most of those who returned responses, participated in the discussions and voted on the issues presented. In chronic lymphoproliferative disorders (CLD), 9 antibodies (anti-CD5, CD19, kappa, lambda, CD3, CD20, CD23, CD10, and CD45) were deemed essential for initial evaluation by 75% of the participants. There was near unanimity that additional markers (selected from CD22, FMC7, CD11c, CD103, CD38, CD25, CD79b and heavy chains for B-cell disorders, and CD4, CD7, CD8, CD2, CD56, CD16, TCRa/b, and TCRg/d for T-cell disorders) would be needed to fully characterize CLD, although not every marker would be useful in all cases. Tissue lymphomas were believed to be similar to CLD, needing a minimum of 12--16 markers. However, for some cases, CD30, bcl-2, TdT, CD71, CD1a, and CD34 were cited as useful by the participants. Markers mentioned for plasma cell disorders included kappa, lambda, CD38, CD45, CD56, CD19, CD20, CD138, and heavy chains. Of 17 voting participants, 16 agreed that between 5 to 8 markers would be essential reagents for plasma cell disorders. For acute leukemia (AL), 10 markers (CD10, CD19, CD13, CD33, CD34, CD45, CD7, CD14, CD3, and HLADR) were considered essential by 75% of participants for initial characterization of the leukemia lineage. Most (>75%) agreed that at least one more B (CD20, CD22, CD79a, IgM), T (CD1a, CD2, CD4, CD5, CD8), myeloid (CD11b, CD15, CD64, CD117, myeloperoxidase), erythroid (CD36, CD71, glycophorin A), and megakaryocytic (CD41, CD61) reagents should be included in the essential panel. However, there was no agreement as to which was optimal. Thus, approximately 13--15 of those reagents would be considered essential in all cases of AL, whereas others (CD16, CD56, CDw65, TdT, and cytoplasmic CD3) were mentioned as useful in some cases. Almost all voting participants believed that the appropriate number of markers for complete characterization of AL would average 20--24. The majority of the responders (11 of 13) indicated that fewer reagents could be used in monitoring or staging patients with previously characterized disease, but not all ventured a specific number of reagents. From the above results, we conclude that the phenotypic analysis of hematologic and lymphoid neoplasia requires a rather extensive panel of reagents. Supplementary reagents might even be necessary if they prove to become relevant for diagnostic purposes. Reducing the number of antibodies could significantly compromise the diagnostic accuracy, appropriate monitoring, or therapy of these disorders

Clinically useful information provided by the flow cytometric immunophenotyping of hematological malignancies: current status and future directions.

BACKGROUND: At present, immunophenotyping of hematological malignancies represents one of the most relevant clinical applications of flow cytometry. In recent years, its use has extended from clinical research to diagnostic laboratories. The aim of this report is to critically review the type of information provided by the flow cytometric immunophenotyping of hematological malignancies and its clinical impact as well as to highlight its potential future applications. METHODS: The currently available information, including that provided by different international consensus groups on the phenotypic characterization of hematologic malignancies, was reviewed. Additionally, recent reports on the immunophenotypic analysis of hematological malignancies published in hematology, oncology, pathology, immunology, and cell biology journals were also analyzed. RESULTS: A careful review of the literature showed that in spite of the well-established utility of immunophenotyping for the diagnosis, classification, prognostic stratification, and monitoring of hematological malignancies, only a small part of the information on the immunophenotypic characteristics of pathological hemopoietic cells has been used routinely. Specific and sensitive identification of neoplastic cells and their accurate enumeration and phenotypic characterization represent the major aims of these procedures. Similarities between leukemic and healthy cells allow the establishment of the lineage and maturation stage of the pathologic cells, this information being of great utility for the diagnosis, classification, and prognostic evaluation of different subtypes of hematological malignancies. On the other hand, the phenotypic aberrations displayed by leukemic cells could allow the selection of cases carrying specific genetic abnormalities in which further confirmatory molecular studies will be performed. CONCLUSIONS: The information provided by the flow cytometric immunophenotyping of hematological malignancies is of great clinical utility, with a major challenge for the near future being the standardization of technical procedures, data interpretation, and reporting

Clinically useful information provided by the flow cytometric immunophenotyping of hematological malignancies: current status and future directions

Background: At present, immunophenotyping of hematological malignancies represents one of the most relevant clinical applications of flow cytometry. In recent years, its use has extended from clinical research to diagnostic laboratories. The aim of this report is to critically review the type of information provided by the flow cytometric immunophenotyping of hematological malignancies and its clinical impact as well as to highlight its potential future applications. Methods: The currently available information, including that provided by different international consensus groups on the phenotypic characterization of hematologic malignancies, was reviewed. Additionally, recent reports on the immunophenotypic analysis of hematological malignancies published in hematology, oncology, pathology, immunology, and cell biology journals were also analyzed. Results: A careful review of the literature showed that in spite of the well-established utility of immunophenotyping for the diagnosis, classification, prognostic stratification, and monitoring of hematological malignancies, only a small part of the information on the immunophenotypic characteristics of pathological hemopoietic cells has been used routinely. Specific and sensitive identification of neoplastic cells and their accurate enumeration and phenotypic characterization represent the major aims of these procedures. Similarities between leukemic and healthy cells allow the establishment of the lineage and maturation stage of the pathologic cells, this information being of great utility for the diagnosis, classification, and prognostic evaluation of different subtypes of hematological malignancies. On the other hand, the phenotypic aberrations displayed by leukemic cells could allow the selection of cases carrying specific genetic abnormalities in which further confirmatory molecular studies will be performed. Conclusions: The information provided by the flow cytometric immunophenotyping of hematological malignancies is of great clinical utility, with a major challenge for the near future being the standardization of technical procedures, data interpretation, and reporting