Differently stained whole slide image registration technique with landmark validation

Abstract. One of the most significant features in digital pathology is to compare and fuse successive differently stained tissue sections, also called slides, visually. Doing so, aligning different images to a common frame, ground truth, is required. Current sample scanning tools enable to create images full of informative layers of digitalized tissues, stored with a high resolution into whole slide images. However, there are a limited amount of automatic alignment tools handling large images precisely in acceptable processing time. The idea of this study is to propose a deep learning solution for histopathology image registration. The main focus is on the understanding of landmark validation and the impact of stain augmentation on differently stained histopathology images. Also, the developed registration method is compared with the state-of-the-art algorithms which utilize whole slide images in the field of digital pathology. There are previous studies about histopathology, digital pathology, whole slide imaging and image registration, color staining, data augmentation, and deep learning that are referenced in this study. The goal is to develop a learning-based registration framework specifically for high-resolution histopathology image registration. Different whole slide tissue sample images are used with a resolution of up to 40x magnification. The images are organized into sets of consecutive, differently dyed sections, and the aim is to register the images based on only the visible tissue and ignore the background. Significant structures in the tissue are marked with landmarks. The quality measurements include, for example, the relative target registration error, structural similarity index metric, visual evaluation, landmark-based evaluation, matching points, and image details. These results are comparable and can be used also in the future research and in development of new tools. Moreover, the results are expected to show how the theory and practice are combined in whole slide image registration challenges. DeepHistReg algorithm will be studied to better understand the development of stain color feature augmentation-based image registration tool of this study. Matlab and Aperio ImageScope are the tools to annotate and validate the image, and Python is used to develop the algorithm of this new registration tool. As cancer is globally a serious disease regardless of age or lifestyle, it is important to find ways to develop the systems experts can use while working with patients’ data. There is still a lot to improve in the field of digital pathology and this study is one step toward it.Eri menetelmin värjättyjen virtuaalinäytelasien rekisteröintitekniikka kiintopisteiden validointia hyödyntäen. Tiivistelmä. Yksi tärkeimmistä digitaalipatologian ominaisuuksista on verrata ja fuusioida peräkkäisiä eri menetelmin värjättyjä kudosleikkeitä toisiinsa visuaalisesti. Tällöin keskenään lähes identtiset kuvat kohdistetaan samaan yhteiseen kehykseen, niin sanottuun pohjatotuuteen. Nykyiset näytteiden skannaustyökalut mahdollistavat sellaisten kuvien luonnin, jotka ovat täynnä kerroksittaista tietoa digitalisoiduista näytteistä, tallennettuna erittäin korkean resoluution virtuaalisiin näytelaseihin. Tällä hetkellä on olemassa kuitenkin vain kourallinen automaattisia työkaluja, jotka kykenevät käsittelemään näin valtavia kuvatiedostoja tarkasti hyväksytyin aikarajoin. Tämän työn tarkoituksena on syväoppimista hyväksikäyttäen löytää ratkaisu histopatologisten kuvien rekisteröintiin. Tärkeimpänä osa-alueena on ymmärtää kiintopisteiden validoinnin periaatteet sekä eri väriaineiden augmentoinnin vaikutus. Lisäksi tässä työssä kehitettyä rekisteröintialgoritmia tullaan vertailemaan muihin kirjallisuudessa esitettyihin algoritmeihin, jotka myös hyödyntävät virtuaalinäytelaseja digitaalipatologian saralla. Kirjallisessa osiossa tullaan siteeraamaan aiempia tutkimuksia muun muassa seuraavista aihealueista: histopatologia, digitaalipatologia, virtuaalinäytelasi, kuvantaminen ja rekisteröinti, näytteen värjäys, data-augmentointi sekä syväoppiminen. Tavoitteena on kehittää oppimispohjainen rekisteröintikehys erityisesti korkearesoluutioisille digitalisoiduille histopatologisille kuville. Erilaisissa näytekuvissa tullaan käyttämään jopa 40-kertaista suurennosta. Kuvat kudoksista on järjestetty eri menetelmin värjättyihin peräkkäisiin kuvasarjoihin ja tämän työn päämääränä on rekisteröidä kuvat pohjautuen ainoastaan kudosten näkyviin osuuksiin, jättäen kuvien tausta huomioimatta. Kudosten merkittävimmät rakenteet on merkattu niin sanotuin kiintopistein. Työn laatumittauksina käytetään arvoja, kuten kohteen suhteellinen rekisteröintivirhe (rTRE), rakenteellisen samankaltaisuuindeksin mittari (SSIM), sekä visuaalista arviointia, kiintopisteisiin pohjautuvaa arviointia, yhteensopivuuskohtia, ja kuvatiedoston yksityiskohtia. Nämä arvot ovat verrattavissa myös tulevissa tutkimuksissa ja samaisia arvoja voidaan käyttää uusia työkaluja kehiteltäessä. DeepHistReg metodi toimii pohjana tässä työssä kehitettävälle näytteen värjäyksen parantamiseen pohjautuvalle rekisteröintityökalulle. Matlab ja Aperio ImageScope ovat ohjelmistoja, joita tullaan hyödyntämään tässä työssä kuvien merkitsemiseen ja validointiin. Ohjelmointikielenä käytetään Pythonia. Syöpä on maailmanlaajuisesti vakava sairaus, joka ei katso ikää eikä elämäntyyliä. Siksi on tärkeää löytää uusia keinoja kehittää työkaluja, joita asiantuntijat voivat hyödyntää jokapäiväisessä työssään potilastietojen käsittelyssä. Digitaalipatologian osa-alueella on vielä paljon innovoitavaa ja tämä työ on yksi askel eteenpäin taistelussa syöpäsairauksia vastaan

Histopathological image analysis : a review

Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe

Acute Myeloid Leukemia

Acute myeloid leukemia (AML) is the most common type of leukemia. The Cancer Genome Atlas Research Network has demonstrated the increasing genomic complexity of acute myeloid leukemia (AML). In addition, the network has facilitated our understanding of the molecular events leading to this deadly form of malignancy for which the prognosis has not improved over past decades. AML is a highly heterogeneous disease, and cytogenetics and molecular analysis of the various chromosome aberrations including deletions, duplications, aneuploidy, balanced reciprocal translocations and fusion of transcription factor genes and tyrosine kinases has led to better understanding and identification of subgroups of AML with different prognoses. Furthermore, molecular classification based on mRNA expression profiling has facilitated identification of novel subclasses and defined high-, poor-risk AML based on specific molecular signatures. However, despite increased understanding of AML genetics, the outcome for AML patients whose number is likely to rise as the population ages, has not changed significantly. Until it does, further investigation of the genomic complexity of the disease and advances in drug development are needed. In this review, leading AML clinicians and research investigators provide an up-to-date understanding of the molecular biology of the disease addressing advances in diagnosis, classification, prognostication and therapeutic strategies that may have significant promise and impact on overall patient survival

Advancing fluorescent contrast agent recovery methods for surgical guidance applications

Fluorescence-guided surgery (FGS) utilizes fluorescent contrast agents and specialized optical instruments to assist surgeons in intraoperatively identifying tissue-specific characteristics, such as perfusion, malignancy, and molecular function. In doing so, FGS represents a powerful surgical navigation tool for solving clinical challenges not easily addressed by other conventional imaging methods. With growing translational efforts, major hurdles within the FGS field include: insufficient tools for understanding contrast agent uptake behaviors, the inability to image tissue beyond a couple millimeters, and lastly, performance limitations of currently-approved contrast agents in accurately and rapidly labeling disease. The developments presented within this thesis aim to address such shortcomings. Current preclinical fluorescence imaging tools often sacrifice either 3D scale or spatial resolution. To address this gap in high-resolution, whole-body preclinical imaging tools available, the crux of this work lays on the development of a hyperspectral cryo-imaging system and image-processing techniques to accurately recapitulate high-resolution, 3D biodistributions in whole-animal experiments. Specifically, the goal is to correct each cryo-imaging dataset such that it becomes a useful reporter for whole-body biodistributions in relevant disease models. To investigate potential benefits of seeing deeper during FGS, we investigated short-wave infrared imaging (SWIR) for recovering fluorescence beyond the conventional top few millimeters. Through phantom, preclinical, and clinical SWIR imaging, we were able to 1) validate the capability of SWIR imaging with conventional NIR-I fluorophores, 2) demonstrate the translational benefits of SWIR-ICG angiography in a large animal model, and 3) detect micro-dose levels of an EGFR-targeted NIR-I probe during a Phase 0 clinical trial. Lastly, we evaluated contrast agent performances for FGS glioma resection and breast cancer margin assessment. To evaluate glioma-labeling performance of untargeted contrast agents, 3D agent biodistributions were compared voxel-by-voxel to gold-standard Gd-MRI and pathology slides. Finally, building on expertise in dual-probe ratiometric imaging at Dartmouth, a 10-pt clinical pilot study was carried out to assess the technique’s efficacy for rapid margin assessment. In summary, this thesis serves to advance FGS by introducing novel fluorescence imaging devices, techniques, and agents which overcome challenges in understanding whole-body agent biodistributions, recovering agent distributions at greater depths, and verifying agents’ performance for specific FGS applications

Enhanced Delivery of Rituximab Into Brain and Lymph Nodes Using Timed-Release Nanocapsules in Non-Human Primates.

Tumor metastasis into the central nervous system (CNS) and lymph nodes (LNs) is a major obstacle for effective therapies. Therapeutic monoclonal antibodies (mAb) have revolutionized tumor treatment; however, their efficacy for treating metastatic tumors-particularly, CNS and LN metastases-is poor due to inefficient penetration into the CNS and LNs following intravenous injection. We recently reported an effective delivery of mAb to the CNS by encapsulating the anti-CD20 mAb rituximab (RTX) within a thin shell of polymer that contains the analogs of choline and acetylcholine receptors. This encapsulated RTX, denoted as n-RTX, eliminated lymphoma cells systemically in a xenografted humanized mouse model using an immunodeficient mouse as a recipient of human hematopoietic stem/progenitor cells and fetal thymus more effectively than native RTX; importantly, n-RTX showed notable anti-tumor effect on CNS metastases which is unable to show by native RTX. As an important step toward future clinical translation of this technology, we further analyzed the properties of n-RTX in immunocompetent animals, rats, and non-human primates (NHPs). Our results show that a single intravenous injection of n-RTX resulted in 10-fold greater levels in the CNS and 2-3-fold greater levels in the LNs of RTX, respectively, than the injection of native RTX in both rats and NHPs. In addition, we demonstrate the enhanced delivery and efficient B-cell depletion in lymphoid organs of NHPs with n-RTX. Moreover, detailed hematological analysis and liver enzyme activity tests indicate n-RTX treatment is safe in NHPs. As this nanocapsule platform can be universally applied to other therapeutic mAbs, it holds great promise for extending mAb therapy to poorly accessible body compartments

Promoting tau secretion and propagation by hyperactive p300/CBP via autophagy-lysosomal pathway in tauopathy.

BackgroundThe trans-neuronal propagation of tau has been implicated in the progression of tau-mediated neurodegeneration. There is critical knowledge gap in understanding how tau is released and transmitted, and how that is dysregulated in diseases. Previously, we reported that lysine acetyltransferase p300/CBP acetylates tau and regulates its degradation and toxicity. However, whether p300/CBP is involved in regulation of tau secretion and propagation is unknown.MethodWe investigated the relationship between p300/CBP activity, the autophagy-lysosomal pathway (ALP) and tau secretion in mouse models of tauopathy and in cultured rodent and human neurons. Through a high-through-put compound screen, we identified a new p300 inhibitor that promotes autophagic flux and reduces tau secretion. Using fibril-induced tau spreading models in vitro and in vivo, we examined how p300/CBP regulates tau propagation.ResultsIncreased p300/CBP activity was associated with aberrant accumulation of ALP markers in a tau transgenic mouse model. p300/CBP hyperactivation blocked autophagic flux and increased tau secretion in neurons. Conversely, inhibiting p300/CBP promoted autophagic flux, reduced tau secretion, and reduced tau propagation in fibril-induced tau spreading models in vitro and in vivo.ConclusionsWe report that p300/CBP, a lysine acetyltransferase aberrantly activated in tauopathies, causes impairment in ALP, leading to excess tau secretion. This effect, together with increased intracellular tau accumulation, contributes to enhanced spreading of tau. Our findings suggest that inhibition of p300/CBP as a novel approach to correct ALP dysfunction and block disease progression in tauopathy

DEVELOPING NOVEL COMPUTER-AIDED DETECTION AND DIAGNOSIS SYSTEMS OF MEDICAL IMAGES

Reading medical images to detect and diagnose diseases is often difficult and has large inter-reader variability. To address this issue, developing computer-aided detection and diagnosis (CAD) schemes or systems of medical images has attracted broad research interest in the last several decades. Despite great effort and significant progress in previous studies, only limited CAD schemes have been used in clinical practice. Thus, developing new CAD schemes is still a hot research topic in medical imaging informatics field. In this dissertation, I investigate the feasibility of developing several new innovative CAD schemes for different application purposes. First, to predict breast tumor response to neoadjuvant chemotherapy and reduce unnecessary aggressive surgery, I developed two CAD schemes of breast magnetic resonance imaging (MRI) to generate quantitative image markers based on quantitative analysis of global kinetic features. Using the image marker computed from breast MRI acquired pre-chemotherapy, CAD scheme enables to predict radiographic complete response (CR) of breast tumors to neoadjuvant chemotherapy, while using the imaging marker based on the fusion of kinetic and texture features extracted from breast MRI performed after neoadjuvant chemotherapy, CAD scheme can better predict the pathologic complete response (pCR) of the patients. Second, to more accurately predict prognosis of stroke patients, quantifying brain hemorrhage and ventricular cerebrospinal fluid depicting on brain CT images can play an important role. For this purpose, I developed a new interactive CAD tool to segment hemorrhage regions and extract radiological imaging marker to quantitatively determine the severity of aneurysmal subarachnoid hemorrhage at presentation and correlate the estimation with various homeostatic/metabolic derangements and predict clinical outcome. Third, to improve the efficiency of primary antibody screening processes in new cancer drug development, I developed a CAD scheme to automatically identify the non-negative tissue slides, which indicate reactive antibodies in digital pathology images. Last, to improve operation efficiency and reliability of storing digital pathology image data, I developed a CAD scheme using optical character recognition algorithm to automatically extract metadata from tissue slide label images and reduce manual entry for slide tracking and archiving in the tissue pathology laboratories. In summary, in these studies, we developed and tested several innovative approaches to identify quantitative imaging markers with high discriminatory power. In all CAD schemes, the graphic user interface-based visual aid tools were also developed and implemented. Study results demonstrated feasibility of applying CAD technology to several new application fields, which has potential to assist radiologists, oncologists and pathologists improving accuracy and consistency in disease diagnosis and prognosis assessment of using medical image