Deep Learning for Semantic Segmentation versus Classification in Computational Pathology: Application to mitosis analysis in Breast Cancer grading

Existing computational pathology approaches did not allow, yet, the emergence of effective/efficient computer-aided tools used as a second opinion for pathologists in the daily practice. Focusing on the case of computer-based qualification for breast cancer diagnosis, the present article proposes two deep learning architectures to efficiently and effectively detect and classify mitosis in a histopathological tissue sample. The first method consisted of two parts, entailing a preprocessing of the digital histological image and a free-handcrafted-feature Convolutional Neural Network (CNN) used for binary classification. Results show that the methodology proposed can achieve 95% accuracy in testing with an F1-score of 94.35%, which is higher than the results from the literature using classical image processing techniques and also higher than the approaches using handcrafted features combined with CNNs. The second approach was an end-to-end methodology using semantic segmentation. Results showed that this algorithm can achieve an accuracy higher than 95% in testing and an average Dice index of 0.6 which is higher than the results from the literature using CNNs (0.9 F1-score). Additionally, due to the semantic properties of the deep learning approach, an end-to-end deep learning framework is viable to perform both tasks: detection and classification of mitosis. The results showed the potential of deep learning in the analysis of Whole Slide Images (WSI) and its integration to computer-aided systems. The extension of this work to whole slide images is also addressed in the last two chapters; as well as, some computational key points that are useful when constructing a computer-aided-system inspired by the described technology.Trabajo de investigació

Mosaic-Based Color-Transform Optimization for Lossy and Lossy-to-Lossless Compression of Pathology Whole-Slide Images

Altres ajuts: This work has been funded by the EU Marie Curie CIG Programme under Grant PIMCO, the Engineering and Physical Sciences Research Council (EPSRC), UKThe use of whole-slide images (WSIs) in pathology entails stringent storage and transmission requirements because of their huge dimensions. Therefore, image compression is an essential tool to enable efficient access to these data. In particular, color transforms are needed to exploit the very high degree of inter-component correlation and obtain competitive compression performance. Even though the state-of-the-art color transforms remove some redundancy, they disregard important details of the compression algorithm applied after the transform. Therefore, their coding performance is not optimal. We propose an optimization method called mosaic optimization for designing irreversible and reversible color transforms simultaneously optimized for any given WSI and the subsequent compression algorithm. Mosaic optimization is designed to attain reasonable computational complexity and enable continuous scanner operation. Exhaustive experimental results indicate that, for JPEG 2000 at identical compression ratios, the optimized transforms yield images more similar to the original than the other state-of-the-art transforms. Specifically, irreversible optimized transforms outperform the Karhunen-Loève Transform in terms of PSNR (up to 1.1 dB), the HDR-VDP-2 visual distortion metric (up to 3.8 dB), and the accuracy of computer-aided nuclei detection tasks (F1 score up to 0.04 higher). In addition, reversible optimized transforms achieve PSNR, HDR-VDP-2, and nuclei detection accuracy gains of up to 0.9 dB, 7.1 dB, and 0.025, respectively, when compared with the reversible color transform in lossy-to-lossless compression regimes

A supervised visual model for finding regions of interest in basal cell carcinoma images

This paper introduces a supervised learning method for finding diagnostic regions of interest in histopathological images. The method is based on the cognitive process of visual selection of relevant regions that arises during a pathologist's image examination. The proposed strategy emulates the interaction of the visual cortex areas V1, V2 and V4, being the V1 cortex responsible for assigning local levels of relevance to visual inputs while the V2 cortex gathers together these small regions according to some weights modulated by the V4 cortex, which stores some learned rules. This novel strategy can be considered as a complex mix of "bottom-up" and "top-down" mechanisms, integrated by calculating a unique index inside each region. The method was evaluated on a set of 338 images in which an expert pathologist had drawn the Regions of Interest. The proposed method outperforms two state-of-the-art methods devised to determine Regions of Interest (RoIs) in natural images. The quality gain with respect to an adaptated Itti's model which found RoIs was 3.6 dB in average, while with respect to the Achanta's proposal was 4.9 dB

Machine Learning Assisted Digital Pathology

Histopatologiset kudosnäytteet sisältävät valtavan määrän tietoa biologisista mekanismeista, jotka vaikuttavat monien tautien ilmenemiseen ja etenemiseen. Tästä syystä histopatologisten näytteiden arviointi on ollut perustana monien tautien diagnostiikassa vuosikymmenien ajan. Perinteinen histopatologinen arviointi on kuitenkin työläs tehtävä ja lisäksi erittäin altis inhimillisille virheille ja voi siten johtaa virheelliseen tai viivästyneeseen diagnoosiin. Viime vuosien teknologinen kehitys on tuonut patologien käyttöön lasiskannerit ja tiedonhallintajarjestelmät ja sitä myötä mahdollistaneet näytelasien digitoinnin ja käynnistäneet koko patologian työnkulun digitalisaation. Histopatologisten näytteiden saatavuus digitaalisina kuvina on puolestaan mahdollistanut älykkäiden algoritmien ja automatisoitujen laskennallisten kuva-analyysityökalujen kehittämisen diagnostiikan tueksi. Koneoppiminen on tekoälyn osa-alue, joka voidaan määritellä datasta oppimiseksi. Kuva-analyysin sovelluksissa, kuvan pikseliarvot muutetaan kvantitatiiviseksi piirre-esitykseksi jonka pohjalta kuva voidaan muuntaa merkitykselliseksi tiedoksi hyvödyntämällä koneoppimista. Vuosien saatossa koneoppimiseen perustuvan kuva-analyysin menetelmät ovat kehittyneet manuaalisesta piirteidenirroituksesta kohti viimevuosien vallitsevia syväoppimiseen pohjautuvia konvoluutioneuroverkkoja. Koneoppimisen hyödyt histopatologisessa arvioinnissa ovat huomattavat, sillä koneoppiminen mahdollistaa kuvien tulkinnan patologiin verrattavalla tarkkuudella ja siten pystyy merkittävästi parantamaan kliinisen patologian diagnostiikan tarkkuutta, toistettavuutta ja tehokkuutta. Tämä väitöstyö esittelee koneoppimiseen pohjautuvia menetelmiä jotka on kehitetty avustamaan kudosnäytteen histopatologista arviointia, vaihetta joka on merkityksellinen niin kliinisessä diagnostiikassa kuin prekliinisissä tutkimuksissa. Työssä esitellään piirteenirroituksen ja koneoppimisen tehokkuus histopatologiseen arviointiin liittyvissä kuva-analyysitehtävissä kuten kudoksen karakterisoinnissa, sekä rintasyövän etäpesäkkeiden, epiteelikudoksen ja tumien tunnistuksessa. Menetelmien lisäksi tässä väitöstyössä on käsitelty keskeisiä haasteita jotka on huomioitava integroitaessa koneoppimismenetelmiä kliiniseen käyttöön. Ennen kaikkea nämä tutkimukset ovat kuitenkin osoittaneet koneoppimisen mahdollisuudet tulevaisuudessa parantaa patologian kliinisten rutiinitehtävien tehokkuutta ja toistettavuutta sekä diagnostiikan laatua.Histopathological tissue samples contain a vast amount of information on underlying biological mechanisms that contribute to disease manifestation and progression. Therefore, diagnosis from histopathological tissue samples has been the gold standard for decades. However, traditional histopathological assessment is a laborious task and prone to human errors, thereby leading to misdiagnosis or delayed diagnosis. The development of whole slide scanners for digitization of tissue glass slides has initiated the transition to a fully digital pathology workflow that allows scanning, interpretation, and management of digital tissue slides. These advances have been the cornerstone for developing intelligent algorithms and automated computational approaches for histopathological assessment and clinical diagnostics. Machine learning is a subcategory of artificial intelligence and can be defined as a process of learning from data. In image analysis tasks, the raw pixel values are transformed into quantitative feature representations. Based on the image data representation, a machine learning model learns a set of rules that can be used to extract meaningful information and knowledge. Over the years, the field of machine learning based image analysis has developed from manually handcrafting complex features to the recent revolution of deep learning and convolutional neural networks. Histopathological assessment can benefit greatly from the ability of machine learning models to discover patterns and connections from the data. Therefore, machine learning holds great promise to improve the accuracy, reproducibility, and efficiency of clinical diagnostics in the field of digital pathology. This thesis is focused on developing machine learning based methods for assisting in the process of histopathological assessment, which is a significant step in clinical diagnostics as well as in preclinical studies. The studies presented in this thesis show the effectiveness of feature engineering and machine learning in histopathological assessment related tasks, such as; tissue characterisation, metastasis detection, epithelial tissue detection, and nuclei detection. Moreover, the studies presented in this thesis address the key challenges related to variation presented in histopathological data as well as the generalisation problem that need to be considered in order to integrate machine learning approaches into clinical practice. Overall, these studies have demonstrated the potential of machine learning for bringing standardization and reproducibility to the process of histopathological assessment

Estudio del armazón arquitectónico y del sistema vascular de los tumores neuroblásticos

Los pacientes con tumores neuroblásticos presentan una evolución clínica heterogénea, desde la regresión espontánea hasta una alta propensión para la diseminación metastática generalizada. Aunque la aplicación de una clasificación de riesgo pre-tratamiento bien definida tiene un papel central en la mejora de la supervivencia durante los últimos años, han de llevarse a cabo más avances para mejorar la superviencia de los pacientes en general y específicamente el subgrupo de pacientes de alto riesgo. El estudio morfológico del tejido tumoral está contribuyendo a dicha mejora. La categoría histológica o el porcentaje de estroma tumoral, así como el grado de diferenciación de las células neuroblásticas, determinadas por el patólogo con el microscopio óptico, son factores con un papel importante en el diagnóstico y el pronóstico de los pacientes. Actualmente, dada la relevancia de la matriz extracelular tumoral en la biotensegridad y la mecanotransducción, su arquitectura y la topología de sus elementos, así como su interacción están siendo cada vez más considerados. Su cuantificación y caracterización con técnicas de imagen microscópicas empiezan a ser utilizadas. Nuestra hipótesis es que el destino de una célula tumoral neuroblástica es complejo y entre otros factores, está determinado por las características de un grupo de elementos estructurales no celulares de la matriz extracelular. Además pensamos que aplicando los patrones derivados del análisis morfométrico de estos elementos y asociandolos al impacto de los factores pronósticos conocidos, se mejorará la supervivencia de los pacientes. Nuestro objetivo es el desarrollo de técnicas morfométricas para caracterizar distintos elementos del andamiaje de la matriz extracelular y de la vascularización con el fin de encontrar usos potenciales como nuevos marcadores con valor pronóstico para mejorar la estratificación de los pacientes, o como dianas terapéuticas para ser capaces de remodelar los elementos aberrantes del andamiaje tisular, incluyendo la microvascularización. Hemos construido 19 micromatrices de tejido incluyendo más de 500 neuroblastomas, que fueron teñidos con alzul alcián a pH 2,5, Gomori, tricrómico de Masson, orceína y anti-CD31 para glicosaminoglicanos, fibras de reticulina, fibras de colágeno tipo I, fibras elásticas y vasos sanguíneos, respectivamente. Las laminillas fueron digitalizadas con un escáner de preparaciones y distintos algoritmos de análisis de imagen fueron diseñados o personalizados para detectar y caracterizar la cantidad, el tamaño y la forma de los distintos elementos estudiados de la matriz extracelular. Estos parámetros se relacionaron con los distintos subgrupos de neuroblastoma, teniendo en cuenta varias características clínicas, histopatológicas y genéticas. Los resultados obtenidos mostraron que las fibras de reticulina eran los componentes mayoritarios del andamiaje fibroso y que la abundancia y arquitectura de la microvascularización era relevante para el pronóstico de los niños con neuroblastoma. Una matriz extracelular rígida y poco porosa con vasos sanguíneos con luces irregulares se detectó principalmente en tumores pertenecientes a pacientes con pronóstico desfavorable. Un subgrupo de la cohorte de alto riesgo con muy mala supervivencia pudo ser definido por variables morfométricas de las fibras de reticulina y de los vasos sanguíneos. Concretamente, las muestras con un mayores áreas ocupadas tanto por fibras de reticulina formando grandes redes entrecruzadas, ramificadas y de organización compleja, como por vasos sanguíneos, junto con capilares y vasos tipo sinusoide de forma irregular y vénulas y arteriolas dilatas, estaban asociadas a un pronóstico muy desfavorable. En esta cohorte, las células con amplificación del gen MYCN conllevaron cambios topológicos detectables en relación a las fibras de reticulina y los vasos sanguíneos. Podemos concluir que es possible y conveniente cuantificar la sustancia fundamental, caracterizar el andamiaje fibroso y el sistema vascular de los tumors neuroblásticos gracias al análisis morfométrico de imágenes microscópicas. Algunas de las características morfométricas relaciondas con los distintos elementos de la matriz extracelular estudiados podrían ser usadas como ayuda diagnóstica del grupo de pacientes con riesgo ultra alto, tras estudiar una mayor cohorte. Los resultados obtenidos sugieren la necesidad de realizar trabajos multidisciplinarios para integrar de estos estudios a nivel internacional y que la información morfométrica de los elementos de la matriz extracelular, incluyendo el sistema vascular, pueda ser utilizada para una terapia basada en la mecanotransducción.Neuroblastic tumor patients present an heterogeneous clinical evolution, from spontaneous regression to a high propensity for widespread metastatic dissemination. Although the application of a well-defined pre-treatment risk classification plays a central role in the improvement of survival during the last years, more efforts must be done to improve patient’s survival in general and specifically in the subgroup of high risk patients. The morphological study of the tumoral tissue is contributing to such improvement. The histological category or the percentage of tumoral stroma, as well as the degree of differentiation of neuroblastic cells, evaluated by the pathologist with light microscopy, are factors that play a role in the diagnosis and prognosis of the patients. Given the role of tumoral extracellular matrix in biotensegrity and mechanotransduction, its architecture and the topology of its elements, as well as their interaction are being increasingly considered. Its quantification and characterization with microscopic image techniques start to be used. We hypothesize that the destiny of a neuroblastic tumor cell is complex and, is in part directed by characteristics of a set of non-cellular extracellular matrix structural elements. Additionally, we think that the application of the patterns derived from the morphometric analysis of such elements and their association with the impact of the known prognostic factors, patient’s survival will be improved. We aim to develop morphometric techniques to characterize different extracellular matrix scaffolding and vascular elements to find out potential uses as new prognostic markers for a better pre-treatment stratification of the patients or as therapeutic targets to be able to remodel the aberrant elements of the tissue scaffolding, including microvascularization. We constructed 19 tissue microarrays including more than 500 neuroblastomas which were stained with alcian blue pH 2.5, Gomori, Masson’s trichrome, orcein and anti-CD31 for glycosaminoglycans, reticulin fibers, collagen type I fibers, elastic fibers and blood vessels, respectively. The slides were digitized with a whole-slide scanner and different image-analysis algorithms were designed or customized to specifically detect and characterize the amount, the size and the shape of the different extracellular matrix elements studied. These parameters were related to different neuroblastoma subgroups, taking into account several clinical, histopathological and genetic features. The results obtained showed that reticulin fibers were the main components of the fibrous scaffolding and that microvasculature amount and architecture were relevant in the prognosis of neuroblastoma patients. A stiff and poorly porous extracellular matrix with irregularly-shaped vascular lumens was mainly detected in tumors belonging to patients with unfavorable prognosis. A subgroup of the high risk cohort with very poor survival could be defined by morphometric variables of reticulin fibers and blood vessels. Specificallly, those samples with high stained areas occupied by reticulin fibers forming large, crosslinking, branching and disorganized networks and by blood vessels, as well as with irregularly-shaped capillaries and sinusoid-like vessels and dilated venules, presented a very unfavorable survival. In this cohort, cells with MYCN gene amplification led to detectable topological changes regarding reticulin fibers and bood vessels. We can conclude that it is possible and convenient to quantify the fundamental substance and characterize the architecture of the fibrous scaffolding and the vascular system of neuroblastic tumors by means of the morphometric analysis of microscopic images. Some of the morphometric features related to the different extracellular matrix elements studied could be used as a diagnostic support for the ultra-high risk group of patients, after studying a larger cohort. The obtained results suggest the need of developing multidisciplinary efforts for an international integration of these studies, and that the morphometric information of the elements of the extracellular matrix, including the vascular system, could be used for a therapy based on mechanotransduction