The application of artificial intelligence and image analysis to novel prognostic classification systems of colorectal cancer

Summary: pp. IV-V

Discovery of novel prognostic tools to stratify high risk stage II colorectal cancer patients utilising digital pathology

Colorectal cancer (CRC) patients are stratified by the Tumour, Node and Metastasis (TNM) staging system for clinical decision making. Additional genomic markers have a limited utility in some cases where precise targeted therapy may be available. Thus, classical clinical pathological staging remains the mainstay of the assessment of this disease. Surgical resection is generally considered curative for Stage II patients, however 20-30% of these patients experience disease recurrence and disease specific death. It is imperative to identify these high risk patients in order to assess if further treatment or detailed follow up could be beneficial to their overall survival. The aim of the thesis was to categorise Stage II CRC patients into high and low risk of disease specific death through novel image based analysis algorithms. Firstly, an image analysis algorithm was developed to quantify and assess the prognostic value of three histopathological features through immuno-fluorescence: lymphatic vessel density (LVD), lymphatic vessel invasion (LVI) and tumour budding (TB). Image analysis provides the ability to standardise their quantification and negates observer variability. All three histopathological features were found to be predictors of CRC specific death within the training set (n=50); TB (HR =5.7; 95% CI, 2.38-13.8), LVD (HR =5.1; 95% CI, 2.04-12.99) and LVI (HR =9.9; 95% CI, 3.57- 27.98). Only TB (HR=2.49; 95% CI, 1.03-5.99) and LVI (HR =2.46; 95%CI, 1 - 6.05), however, were significant predictors of disease specific death in the validation set (n=134). Image analysis was further employed to characterise TB and quantify intra-tumoural heterogeneity. Tumour subpopulations within CRC tissue sections were segmented for the quantification of differential biomarker expression associated with epithelial mesenchymal transition and aggressive disease. Secondly, a novel histopathological feature ‘Sum Area Large Tumour Bud’ (ALTB) was identified through immunofluorescence coupled to a novel tissue phenomics approach. The tissue phenomics approach created a complex phenotypic fingerprint consisting of multiple parameters extracted from the unbiased segmentation of all objects within a digitised image. Data mining was employed to identify the significant parameters within the phenotypic fingerprint. ALTB was found to be a more significant predictor of disease specific death than LVI or TB in both the training set (HR = 20.2; 95% CI, 4.6 – 87.9) and the validation set (HR = 4; 95% CI, 1.5 – 11.1). Finally, ALTB was combined with two parameters, ‘differentiation’ and ‘pT stage’, which were exported from the original patient pathology report to form an integrative pathology score. The integrative pathology score was highly significant at predicting disease specific death within the validation set (HR = 7.5; 95% CI, 3 – 18.5). In conclusion, image analysis allows the standardised quantification of set histopathological features and the heterogeneous expression of biomarkers. A novel image based histopathological feature combined with classical pathology allows the highly significant stratification of Stage II CRC patients into high and low risk of disease specific death

Automated tumour budding quantification by machine learning augments TNM staging in muscle-invasive bladder cancer prognosis

Tumour budding has been described as an independent prognostic feature in several tumour types. We report for the first time the relationship between tumour budding and survival evaluated in patients with muscle invasive bladder cancer. A machine learning-based methodology was applied to accurately quantify tumour buds across immunofluorescence labelled whole slide images from 100 muscle invasive bladder cancer patients. Furthermore, tumour budding was found to be correlated to TNM (p = 0.00089) and pT (p = 0.0078) staging. A novel classification and regression tree model was constructed to stratify all stage II, III, and IV patients into three new staging criteria based on disease specific survival. For the stratification of non-metastatic patients into high or low risk of disease specific death, our decision tree model reported that tumour budding was the most significant feature (HR = 2.59, p = 0.0091), and no clinical feature was utilised to categorise these patients. Our findings demonstrate that tumour budding, quantified using automated image analysis provides prognostic value for muscle invasive bladder cancer patients and a better model fit than TNM staging.Publisher PDFPeer reviewe

Preprocessing algorithms for the digital histology of colorectal cancer

Pre-processing techniques were developed for cell identification algorithms. These algorithms which locate and classify cells in digital microscopy images are important in digital pathology. The pre-processing methods included image sampling and colour normalisation for standard Haemotoxilyn and Eosin (H&E) images and co-localisation algorithms for multiplexed images. Data studied in the thesis came from patients with colorectal cancer. Patient histology images came from `The Cancer Genome Atlas' (TCGA), a repository with contributions from many different institutional sites. The multiplexed images were created by TIS, the Toponome Imaging System. Experiments with image sampling were applied to TCGA diagnostic images. The effect of sample size and sampling policy were evaluated. TCGA images were also used in experiments with colour normalisation algorithms. For TIS multiplexed images, probabilistic graphical models were developed as well as clustering applications. NW-BHC, an extension to Bayesian Hierarchical Clustering, was developed and, for TIS antibodies, applied to TCGA expression data. Using image sampling with a sample size of 100 tiles gave accurate prediction results while being seven to nine times faster than processing the entire image. The two most accurate colour normalisation methods were that of Macenko and a `Nave' algorithm. Accuracy varied by TCGA site, indicating that researchers should use several independent data sets when evaluating colour normalisation algorithms. Probabilistic graphical models, applied to multiplexed images, calculated links between pairs of antibodies. The application of clustering to cell nuclei resulted in two main groups, one associated with epithelial cells and the second associated with the stromal environment. For TCGA expression data and for several clustering metrics, NW-BHC improved on the standard EM algorithm

Diffusion-weighted Imaging of Lymph Node Tissue

Purpose: The study investigates the hypothesis of clinically observed decreased apparent diffusion coefficient (ADC) of cancerous lymph nodes can be attributed to increased cellularity. The study characterises the mean diffusivity (MD) of lymph node sub-structures and investigates correlation between MD and cellularity metrics. The study also investigates the theoretical information content of single and multi-biophysical models. Methods:. A 3 mm diameter core sample was extracted from a formalin fixed lymph node tissue post-surgery and imaged using 9.4T and 16.4T Bruker MRI system. Samples were sectioned and stained with haematoxylin and eosin (H&E). Diffusion tensor model was fitted voxelwise and MD values were computed using Matlab. Cellularity metrics includes measurement of nuclear count and nuclear area. Eleven models with combinations of isotropic, anisotropic, and restricted components were tested for diffusion modelling and ranked using the Akaike information criterion (AIC). Results: The findings showed distinct diffusivities of lymph node sub-structures (capsule and parenchyma). Parenchyma in normal lymph node tissues had higher MD (0.71 ± 0.17 µm2/ms) than metastatic parenchyma (0.52 ± 0.08 µm2/ms) and lymphoma (0.47 ± 0.19 µm2/ms). No correlation were observed between MD and nuclear count (r = 0.368) and nuclear area (r = 0.368) respectively at 95 % confidence intervals. The single biophysical models (ADC and DTI) were ranked lowest by AIC. Multi-biophysical models consist of anisotropic and restricted diffusion (Zeppelin-sphere, Ball-stick-sphere, and Ball-sphere) were ranked highest in the majority of voxels of the tissue samples. Conclusion: A distinct diffusivity value were found in lymph node sub-structures with no correlation to cellularity. Multi-biophysical models were ranked highest and extract more information from the measurement data than simple single biophysical models

Machine learning strategies for diagnostic imaging support on histopathology and optical coherence tomography

Tesis por compendio[ES] Esta tesis presenta soluciones de vanguardia basadas en algoritmos de computer vision (CV) y machine learning (ML) para ayudar a los expertos en el diagnóstico clínico. Se centra en dos áreas relevantes en el campo de la imagen médica: la patología digital y la oftalmología. Este trabajo propone diferentes paradigmas de machine learning y deep learning para abordar diversos escenarios de supervisión en el estudio del cáncer de próstata, el cáncer de vejiga y el glaucoma. En particular, se consideran métodos supervisados convencionales para segmentar y clasificar estructuras específicas de la próstata en imágenes histológicas digitalizadas. Para el reconocimiento de patrones específicos de la vejiga, se llevan a cabo enfoques totalmente no supervisados basados en técnicas de deep-clustering. Con respecto a la detección del glaucoma, se aplican algoritmos de memoria a corto plazo (LSTMs) que permiten llevar a cabo un aprendizaje recurrente a partir de volúmenes de tomografía por coherencia óptica en el dominio espectral (SD-OCT). Finalmente, se propone el uso de redes neuronales prototípicas (PNN) en un marco de few-shot learning para determinar el nivel de gravedad del glaucoma a partir de imágenes OCT circumpapilares. Los métodos de inteligencia artificial (IA) que se detallan en esta tesis proporcionan una valiosa herramienta de ayuda al diagnóstico por imagen, ya sea para el diagnóstico histológico del cáncer de próstata y vejiga o para la evaluación del glaucoma a partir de datos de OCT.[CA] Aquesta tesi presenta solucions d'avantguarda basades en algorismes de *computer *vision (CV) i *machine *learning (ML) per a ajudar als experts en el diagnòstic clínic. Se centra en dues àrees rellevants en el camp de la imatge mèdica: la patologia digital i l'oftalmologia. Aquest treball proposa diferents paradigmes de *machine *learning i *deep *learning per a abordar diversos escenaris de supervisió en l'estudi del càncer de pròstata, el càncer de bufeta i el glaucoma. En particular, es consideren mètodes supervisats convencionals per a segmentar i classificar estructures específiques de la pròstata en imatges histològiques digitalitzades. Per al reconeixement de patrons específics de la bufeta, es duen a terme enfocaments totalment no supervisats basats en tècniques de *deep-*clustering. Respecte a la detecció del glaucoma, s'apliquen algorismes de memòria a curt termini (*LSTMs) que permeten dur a terme un aprenentatge recurrent a partir de volums de tomografia per coherència òptica en el domini espectral (SD-*OCT). Finalment, es proposa l'ús de xarxes neuronals *prototípicas (*PNN) en un marc de *few-*shot *learning per a determinar el nivell de gravetat del glaucoma a partir d'imatges *OCT *circumpapilares. Els mètodes d'intel·ligència artificial (*IA) que es detallen en aquesta tesi proporcionen una valuosa eina d'ajuda al diagnòstic per imatge, ja siga per al diagnòstic histològic del càncer de pròstata i bufeta o per a l'avaluació del glaucoma a partir de dades d'OCT.[EN] This thesis presents cutting-edge solutions based on computer vision (CV) and machine learning (ML) algorithms to assist experts in clinical diagnosis. It focuses on two relevant areas at the forefront of medical imaging: digital pathology and ophthalmology. This work proposes different machine learning and deep learning paradigms to address various supervisory scenarios in the study of prostate cancer, bladder cancer and glaucoma. In particular, conventional supervised methods are considered for segmenting and classifying prostate-specific structures in digitised histological images. For bladder-specific pattern recognition, fully unsupervised approaches based on deep-clustering techniques are carried out. Regarding glaucoma detection, long-short term memory algorithms (LSTMs) are applied to perform recurrent learning from spectral-domain optical coherence tomography (SD-OCT) volumes. Finally, the use of prototypical neural networks (PNNs) in a few-shot learning framework is proposed to determine the severity level of glaucoma from circumpapillary OCT images. The artificial intelligence (AI) methods detailed in this thesis provide a valuable tool to aid diagnostic imaging, whether for the histological diagnosis of prostate and bladder cancer or glaucoma assessment from OCT data.García Pardo, JG. (2022). Machine learning strategies for diagnostic imaging support on histopathology and optical coherence tomography [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/182400Compendi

Microscopical evaluation of prognostic factors in colorectal cancer

Aims and outline of the thesis. Since Fearon and Vogelstein in 1990 presented the genetic model for the adeno-carcinoma sequence of colorectal cancer, many prognostic studies varying from early stage markers to markers involved in late progression and liver metastases have followed. As has become evident from this introduction there is an ongoing need for prognostic markers that can be used for individualized prediction of clinical outcome. Chapter 2. Many systems are available for the detection of occult tumor cells in the bone marrow, blood and lymph nodes of cancer patients. In this chapter an overview is given of the various commercially available automated microscopy systems, and their capabilities. Furthermore the current status of the application of these instruments for bone marrow, blood and lymph nodes is presented. Chapter 3. Spread to locoregional lymph nodes is one of the most important prognostic indicators of the TNM classification. Detection of micrometastases in node-negative patients might upstage patients in need for additional chemotherapy. In this chapter an approach is described by which immunohistochemical staining and multiple sectioning is combined and is subjected to novel high-throughput automated imaging. Chapter 4. The presence of tumor cells in the bone marrow (BM) of cancer patients has shown to be related to a worse prognosis. This paper describes the use of array-CGH to detect genome alterations (gains and losses) in primary tumor tissue from BM-positive patients compared to matched (on stage and site) BM-negative patients. A higher number of differential aberrations and a distinct chromosome pattern, confirmed by interphase FISH, were found in the BM-positive group as compared to the BM-negative group. Chapter 5. While analyzing primary tumor tissue for a pilot study for array-CGH it was noticed that the set of patients with bad prognosis could not be analyzed, due to the fact that the amount of tumor material was less than 50%. This lower threshold is important for array-CGH to obtain reliable DNA profiles of the tumor cells and to avoid contamination with normal cells. Morphological evaluation of H&E stained sections showed that these tumors with bad prognosis had a high proportion of stroma and few tumor cells. The tumors with good prognosis showed the opposite, abundant tumor and less stroma. This phenomenon has led to the prognostic evaluation of this parameter in a larger patient study of which the results are shown in this chapter. Chapter 6. In this chapter the work presented in chapter 4 was continued but now focused on stage I-II colon patients. This subgroup of patients is in need for additional markers to select specific __high risk____ patients. Immunohistochemical staining of three molecular markers known to be involved in stroma production was performed. SMAD4 expression status was found to further improve the prognostic value of the presence of stroma in the primary tumor. Chapter 7. The conclusions of the studies presented in this thesis and the future perspectives of the presented parameters are discussed in this chapter.Financial support for publication of this thesis was kindly provided by: Leica Microsystems B.V, Covidien Nederland B.V., Merck Serono. With special thanks for their generous support to Applied Imaging a Genetix CompanyUBL - phd migration 201