Improving cancer subtype diagnosis and grading using clinical decision support system based on computer-aided tissue image analysis

This research focuses towards the development of a clinical decision support system (CDSS) based on cellular and tissue image analysis and classification system that improves consistency and facilitates the clinical decision making process. In a typical cancer examination, pathologists make diagnosis by manually reading morphological features in patient biopsy images, in which cancer biomarkers are highlighted by using different staining techniques. This process is subjected to pathologist's training and experience, especially when the same cancer has several subtypes (i.e. benign tumor subtype vs. malignant subtype) and the same cancer tissue biopsy contains heterogeneous morphologies in different locations. The variability in pathologist's manual reading may result in varying cancer diagnosis and treatment. This Ph.D. research aims to reduce the subjectivity and variation existing in traditional histo-pathological reading of patient tissue biopsy slides through Computer-Aided Diagnosis (CAD). Using the CAD, quantitative molecular profiling of cancer biomarkers of stained biopsy images are obtained by extracting and analyzing texture and cellular structure features. In addition, cancer sub-type classification and a semi-automatic grade scoring (i.e. clinical decision making) for improved consistency over a large number of cancer subtype images can be performed. The CAD tools do have their own limitations and in certain cases the clinicians, however, prefer systems which are flexible and take into account their individuality when necessary by providing some control rather than fully automated system. Therefore, to be able to introduce CDSS in health care, we need to understand users' perspectives and preferences on the new information technology. This forms as the basis for this research where we target to present the quantitative information acquired through the image analysis, annotate the images and provide suitable visualization which can facilitate the process of decision making in a clinical setting.PhDCommittee Chair: Dr. May D. Wang; Committee Member: Dr. Andrew N. Young; Committee Member: Dr. Anthony J. Yezzi; Committee Member: Dr. Edward J. Coyle; Committee Member: Dr. Paul Benkese

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of eXplainable Artificial Intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outlook of future opportunities for XAI in medical image analysis.Comment: Submitted for publication. Comments welcome by email to first autho

Towards PACE-CAD Systems

Despite phenomenal advancements in the availability of medical image datasets and the development of modern classification algorithms, Computer-Aided Diagnosis (CAD) has had limited practical exposure in the real-world clinical workflow. This is primarily because of the inherently demanding and sensitive nature of medical diagnosis that can have far-reaching and serious repercussions in case of misdiagnosis. In this work, a paradigm called PACE (Pragmatic, Accurate, Confident, & Explainable) is presented as a set of some of must-have features for any CAD. Diagnosis of glaucoma using Retinal Fundus Images (RFIs) is taken as the primary use case for development of various methods that may enrich an ordinary CAD system with PACE. However, depending on specific requirements for different methods, other application areas in ophthalmology and dermatology have also been explored. Pragmatic CAD systems refer to a solution that can perform reliably in day-to-day clinical setup. In this research two, of possibly many, aspects of a pragmatic CAD are addressed. Firstly, observing that the existing medical image datasets are small and not representative of images taken in the real-world, a large RFI dataset for glaucoma detection is curated and published. Secondly, realising that a salient attribute of a reliable and pragmatic CAD is its ability to perform in a range of clinically relevant scenarios, classification of 622 unique cutaneous diseases in one of the largest publicly available datasets of skin lesions is successfully performed. Accuracy is one of the most essential metrics of any CAD system's performance. Domain knowledge relevant to three types of diseases, namely glaucoma, Diabetic Retinopathy (DR), and skin lesions, is industriously utilised in an attempt to improve the accuracy. For glaucoma, a two-stage framework for automatic Optic Disc (OD) localisation and glaucoma detection is developed, which marked new state-of-the-art for glaucoma detection and OD localisation. To identify DR, a model is proposed that combines coarse-grained classifiers with fine-grained classifiers and grades the disease in four stages with respect to severity. Lastly, different methods of modelling and incorporating metadata are also examined and their effect on a model's classification performance is studied. Confidence in diagnosing a disease is equally important as the diagnosis itself. One of the biggest reasons hampering the successful deployment of CAD in the real-world is that medical diagnosis cannot be readily decided based on an algorithm's output. Therefore, a hybrid CNN architecture is proposed with the convolutional feature extractor trained using point estimates and a dense classifier trained using Bayesian estimates. Evaluation on 13 publicly available datasets shows the superiority of this method in terms of classification accuracy and also provides an estimate of uncertainty for every prediction. Explainability of AI-driven algorithms has become a legal requirement after Europe’s General Data Protection Regulations came into effect. This research presents a framework for easy-to-understand textual explanations of skin lesion diagnosis. The framework is called ExAID (Explainable AI for Dermatology) and relies upon two fundamental modules. The first module uses any deep skin lesion classifier and performs detailed analysis on its latent space to map human-understandable disease-related concepts to the latent representation learnt by the deep model. The second module proposes Concept Localisation Maps, which extend Concept Activation Vectors by locating significant regions corresponding to a learned concept in the latent space of a trained image classifier. This thesis probes many viable solutions to equip a CAD system with PACE. However, it is noted that some of these methods require specific attributes in datasets and, therefore, not all methods may be applied on a single dataset. Regardless, this work anticipates that consolidating PACE into a CAD system can not only increase the confidence of medical practitioners in such tools but also serve as a stepping stone for the further development of AI-driven technologies in healthcare

Exfoliative cytology for diagnosing basal cell carcinoma and other skin cancers in adults

Background: Early accurate detection of all skin cancer types is essential to guide appropriate management and to reduce morbidity and improve survival. Basal cell carcinoma (BCC) is usually localised to the skin with potential to infiltrate and damage surrounding tissue, while cutaneous squamous cell carcinoma (cSCC) and melanoma have a much higher potential to metastasise and ultimately lead to death. Exfoliative cytology is a non–invasive test that uses the Tzanck smear technique to identify disease by examining the structure of cells obtained from scraped samples. This simple procedure is a less invasive diagnostic test than a skin biopsy, and for BCC has the potential to provide an immediate diagnosis that avoids an additional visit to clinic to receive skin biopsy results. This may benefit patients scheduled for either Mohs micrographic surgery or non–surgical treatments such as radiotherapy. A cytology scrape can never give the same information as a skin biopsy, however, so it is important to know more about which skin cancer situations it may be helpful.Objectives: The primary objective was to determine the diagnostic accuracy of exfoliative cytology for the detection of basal cell carcinoma (BCC) in adults. Secondary objectives were to determine diagnostic accuracy for the detection of i) cutaneous squamous cell carcinoma, ii) invasive melanoma and atypical intraepidermal melanocytic variants, and iii) any skin cancer, including keratinocyte skin cancer, invasive melanoma and atypical intraepidermal melanocytic variants, or any other skin cancer.Search methods: We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; EMBASE; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We also studied the reference lists of published systematic review articles.Selection criteria: Studies evaluating exfoliative cytology in adults with lesions suspicious for BCC, cSCC or melanoma, compared with a reference standard of histological confirmation.Data collection and analysis: Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). Where possible we estimated summary sensitivities and specificities using the bivariate hierarchical model.Main results: This review reports on nine studies with a total of 1655 lesions including 1120 BCCs (14 datasets), 401 lesions with 44 cSCCs (two datasets), and 200 lesions with 10 melanomas (one dataset). Three of these datasets (one each for BCC, melanoma, and any malignant condition) were derived from one study which also performed a direct comparison with dermoscopy. Studies were of moderate to poor quality providing inadequate descriptions of participant selection, thresholds used to make cytological and histological diagnoses, and blinding. Reporting of patients’ prior referral pathways was particularly poor, as were descriptions of the cytodiagnostic criteria used to make diagnoses. No studies evaluated the use of exfoliative cytology as a primary diagnostic test for detecting BCC or other skin cancers in lesions suspicious for skin cancer. Pooled data from seven studies using standard cytomorphological criteria (but various stain methods) to detect BCC in patients with a high clinical suspicion of BCC estimated the sensitivity and specificity of exfoliative cytology as 97.5% (95% CI: 94.5 to 98.9%) and 90.1% (95% CI: 81.1 to 95.1%) respectively. When applied to a hypothetical population of 1000 clinically suspected BCC lesions with a median observed BCC prevalence of 86%, exfoliative cytology would miss 21 BCCs and would lead to 14 false positive diagnoses of BCC. No false positive cases were histologically confirmed to be melanoma. Insufficient data are available to make summary statements regarding the accuracy of exfoliative cytology to detect melanoma or cSCC, or its accuracy compared to dermoscopy.Authors' conclusions: The utility of exfoliative cytology for the primary diagnosis of skin cancer is unknown, as all included studies focused on the use of this technique for confirming strongly suspected clinical diagnoses. For the confirmation of BCC in lesions with a high clinical suspicion, there is evidence of high sensitivity and specificity for exfoliative cytology. Since decisions to treat low riskBCCs are unlikely in practice to require diagnostic confirmation given that clinical suspicion is already high, exfoliative cytology might be most useful for cases of BCC where the treatments being contemplated require a tissue diagnosis (e.g. radiotherapy). The small number of included studies, poor reporting and varying methodological quality means that no strong conclusions can currently be drawn to guide clinical practice. Despite insufficient data on the use of cytology for cSCC or melanoma, it is unlikely that cytology would be useful in these scenarios since preservation of the architecture of the whole lesion that would be available from a biopsy provides crucial diagnostic information. Given the paucity of good quality data, appropriately designed prospective comparative studies may be required to evaluate both the diagnostic value of exfoliative cytology by comparison to dermoscopy, and its confirmatory value in adequately reported populations with a high probability of BCC scheduled for further treatment requiring a tissue diagnosis

Modelling the head and neck region for microwave imaging of cervical lymph nodes

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Radiações em Diagnóstico e Terapia), Universidade de Lisboa, Faculdade de Ciências, 2020O termo “cancro da cabeça e pescoço” refere-se a um qualquer tipo de cancro com início nas células epiteliais das cavidades oral e nasal, seios perinasais, glândulas salivares, faringe e laringe. Estes tumores malignos apresentaram, em 2018, uma incidência mundial de cerca de 887.659 novos casos e taxa de mortalidade superior a 51%. Aproximadamente 80% dos novos casos diagnosticados nesse ano revelaram a proliferação de células cancerígenas dos tumores para outras regiões do corpo através dos vasos sanguíneos e linfáticos das redondezas. De forma a determinar o estado de desenvolvimento do cancro e as terapias a serem seguidas, é fundamental a avaliação dos primeiros gânglios linfáticos que recebem a drenagem do tumor primário – os gânglios sentinela – e que, por isso, apresentam maior probabilidade de se tornarem os primeiros alvos das células tumorais. Gânglios sentinela saudáveis implicam uma menor probabilidade de surgirem metástases, isto é, novos focos tumorais decorrentes da disseminação do cancro para outros órgãos. O procedimento standard que permite o diagnóstico dos gânglios linfáticos cervicais, gânglios que se encontram na região da cabeça e pescoço, e o estadiamento do cancro consiste na remoção cirúrgica destes gânglios e subsequente histopatologia. Para além de ser um procedimento invasivo, a excisão cirúrgica dos gânglios linfáticos representa perigos tanto para a saúde mental e física dos pacientes, como para a sua qualidade de vida. Dores, aparência física deformada (devido a cicatrizes), perda da fala ou da capacidade de deglutição são algumas das repercussões que poderão advir da remoção de gânglios linfáticos da região da cabeça e pescoço. Adicionalmente, o risco de infeção e linfedema – acumulação de linfa nos tecidos intersticiais – aumenta significativamente com a remoção de uma grande quantidade de gânglios linfáticos saudáveis. Também os encargos para os sistemas de saúde são elevados devido à necessidade de monitorização destes pacientes e subsequentes terapias e cuidados associados à morbilidade, como é o caso da drenagem linfática manual e da fisioterapia. O desenvolvimento de novas tecnologias de imagem da cabeça e pescoço requer o uso de modelos realistas que simulem o comportamento e propriedades dos tecidos biológicos. A imagem médica por micro-ondas é uma técnica promissora e não invasiva que utiliza radiação não ionizante, isto é, sinais com frequências na gama das micro-ondas cujo comportamento depende do contraste dielétrico entre os diferentes tecidos atravessados, pelo que é possível identificar regiões ou estruturas de interesse e, consequentemente, complementar o diagnóstico. No entanto, devido às suas características, este tipo de modalidade apenas poderá ser utilizado para a avaliação de regiões anatómicas pouco profundas. Estudos indicam que os gânglios linfáticos com células tumorais possuem propriedades dielétricas distintas dos gânglios linfáticos saudáveis. Por esta razão e juntamente pelo facto da sua localização pouco profunda, consideramos que os gânglios linfáticos da região da cabeça e pescoço constituem um excelente candidato para a utilização de imagem médica por radar na frequência das micro-ondas como ferramenta de diagnóstico. Até à data, não foram efetuados estudos de desenvolvimento de modelos da região da cabeça e pescoço focados em representar realisticamente os gânglios linfáticos cervicais. Por este motivo, este projeto consistiu no desenvolvimento de dois geradores de fantomas tridimensionais da região da cabeça e pescoço – um gerador de fantomas numéricos simples (gerador I) e um gerador de fantomas numéricos mais complexos e anatomicamente realistas, que foi derivado de imagens de ressonância magnética e que inclui as propriedades dielétricas realistas dos tecidos biológicos (gerador II). Ambos os geradores permitem obter fantomas com diferentes níveis de complexidade e assim acompanhar diferentes fases no processo de desenvolvimento de equipamentos médicos de imagiologia por micro-ondas. Todos os fantomas gerados, e principalmente os fantomas anatomicamente realistas, poderão ser mais tarde impressos a três dimensões. O processo de construção do gerador I compreendeu a modelação da região da cabeça e pescoço em concordância com a anatomia humana e distribuição dos principais tecidos, e a criação de uma interface para a personalização dos modelos (por exemplo, a inclusão ou remoção de alguns tecidos é dependente do propósito para o qual cada modelo é gerado). O estudo minucioso desta região levou à inclusão de tecidos ósseos, musculares e adiposos, pele e gânglios linfáticos nos modelos. Apesar destes fantomas serem bastante simples, são essenciais para o início do processo de desenvolvimento de dispositivos de imagem médica por micro-ondas dedicados ao diagnóstico dos gânglios linfáticos cervicais. O processo de construção do gerador II foi fracionado em 3 grandes etapas devido ao seu elevado grau de complexidade. A primeira etapa consistiu na criação de uma pipeline que permitiu o processamento das imagens de ressonância magnética. Esta pipeline incluiu: a normalização dos dados, a subtração do background com recurso a máscaras binárias manualmente construídas, o tratamento das imagens através do uso de filtros lineares (como por exemplo, filtros passa-baixo ideal, Gaussiano e Butterworth) e não-lineares (por exemplo, o filtro mediana), e o uso de algoritmos não supervisionados de machine learning para a segmentação dos vários tecidos biológicos presentes na região cervical, tais como o K-means, Agglomerative Hierarchical Clustering, DBSCAN e BIRCH. Visto que cada algoritmo não supervisionado de machine learning anteriormente referido requer diferentes hiperparâmetros, é necessário proceder a um estudo pormenorizado que permita a compreensão do modo de funcionamento de cada algoritmo individualmente e a sua interação / performance com o tipo de dados tratados neste projeto (isto é, dados de exames de ressonâncias magnéticas) com vista a escolher empiricamente o leque de valores de cada hiperparâmetro que deve ser considerado, e ainda as combinações que devem ser testadas. Após esta fase, segue-se a avaliação da combinação de hiperparâmetros que resulta na melhor segmentação das estruturas anatómicas. Para esta avaliação são consideradas duas metodologias que foram combinadas: a utilização de métricas que permitam avaliar a qualidade do clustering (como por exemplo, o Silhoeutte Coefficient, o índice de Davies-Bouldin e o índice de Calinski-Harabasz) e ainda a inspeção visual. A segunda etapa foi dedicada à introdução manual de algumas estruturas, como a pele e os gânglios linfáticos, que não foram segmentadas pelos algoritmos de machine learning devido à sua fina espessura e pequena dimensão, respetivamente. Finalmente, a última etapa consistiu na atribuição das propriedades dielétricas, para uma frequência pré-definida, aos tecidos biológicos através do Modelo de Cole-Cole de quatro pólos. Tal como no gerador I, foi criada uma interface que permitiu ao utilizador decidir que características pretende incluir no fantoma, tais como: os tecidos a incluir (tecido adiposo, tecido muscular, pele e / ou gânglios linfáticos), relativamente aos gânglios linfáticos o utilizador poderá ainda determinar o seu número, dimensões, localização em níveis e estado clínico (saudável ou metastizado) e finalmente, o valor de frequência para o qual pretende obter as propriedades dielétricas (permitividade relativa e condutividade) de cada tecido biológico. Este projeto resultou no desenvolvimento de um gerador de modelos realistas da região da cabeça e pescoço com foco nos gânglios linfáticos cervicais, que permite a inserção de tecidos biológicos, tais como o tecidos muscular e adiposo, pele e gânglios linfáticos e aos quais atribui as propriedades dielétricas para uma determinada frequência na gama de micro-ondas. Estes modelos computacionais resultantes do gerador II, e que poderão ser mais tarde impressos em 3D, podem vir a ter grande impacto no processo de desenvolvimento de dispositivos médicos de imagem por micro-ondas que visam diagnosticar gânglios linfáticos cervicais, e consequentemente, contribuir para um processo não invasivo de estadiamento do cancro da cabeça e pescoço.Head and neck cancer is a broad term referring to any epithelial malignancies arising in the paranasal sinuses, nasal and oral cavities, salivary glands, pharynx, and larynx. In 2018, approximately 80% of the newly diagnosed head and neck cancer cases resulted in tumour cells spreading to neighbouring lymph and blood vessels. In order to determine cancer staging and decide which follow-up exams and therapy to follow, physicians excise and assess the Lymph Nodes (LNs) closest to the primary site of the head and neck tumour – the sentinel nodes – which are the ones with highest probability of being targeted by cancer cells. The standard procedure to diagnose the Cervical Lymph Nodes (CLNs), i.e. lymph nodes within the head and neck region, and determine the cancer staging frequently involves their surgical removal and subsequent histopathology. Besides being invasive, the removal of the lymph nodes also has negative impact on patients’ quality of life, it can be health threatening, and it is costly to healthcare systems due to the patients’ needs for follow-up treatments/cares. Anatomically realistic phantoms are required to develop novel technologies tailored to image head and neck regions. Medical MicroWave Imaging (MWI) is a promising non-invasive approach which uses non-ionizing radiation to screen shallow body regions, therefore cervical lymph nodes are excellent candidates to this imaging modality. In this project, a three-dimensional (3D) numerical phantom generator (generator I) and a Magnetic Resonance Imaging (MRI)-derived anthropomorphic phantom generator (generator II) of the head and neck region were developed to create phantoms with different levels of complexity and realism, which can be later 3D printed to test medical MWI devices. The process of designing the numerical phantom generator included the modelling of the head and neck regions according to their anatomy and the distribution of their main tissues, and the creation of an interface which allowed the users to personalise the model (e.g. include or remove certain tissues, depending on the purpose of each generated model). To build the anthropomorphic phantom generator, the modelling process included the creation of a pipeline of data processing steps to be applied to MRIs of the head and neck, followed by the development of algorithms to introduce additional tissues to the models, such as skin and lymph nodes, and finally, the assignment of the dielectric properties to the biological tissues. Similarly, this generator allowed users to decide the features they wish to include in the phantoms. This project resulted in the creation of a generator of 3D anatomically realistic head and neck phantoms which allows the inclusion of biological tissues such as skin, muscle tissue, adipose tissue, and LNs, and assigns state-of-the-art dielectric properties to the tissues. These phantoms may have a great impact in the development process of MWI devices aimed at screening and diagnosing CLNs, and consequently, contribute to a non-invasive staging of the head and neck cancer

Machine Learning/Deep Learning in Medical Image Processing

Many recent studies on medical image processing have involved the use of machine learning (ML) and deep learning (DL). This special issue, “Machine Learning/Deep Learning in Medical Image Processing”, has been launched to provide an opportunity for researchers in the area of medical image processing to highlight recent developments made in their fields with ML/DL. Seven excellent papers that cover a wide variety of medical/clinical aspects are selected in this special issue

Learning from limited labelled data: contributions to weak, few-shot, and unsupervised learning

Tesis por compendio[ES] En la última década, el aprendizaje profundo (DL) se ha convertido en la principal herramienta para las tareas de visión por ordenador (CV). Bajo el paradigma de aprendizaje supervisado, y gracias a la recopilación de grandes conjuntos de datos, el DL ha alcanzado resultados impresionantes utilizando redes neuronales convolucionales (CNNs). Sin embargo, el rendimiento de las CNNs disminuye cuando no se dispone de suficientes datos, lo cual dificulta su uso en aplicaciones de CV en las que sólo se dispone de unas pocas muestras de entrenamiento, o cuando el etiquetado de imágenes es una tarea costosa. Estos escenarios motivan la investigación de estrategias de aprendizaje menos supervisadas. En esta tesis, hemos explorado diferentes paradigmas de aprendizaje menos supervisados. Concretamente, proponemos novedosas estrategias de aprendizaje autosupervisado en la clasificación débilmente supervisada de imágenes histológicas gigapixel. Por otro lado, estudiamos el uso del aprendizaje por contraste en escenarios de aprendizaje de pocos disparos para la vigilancia automática de cruces de ferrocarril. Por último, se estudia la localización de lesiones cerebrales en el contexto de la segmentación no supervisada de anomalías. Asimismo, prestamos especial atención a la incorporación de conocimiento previo durante el entrenamiento que pueda mejorar los resultados en escenarios menos supervisados. En particular, introducimos proporciones de clase en el aprendizaje débilmente supervisado en forma de restricciones de desigualdad. Además, se incorpora la homogeneización de la atención para la localización de anomalías mediante términos de regularización de tamaño y entropía. A lo largo de esta tesis se presentan diferentes métodos menos supervisados de DL para CV, con aportaciones sustanciales que promueven el uso de DL en escenarios con datos limitados. Los resultados obtenidos son prometedores y proporcionan a los investigadores nuevas herramientas que podrían evitar la anotación de cantidades masivas de datos de forma totalmente supervisada.[CA] En l'última dècada, l'aprenentatge profund (DL) s'ha convertit en la principal eina per a les tasques de visió per ordinador (CV). Sota el paradigma d'aprenentatge supervisat, i gràcies a la recopilació de grans conjunts de dades, el DL ha aconseguit resultats impressionants utilitzant xarxes neuronals convolucionals (CNNs). No obstant això, el rendiment de les CNNs disminueix quan no es disposa de suficients dades, la qual cosa dificulta el seu ús en aplicacions de CV en les quals només es disposa d'unes poques mostres d'entrenament, o quan l'etiquetatge d'imatges és una tasca costosa. Aquests escenaris motiven la investigació d'estratègies d'aprenentatge menys supervisades. En aquesta tesi, hem explorat diferents paradigmes d'aprenentatge menys supervisats. Concretament, proposem noves estratègies d'aprenentatge autosupervisat en la classificació feblement supervisada d'imatges histològiques gigapixel. D'altra banda, estudiem l'ús de l'aprenentatge per contrast en escenaris d'aprenentatge de pocs trets per a la vigilància automàtica d'encreuaments de ferrocarril. Finalment, s'estudia la localització de lesions cerebrals en el context de la segmentació no supervisada d'anomalies. Així mateix, prestem especial atenció a la incorporació de coneixement previ durant l'entrenament que puga millorar els resultats en escenaris menys supervisats. En particular, introduïm proporcions de classe en l'aprenentatge feblement supervisat en forma de restriccions de desigualtat. A més, s'incorpora l'homogeneïtzació de l'atenció per a la localització d'anomalies mitjançant termes de regularització de grandària i entropia. Al llarg d'aquesta tesi es presenten diferents mètodes menys supervisats de DL per a CV, amb aportacions substancials que promouen l'ús de DL en escenaris amb dades limitades. Els resultats obtinguts són prometedors i proporcionen als investigadors noves eines que podrien evitar l'anotació de quantitats massives de dades de forma totalment supervisada.[EN] In the last decade, deep learning (DL) has become the main tool for computer vision (CV) tasks. Under the standard supervised learnng paradigm, and thanks to the progressive collection of large datasets, DL has reached impressive results on different CV applications using convolutional neural networks (CNNs). Nevertheless, CNNs performance drops when sufficient data is unavailable, which creates challenging scenarios in CV applications where only few training samples are available, or when labeling images is a costly task, that require expert knowledge. Those scenarios motivate the research of not-so-supervised learning strategies to develop DL solutions on CV. In this thesis, we have explored different less-supervised learning paradigms on different applications. Concretely, we first propose novel self-supervised learning strategies on weakly supervised classification of gigapixel histology images. Then, we study the use of contrastive learning on few-shot learning scenarios for automatic railway crossing surveying. Finally, brain lesion segmentation is studied in the context of unsupervised anomaly segmentation, using only healthy samples during training. Along this thesis, we pay special attention to the incorporation of tasks-specific prior knowledge during model training, which may be easily obtained, but which can substantially improve the results in less-supervised scenarios. In particular, we introduce relative class proportions in weakly supervised learning in the form of inequality constraints. Also, attention homogenization in VAEs for anomaly localization is incorporated using size and entropy regularization terms, to make the CNN to focus on all patterns for normal samples. The different methods are compared, when possible, with their supervised counterparts. In short, different not-so-supervised DL methods for CV are presented along this thesis, with substantial contributions that promote the use of DL in data-limited scenarios. The obtained results are promising, and provide researchers with new tools that could avoid annotating massive amounts of data in a fully supervised manner.The work of Julio Silva Rodríguez to carry out this research and to elaborate this dissertation has been supported by the Spanish Government under the FPI Grant PRE2018-083443.Silva Rodríguez, JJ. (2022). Learning from limited labelled data: contributions to weak, few-shot, and unsupervised learning [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/190633Compendi

Risk factors and biomarkers for metastatic cutaneous squamous cell carcinoma

The incidence of cutaneous squamous cell carcinoma (cSCC), the most common skin cancer with metastatic potential, continues to increase. Although proportion of cSCCs metastasize and cause mortality, sufficient means to identify the metastasis-prone tumors are not available. In this thesis the metastatic cSCCs from the area served by Turku University Hospital were identified and characterized revealing that the rate of metastasis in the study region was 2.3%. Further, it was discovered that metastasis occurs rapidly and that there was no history of cSCC in 85% of patients with metastatic cSCC. Invasion depth, tumor diameter, age and location on lower lip or forehead were associated with increased risk of metastasis. On the other hand, usage of isosorbide mono-/dinitrate and aspirin as well as comorbidity with premalignant lesions or basal cell carcinoma were associated with lower risk of metastasis. With multiplexed immunohistochemistry, it was demonstrated that the activity and phenotype of cancer-associated fibroblasts (CAFs) evolve during the progression of cSCC. Elevation of α-smooth muscle actin (αSMA), secreted protein acidic and rich in cysteine (SPARC) and fibroblast activating protein (FAP) expression was associated with invasion and expression of FAP and platelet-derived growth factor receptor-β (PDGFRβ) with metastasis. High expression of stromal PDGFRβ and periostin were associated with worse prognosis. CAF107 (PDGFRα-/PDGFRβ+/FAP+) subset was associated with invasion and metastasis, and predicted poor prognosis of cSCC. A deep learning algorithm was harnessed to distinguish primary tumors that metastasize rapidly from non-metastatic cSCCs with slide level area under the receiver operating characteristic curve (AUROC) of 0.747 on whole slide images representing primary cSCCs. Furthermore, a risk factor model, that utilized prediction by AI, was created and provided staging systems and comparative risk factor models surpassing classification and prognostivity. These results characterize features associated with the metastasis risk of cSCC and indicate that CAF-markers and AI could provide clinical tools for the metastasis risk assessment and thus improve the prognosis of patient with metastatic cSCC.Etäpesäkkeitä lähettävän okasolusyövän riskitekijät ja biomarkkerit Yleisimmän etäpesäkkeitä lähettävän ihosyövän, okasolusyövän, ilmaantuvuus jatkaa kasvuaan. Vaikka osa okasolusyövistä lähettää etäpesäkkeitä ja aiheuttaa kuolleisuutta, ei etäpesäkkeitä lähettämään tulevien okasolusyöpien tunnistamiseksi ole toistaiseksi riittäviä keinoja. Tässä väitöskirjassa karakterisoitiin Turun yliopistollisen keskussairaalan vastuualueen metastasoituneet okasolusyövät ja osoitettiin että tutkimusalueen okasolusyövistä 2.3% etenee etäpesäkkeitä lähettäväksi. Metastasoituminen tapahtui nopeasti ja valtaosassa tapauksista (85%) etäpesäkkeen lähetti ensimmäinen potilaalla todettu okasolusyöpä. Ikä, kasvaimen invaasiosyvyys, halkaisija ja sijainti alahuulessa tai otsalla yhdistyivät kohonneeseen metastaasiriskiin. Isosorbidinitraatin ja aspiriinin käyttö sekä esiasteiden ja tyvisolusyövän esiintyminen taas liittyivät alentuneeseen metastaasiriskiin. Multiplex-immunohistokemiaa hyödyntäen osoitettin, että syöpään liittyvien fibroblastien (CAF) aktiviteetti ja ilmiasu muuttuu okasolusyövän edetessä. Kohonnut sileälihasaktiini alfan (αSMA), osteonektiinin ja fibroblastia aktivoivan proteiinin (FAP) ilmentyminen liittyi invaasioon ja FAP:n sekä verihiutaleista johdetun kasvutekijäreseptori β:n (PDGFRβ) etäpesäkkeiden lähettämiseen. PDGFRβ:n ja periostiinin ilmentyminen taas yhdistyi huonoon ennusteeseen. CAF107 (PDGFRα-/PDGFRβ+/FAP+) alatyyppi liittyi invaasioon, metastasointiin ja huonoon ennusteeeseen. Etäpesäkkeitä lähettämään tulevien okasolusyöpien tunnistamiseen valjastettu syväoppimisalgoritmi erotti okasolusyöpiä edustavista digitalisoiduista mikroskopiakuvista nopeasti etäpesäkkeitä lähettävät okasolusyövät okasolusyövistä, jotka eivät lähetä etäpesäkkeitä, leiketason AUROC-arvolla 0.747. Tekoälyarviota hyödyntävä riskitekijämalli voitti luokittelujärjestelmät ja kilpailevat riskitekijämallit okasolusyöpien luokittelussa ja ennusteen arvioinnissa. Tulokset antavat lisätietoa metastasoituvan okasolusyövän luonteesta ja osoittavat CAF-markkereiden sekä tekoälyn voivan tarjota kliinisiä työkaluja okasolusyövän metastaasiriskin arviointiin ja täten voivan parantaa etäpesäkkeitä lähettävän okasolusyöpäpotilaan ennustetta tulevaisuudessa

Characterization of vascular heterogeneity of astrocytomas grade 4 for supporting patient prognosis estimation, and treatment response assessment

[ES] Los tumores cerebrales son una de las enfermedades más devastadoras en la actualidad por el importante deterioro cognitivo que sufren los pacientes, la elevada tasa de mortalidad y el mal pronóstico. Los astrocitomas de grado 4 conllevan una supervivencia de cinco años en aproximadamente el 5% de los pacientes diagnosticados, siendo los tumores más agresivos y letales del Sistema Nervioso Central (SNC). Los astrocitomas de grado 4 siguen siendo un problema médico complejo aún sin resolver. A pesar de representar más del 60% de los tumores cerebrales malignos en adultos, estos tumores tienen una baja prevalencia relativa y se consideran una enfermedad huérfana, lo que dificulta el desarrollo de nuevos fármacos o tratamientos que puedan beneficiar a los pacientes. La agresividad de estos tumores se debe a diferentes características, como la fuerte angiogénesis, la necrosis, la microproliferación vascular, la capacidad de invasión e infiltración de las células tumorales y un microambiente inmunológico particular. Además, debido a la rápida progresión de los astrocitomas de grado 4, en la zona de la lesión coexisten diferentes regiones específicas que cambian con el tiempo. Esta naturaleza compleja, junto con la marcada heterogeneidad interpaciente, intratumoral y longitudinal, complica el éxito de un único tratamiento eficaz para todos los pacientes. La imagen de resonancia magnética (MRI) supone una técnica útil para caracterizar la morfología y la vascularidad del tumor. El uso de métodos avanzados y robustos para analizar las imágenes de MR recogidas en las fases iniciales del tratamiento de los pacientes permite la delimitación de las diferentes regiones de los astrocitomas de grado 4, convirtiéndose en herramientas útiles para investigadores, radiólogos y neurocirujanos. Además, el cálculo de biomarcadores vasculares de imagen, como los propuestos en esta tesis, facilitaría la caracterización del tumor, la estimación del pronóstico y los enfoques de tratamiento más personalizados. Esta tesis propone cuatro pilares fundamentales para avanzar en el manejo de los astrocitomas de grado 4. Estos incluyen I) la caracterización multinivel del tumor para mejorar las clasificaciones de los gliomas de alto grado del SNC; II) la búsqueda y desarrollo de biomarcadores robustos para estimar el pronóstico de los pacientes desde el momento prequirúrgico; III) así como para evaluar la respuesta a los tratamientos y la selección de los pacientes que pueden beneficiarse de terapias específicas; y IV) el diseño e implementación de estudios clínicos y protocolos para la recogida de datos a largo plazo de cohortes de pacientes notables a nivel internacional. Para abordar estos cuatro pilares, se ha utilizado un enfoque interdisciplinario que combina el análisis de imágenes médicas, técnicas avanzadas de inteligencia artificial y variables moleculares, histopatológicas y clínicas. En conclusión, hemos abordado la influencia de la heterogeneidad interpaciente e intratumoral del astrocitoma de grado 4 para la caracterización y clasificación del tumor, la estimación del pronóstico del paciente y la predicción de las respuestas al tratamiento. Además, se han diseñado e implementado diferentes estudios clínicos que permiten la recogida de datos multinivel de cohortes internacionales de pacientes con astrocitoma de grado 4.[CA] Els tumors cerebrals són una de les malalties més devastadores en l'actualitat per la important deterioració cognitiva que pateixen els pacients, l'elevada taxa de mortalitat i el mal pronòstic. Els astrocitomes de grau 4 comporten una supervivència de cinc anys en aproximadament el 5% dels pacients diagnosticats, sent els tumors més agressius i letals del Sistema Nerviós Central (SNC). Els astrocitomes de grau 4 continuen sent un problema mèdic complex encara sense resoldre. Malgrat representar més del 60% dels tumors cerebrals malignes en adults, aquests tumors tenen una baixa prevalença relativa i es consideren una malaltia òrfena, la qual cosa dificulta el desenvolupament de nous fàrmacs o tractaments que puguen beneficiar als pacients. L'agressivitat d'aquests tumors es deu a diferents característiques, com la forta angiogènesis, la necrosi, la microproliferació vascular, la capacitat d'invasió i infiltració de les cèl·lules tumorals i un microambient immunològic particular. A més, a causa de la ràpida progressió dels astrocitomes de grau 4, en la zona de la lesió coexisteixen diferents regions específiques que canvien amb el temps. Aquesta naturalesa complexa, juntament amb la marcada heterogeneïtat interpacient, intratumoral i longitudinal fa que es complique l'èxit d'un únic tractament eficaç per a tots els pacients. L'imatge de ressonància magnètica (MRI) suposa una tècnica útil per a caracteritzar la morfologia i la vascularitat del tumor. L'ús de mètodes avançats i robustos per a analitzar les imatges de MR recollides en les fases inicials del tractament dels pacients permet la delimitació de les diferents regions dels astrocitomes de grau 4, convertint-se en eines útils per a investigadors, radiòlegs i neurocirugians. A més, el càlcul de biomarcadors vasculars d'imatge, com els proposats en aquesta tesi, facilitaria la caracterització del tumor, l'estimació del pronòstic i els enfocaments de tractament més personalitzats. Aquesta tesi proposa quatre pilars fonamentals per a avançar en el maneig dels astrocitomes de grau 4. Aquests inclouen I) la caracterització multinivell del tumor per a millorar les classificacions dels gliomes d'alt grau del SNC; II) la cerca i desenvolupament de biomarcadors robustos per a estimar el pronòstic dels pacients des del moment prequirúrgic; III) així com per a avaluar la resposta als tractaments i la selecció dels pacients que poden beneficiar-se de teràpies específiques; i IV) el disseny i implementació d'estudis clínics i protocols per a la recollida de dades a llarg termini de cohorts de pacients notables a nivell internacional. Per a abordar aquests quatre pilars, s'ha utilitzat un enfocament interdisciplinari que combina l'anàlisi d'imatges mèdiques, tècniques avançades d'intel·ligència artificial i variables moleculars, histopatològiques i clíniques. En conclusió, hem abordat la influència de l'heterogeneïtat interpacient i intratumoral del astrocitoma de grau 4 per a la caracterització i classificació del tumor, l'estimació del pronòstic del pacient i la predicció de les respostes al tractament. A més, s'han dissenyat i implementat diferents estudis clínics que permeten la recollida de dades multinivell de cohorts internacionals de pacients amb astrocitoma de grau 4.[EN] Brain tumors are one of the most devastating diseases today because of the significant cognitive impairment suffered by patients, high mortality rates, and poor prognosis. Astrocytomas grade 4 bring five-year survival in approximately 5% of diagnosed patients, being the most aggressive and lethal tumors of the Central Nervous System (CNS). Astrocytomas grade 4 continue to be an unresolved complex medical problem. Despite accounting for more than 60% of malignant brain tumors in adults, these tumors have a low relative prevalence and are considered an orphan disease, making difficult developing new drugs or treatments that might benefit patients. The aggressiveness of these tumors is due to different characteristics, such as strong angiogenesis, necrosis, vascular microproliferation, the capacity of the tumor cells to invade and infiltrate, and a particular immune microenvironment. In addition, due to the rapid progression of astrocytomas grade 4, different specific regions coexist in the lesion area which change over time. This complex nature, along with the marked interpatient, intratumor, and longitudinal heterogeneity, makes complicate the success of a single efficient treatment for all patients. Magnetic Resonance Imaging (MRI) represents a useful technique to characterize tumor morphology and vascularity. Using advanced and robust methods to analyze MR images collected from initial stages of patient management allows the delineation of different regions of astrocytomas grade 4, becoming useful tools for researchers, radiologists and neurosurgeons. In addition, the calculation of imaging vascular biomarkers, such as those proposed in this thesis, would facilitate tumor characterization, prognosis estimation and more personalized treatment approaches. This thesis proposes four fundamental pillars to advance the management of astrocytomas grade 4. These include I) the multilevel characterization of the tumor to improve classifications of high-grade CNS gliomas; II) the search and development of robust biomarkers for estimating patient prognosis from the presurgical moment; III) as well as for evaluating the response to treatments and the selection of patients who may benefit from specific therapies; and IV) the design and implementation of clinical studies and protocols for long-term collecting data from internationally remarkable cohorts of patients. To address these four pillars, an interdisciplinary approach has been used that combines medical imaging analysis, advanced artificial intelligence techniques, and molecular, histopathological, and clinical variables. Concluding, we have addressed the influence of both interpatient and intratumor heterogeneity of astrocytoma grade 4 for tumor characterization and classification, patient prognosis estimation and predicting treatment responses. In addition, different clinical studies have been designed and implemented allowing the collection of multilevel data from international cohorts of patients with astrocytoma grade 4.Álvarez Torres, MDM. (2022). Characterization of vascular heterogeneity of astrocytomas grade 4 for supporting patient prognosis estimation, and treatment response assessment [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/18895