Deep learning interpretability methods for the classification of blood cell images

Durant l’última dècada, el sector mèdic ha adoptat les xarxes neuronals com a eina per ajudar a diagnosticar i comprendre diferents malalties, degut a la seva elevada precisió i versatilitat. No obstant, la seva integració al flux de treball dels patòlegs s'ha vist greument afectada per la naturalesa “Black-Box” que presenten aquests models. Els complexos conceptes matemàtics i estadístics en què es basen aquests models, dificulten enormement la comprensió directa dels criteris de decisió en el qual es basen per fer les seves prediccions. La interpretabilitat de xarxes neuronal té com a objectiu proporcionar explicacions en termes comprensibles a un ésser humà. En aquest projecte, es duu a terme un estudi d’interpretabilitat a la xarxa DisplasiaNet, una xarxa neuronal convolucional especialment optimitzada per classificar les imatges de neutròfils sanguinis perifèrics en Normals o Displastics. Treballant estretament amb patòlegs i amb l’ajut d’una aplicació d’anotacions web construïda a propòsit, s’extreuen les principals característiques morfològiques dels diferents estats cel·lulars. En paral·lel, s’apliquen tècniques d’interpretabilitat d’imatges a la xarxa DisplasiNet, com ara mapes de saliència, mapes d’activació de classes i mapes de sensibilitat envers l’oclusió, per obtenir les caracteristiques que el model considera més rellevants. L'estudi ha descobert que DisplasiaNet detecta displàsia en neutròfils de manera similar als patòlegs, validant així la seva precisió. En primer lloc, es centra en la granularitat del citoplasma i, en segon lloc, en la densitat cromatínica del nucli i la segmentació lobular.Durante la última década, el sector médico ha adoptado ampliamente las redes neuronales como una herramienta para ayudar a diagnosticar y comprender diferentes enfermedades. Sin embargo, su integración en el flujo de trabajo de los patólogos se ha visto gravemente afectado debido a la naturaleza “Black-Box” que presentan estos modelos. Los complejos conceptos matemáticos y estadísticos en los que se basan estos modelos dificultan enormemente la comprensión directa de los criterios decisivos que el modelo emplea para realizar predicciones. La interpretabilidad de redes neuronales tiene como objetivo proporcionar explicaciones en términos comprensibles para un ser humano. En este proyecto, se lleva a cabo un estudio de interpretabilidad de de la red nuronal DisplasiaNet, una red convolucional especialmente optimizada para clasificar imágenes de neutrófilos de sangre periférica en displásicas o normales. Trabajando en estrecha colaboración con patólogos expertos y con la ayuda de una aplicación de anotación web expresamente diseñada, se extraen las principales características morfológicas que presentan los diferentes estados celulares. En paralelo se aplican a DisplasiaNet técnicas de Interpretabilidad de redes neuronales especializadas en el analisis de imágenes tales como Mapas de relevancia, Mapas de activación de clases y Mapas de sensibilidad de oclusión para obtener las características que el modelo considera más relevantes. El estudio ha encontrado que DisplasiaNet detecta displasia en neutrófilos de manera similar a los patólogos expertos, validando así su precisión. En primer lugar, se centra en la granularidad del citoplasma y, en segundo lugar, en la densidad cromatínica del núcleo y la segmentación lobular.During the past decade, the Medical Sector has widely adopted Neural Networks as a tool to help diagnose and to further understand different diseases. This is due to their proven high accuracy and versatility. However, its integration into the pathologists' workflow has been severely affected due to the black box nature these models present. The complex mathematical and statistical concepts these models are based on greatly hinder the direct understanding of the model's decision criteria when these perform predictions. Neural Network Interpretability aims to provide explanations in understandable terms to a human. In this project, a deep learning interpretability study is carried out on DisplasiaNet, a Convolutional Neural Network specially optimized to classify Peripheral Blood Neutrophil images into Dysplastic or Normal. Working closely with expert pathologists and with the help of a purposely built web annotation app, the main morphological characteristics of the different cell states are extracted. Image interpretability techniques such as Saliency Maps, Class Activation Maps, and Occlusion Sensitivity Maps are applied to DisplasiaNet to obtain the features the model considers the most relevant. The study has found that DisplasiaNet detects dysplasia in Neutrophils in a similar manner to expert pathologists, thus validating its accuracy. Firstly it focuses on the granularity of the cytoplasm, and secondly on the nucleus chromatinic density and lobular segmentation

Data Enrichment for Data Mining Applied to Bioinformatics and Cheminformatics Domains

Problemas cada vez mais complexos estão a ser tratados na àrea das ciências da vida. A aquisição de todos os dados que possam estar relacionados com o problema em questão é primordial. Igualmente importante é saber como os dados estão relacionados uns com os outros e com o próprio problema. Por outro lado, existem grandes quantidades de dados e informações disponíveis na Web. Os investigadores já estão a utilizar Data Mining e Machine Learning como ferramentas valiosas nas suas investigações, embora o procedimento habitual seja procurar a informação baseada nos modelos indutivos. Até agora, apesar dos grandes sucessos já alcançados com a utilização de Data Mining e Machine Learning, não é fácil integrar esta vasta quantidade de informação disponível no processo indutivo, com algoritmos proposicionais. A nossa principal motivação é abordar o problema da integração de informação de domínio no processo indutivo de técnicas proposicionais de Data Mining e Machine Learning, enriquecendo os dados de treino a serem utilizados em sistemas de programação de lógica indutiva. Os algoritmos proposicionais de Machine Learning são muito dependentes dos atributos dos dados. Ainda é difícil identificar quais os atributos mais adequados para uma determinada tarefa na investigação. É também difícil extrair informação relevante da enorme quantidade de dados disponíveis. Vamos concentrar os dados disponíveis, derivar características que os algoritmos de ILP podem utilizar para induzir descrições, resolvendo os problemas. Estamos a criar uma plataforma web para obter informação relevante para problemas de Bioinformática (particularmente Genómica) e Quimioinformática. Esta vai buscar os dados a repositórios públicos de dados genómicos, proteicos e químicos. Após o enriquecimento dos dados, sistemas Prolog utilizam programação lógica indutiva para induzir regras e resolver casos específicos de Bioinformática e Cheminformática. Para avaliar o impacto do enriquecimento dos dados com ILP, comparamos com os resultados obtidos na resolução dos mesmos casos utilizando algoritmos proposicionais.Increasingly more complex problems are being addressed in life sciences. Acquiring all the data that may be related to the problem in question is paramount. Equally important is to know how the data is related to each other and to the problem itself. On the other hand, there are large amounts of data and information available on the Web. Researchers are already using Data Mining and Machine Learning as a valuable tool in their researches, albeit the usual procedure is to look for the information based on induction models. So far, despite the great successes already achieved using Data Mining and Machine Learning, it is not easy to integrate this vast amount of available information in the inductive process with propositional algorithms. Our main motivation is to address the problem of integrating domain information into the inductive process of propositional Data Mining and Machine Learning techniques by enriching the training data to be used in inductive logic programming systems. The algorithms of propositional machine learning are very dependent on data attributes. It still is hard to identify which attributes are more suitable for a particular task in the research. It is also hard to extract relevant information from the enormous quantity of data available. We will concentrate the available data, derive features that ILP algorithms can use to induce descriptions, solving the problems. We are creating a web platform to obtain relevant bioinformatics (particularly Genomics) and Cheminformatics problems. It fetches the data from public repositories with genomics, protein and chemical data. After the data enrichment, Prolog systems use inductive logic programming to induce rules and solve specific Bioinformatics and Cheminformatics case studies. To assess the impact of the data enrichment with ILP, we compare with the results obtained solving the same cases using propositional algorithms

Improving OCR Post Processing with Machine Learning Tools

Optical Character Recognition (OCR) Post Processing involves data cleaning steps for documents that were digitized, such as a book or a newspaper article. One step in this process is the identification and correction of spelling and grammar errors generated due to the flaws in the OCR system. This work is a report on our efforts to enhance the post processing for large repositories of documents. The main contributions of this work are: • Development of tools and methodologies to build both OCR and ground truth text correspondence for training and testing of proposed techniques in our experiments. In particular, we will explain the alignment problem and tackle it with our de novo algorithm that has shown a high success rate. • Exploration of the Google Web 1T corpus to correct errors using context. We show that over half of the errors in the OCR text can be detected and corrected. • Applications of machine learning tools to generalize the past ad hoc approaches to OCR error corrections. As an example, we investigate the use of logistic regression to select the correct replacement for misspellings in the OCR text. • Use of container technology to address the state of reproducible research in OCR and Computer Science as a whole. Many of the past experiments in the field of OCR are not considered reproducible research questioning whether the original results were outliers or finessed

Reutilización de técnicas de XAI para Sistemas de IA personalizados

Trabajo de Fin de Grado en Ingeniería Informática, Facultad de Informática UCM, Departamento de Ingeniería del Software e Inteligencia Artificial, Curso 2021/2022.Interpretability has become a crucial aspect in the field of Artificial Intelligence. Although there are many Explainable AI (XAI) techniques that provide explanations and improve the interpretability of AI systems, there are certain limitations. One of the most important limitations is that it is necessary to have a deep understanding of XAI to determine which techniques are better to explain a specific AI model to a specific user. In this project, I reviewed some of the most well-known XAI libraries and methods. Then, I used these tools to build an API that unifies model-agnostic methods used to explain different types of models. Using the API as a basis, I built a case-based reasoning system that aims to recommend the best explanation techniques to users based on the feedback of previous users with similar background knowledge and AI models. The CBR system case base was populated by using real-user feedback on explanations for a set of use cases. Finally, I evaluate the case base and the performance of the system.La interpretabilidad se ha vuelto un aspecto crucial en el campo de la Inteligencia Artificial. Aunque existen muchas técnicas de Inteligencia Artificial Explicable (XAI) que proporcionan explicaciones y mejoran la interpretabilidad de sistemas de IA, existen ciertas limitaciones. Una las limitaciones más importantes es la necesidad de tener un conocimiento profundo en XAI para determinar cuáles son los mejores métodos para explicar un modelo de IA específico a un usuario específico. En este trabajo hago una revisión de algunas de las librerías y métodos de XAI más conocidos. Luego, utilizando estas herramientas, desarrollo una API que unifica métodos agnósticos para explicar diferentes tipos de modelos. Utilizando dicha API como base, construyo un sistema de razonamiento basado en casos que busca recomendar las mejores técnicas de explicación a los usuarios, basándose en la evaluación previa de otros usuarios con conocimientos y modelos de IA similares. La base de casos del sistema CBR se generó utilizando la evaluación de usuarios reales acerca de explicaciones generadas sobre un conjunto de casos de uso. Finalmente, evalúo la base de casos y el rendimiento del sistema.Depto. de Ingeniería de Software e Inteligencia Artificial (ISIA)Fac. de InformáticaTRUEunpu

Shopbot: An Image Based Search Application for E-Commerce Domain

For the past few years, e-commerce has changed the way people buy and sell products. People use this business model to do business over the Internet. In this domain, Human-Computer Interaction has been gaining momentum. Lately, there has been an upsurge in agent based applications in the form of intelligent personal assistants (also known as Chatbots) which make it easier for users to interact with digital services via a conversation, in the same way we talk to humans. In e- commerce, these assistants offer mainly text-based or speech based search capabilities. They can handle search for most products, but cannot handle search that is based on product features, for instance color or pattern of a T-shirt. Most of the times, it is difficult for users to define these characteristics while searching for a product. Furthermore, a growing number of consumers rely on social media to make a purchasing decision. They try to find out what is trending right now and look for similar items. This brings us the need of a virtual shopping assistant or a shopbot which recommends products based on an image of the product provided by a user. It will be designed to provide relevant responses to the user queries by performing image recognition. This report explains the proposed approach along with the implementation for the virtual shopping assistant

Shopbot: An Image Based Search Application for E-Commerce Domain

For the past few years, e-commerce has changed the way people buy and sell products. People use this business model to do business over the Internet. In this domain, Human-Computer Interaction has been gaining momentum. Lately, there has been an upsurge in agent based applications in the form of intelligent personal assistants (also known as Chatbots) which make it easier for users to interact with digital services via a conversation, in the same way we talk to humans. In e- commerce, these assistants offer mainly text-based or speech based search capabilities. They can handle search for most products, but cannot handle search that is based on product features, for instance color or pattern of a T-shirt. Most of the times, it is difficult for users to define these characteristics while searching for a product. Furthermore, a growing number of consumers rely on social media to make a purchasing decision. They try to find out what is trending right now and look for similar items. This brings us the need of a virtual shopping assistant or a shopbot which recommends products based on an image of the product provided by a user. It will be designed to provide relevant responses to the user queries by performing image recognition. This report explains the proposed approach along with the implementation for the virtual shopping assistant