Discovering a Domain Knowledge Representation for Image Grouping: Multimodal Data Modeling, Fusion, and Interactive Learning

In visually-oriented specialized medical domains such as dermatology and radiology, physicians explore interesting image cases from medical image repositories for comparative case studies to aid clinical diagnoses, educate medical trainees, and support medical research. However, general image classification and retrieval approaches fail in grouping medical images from the physicians\u27 viewpoint. This is because fully-automated learning techniques cannot yet bridge the gap between image features and domain-specific content for the absence of expert knowledge. Understanding how experts get information from medical images is therefore an important research topic. As a prior study, we conducted data elicitation experiments, where physicians were instructed to inspect each medical image towards a diagnosis while describing image content to a student seated nearby. Experts\u27 eye movements and their verbal descriptions of the image content were recorded to capture various aspects of expert image understanding. This dissertation aims at an intuitive approach to extracting expert knowledge, which is to find patterns in expert data elicited from image-based diagnoses. These patterns are useful to understand both the characteristics of the medical images and the experts\u27 cognitive reasoning processes. The transformation from the viewed raw image features to interpretation as domain-specific concepts requires experts\u27 domain knowledge and cognitive reasoning. This dissertation also approximates this transformation using a matrix factorization-based framework, which helps project multiple expert-derived data modalities to high-level abstractions. To combine additional expert interventions with computational processing capabilities, an interactive machine learning paradigm is developed to treat experts as an integral part of the learning process. Specifically, experts refine medical image groups presented by the learned model locally, to incrementally re-learn the model globally. This paradigm avoids the onerous expert annotations for model training, while aligning the learned model with experts\u27 sense-making

Knowledge-Informed Machine Learning for Cancer Diagnosis and Prognosis: A review

Cancer remains one of the most challenging diseases to treat in the medical field. Machine learning has enabled in-depth analysis of rich multi-omics profiles and medical imaging for cancer diagnosis and prognosis. Despite these advancements, machine learning models face challenges stemming from limited labeled sample sizes, the intricate interplay of high-dimensionality data types, the inherent heterogeneity observed among patients and within tumors, and concerns about interpretability and consistency with existing biomedical knowledge. One approach to surmount these challenges is to integrate biomedical knowledge into data-driven models, which has proven potential to improve the accuracy, robustness, and interpretability of model results. Here, we review the state-of-the-art machine learning studies that adopted the fusion of biomedical knowledge and data, termed knowledge-informed machine learning, for cancer diagnosis and prognosis. Emphasizing the properties inherent in four primary data types including clinical, imaging, molecular, and treatment data, we highlight modeling considerations relevant to these contexts. We provide an overview of diverse forms of knowledge representation and current strategies of knowledge integration into machine learning pipelines with concrete examples. We conclude the review article by discussing future directions to advance cancer research through knowledge-informed machine learning.Comment: 41 pages, 4 figures, 2 table

Expertise effects in memory recall: A reply to Vicente and Wang

This article may not exactly replicate the final version published in the APA journal. It is not the copy of record.In the January 1998 Psychological Review, Vicente and Wang propose a "constraint attunement hypothesis" to explain the large effects of domain expertise upon memory recall observed in a number of task domains. They claim to find serious defects in alternative explanations of these effects which their theory overcomes. Re-examination of the evidence shows that their theory is not novel, but has been anticipated by those they criticize, and that other current published theories of the phenomena do not have the defects Vicente and Wang attribute to them. Vicente and Wang's views reflect underlying differences (a) about emphasis upon performance versus process in psychology, and (b) about how theories and empirical knowledge interact and progress with the development of a science

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

ICE: Enabling Non-Experts to Build Models Interactively for Large-Scale Lopsided Problems

Quick interaction between a human teacher and a learning machine presents numerous benefits and challenges when working with web-scale data. The human teacher guides the machine towards accomplishing the task of interest. The learning machine leverages big data to find examples that maximize the training value of its interaction with the teacher. When the teacher is restricted to labeling examples selected by the machine, this problem is an instance of active learning. When the teacher can provide additional information to the machine (e.g., suggestions on what examples or predictive features should be used) as the learning task progresses, then the problem becomes one of interactive learning. To accommodate the two-way communication channel needed for efficient interactive learning, the teacher and the machine need an environment that supports an interaction language. The machine can access, process, and summarize more examples than the teacher can see in a lifetime. Based on the machine's output, the teacher can revise the definition of the task or make it more precise. Both the teacher and the machine continuously learn and benefit from the interaction. We have built a platform to (1) produce valuable and deployable models and (2) support research on both the machine learning and user interface challenges of the interactive learning problem. The platform relies on a dedicated, low-latency, distributed, in-memory architecture that allows us to construct web-scale learning machines with quick interaction speed. The purpose of this paper is to describe this architecture and demonstrate how it supports our research efforts. Preliminary results are presented as illustrations of the architecture but are not the primary focus of the paper

Online Robot Introspection via Wrench-based Action Grammars

Robotic failure is all too common in unstructured robot tasks. Despite well-designed controllers, robots often fail due to unexpected events. How do robots measure unexpected events? Many do not. Most robots are driven by the sense-plan act paradigm, however more recently robots are undergoing a sense-plan-act-verify paradigm. In this work, we present a principled methodology to bootstrap online robot introspection for contact tasks. In effect, we are trying to enable the robot to answer the question: what did I do? Is my behavior as expected or not? To this end, we analyze noisy wrench data and postulate that the latter inherently contains patterns that can be effectively represented by a vocabulary. The vocabulary is generated by segmenting and encoding the data. When the wrench information represents a sequence of sub-tasks, we can think of the vocabulary forming a sentence (set of words with grammar rules) for a given sub-task; allowing the latter to be uniquely represented. The grammar, which can also include unexpected events, was classified in offline and online scenarios as well as for simulated and real robot experiments. Multiclass Support Vector Machines (SVMs) were used offline, while online probabilistic SVMs were are used to give temporal confidence to the introspection result. The contribution of our work is the presentation of a generalizable online semantic scheme that enables a robot to understand its high-level state whether nominal or abnormal. It is shown to work in offline and online scenarios for a particularly challenging contact task: snap assemblies. We perform the snap assembly in one-arm simulated and real one-arm experiments and a simulated two-arm experiment. This verification mechanism can be used by high-level planners or reasoning systems to enable intelligent failure recovery or determine the next most optima manipulation skill to be used.Comment: arXiv admin note: substantial text overlap with arXiv:1609.0494

Characterization of age-related ocular diseases in OCT images through deep learning techniques

[Abstract]: Age-Related Macular Degeneration (AMD) is the main cause of severe visual impairment and blindness in Europe, and its prevalence is expected to increase worldwide due to population aging. Optical Coherence Tomography (OCT) is a noninvasive retinal imaging technique that has become the standard of care in the diagnosis and monitoring of late AMD, where the great majority of severe symptoms are manifested. Neovascular late AMD, where new pathological blood vessels are formed that may leak fluid, often results in relatively rapid vision loss. Treatment exists for neovascular AMD, such that its detection and characterization plays a key role in patient outcomes. This project applies deep learning techniques to the task of AMD characterization. To do so, a data set of OCT scans labeled as to the presence of fluid and neovascularisation is used to train deep convolutional networks. Analysis of this initial experiment produced two hypotheses of performance limiting factors: intra-expert variability and data scarcity. The former was addressed through the development of a machine-assisted review process based on the Class Activation Mapping (CAM) interpretability technique. A small blinded trial was favorable to the methodology. The latter resulted in the adaptation of a large public data set to explore domain-specific transfer learning.[Resumo]: A Dexeneración Macular Asociada á Idade (DMAI) é a principal causa de discapacidade visual severa e cegueira en Europa, e espérase que a súa prevalencia aumente a nivel mundial debido ó envellecemento poboacional. A Tomografía de Coherencia Óptica (TCO) é un método non invasivo de imaxe retiniana que se converteu no estándar no diagnóstico e monitorización da DMAI tardía, onde se manifestan a maioría de síntomas graves. A DMAI tardía neovascular, onde se forman novos vasos sanguíneos patolóxicos que poden derramar fluído, a miúdo resulta en perda de visión de forma relativamente repentina. Existen tratamentos para a DMAI neovascular, de modo que a súa detección e caracterización xoga un papel crucial no prognóstico dos pacientes. Este proxecto aplica técnicas de aprendizaxe profunda á tarefa de caracterización de DMAI. Con ese fin, un conxunto de datos de TCO anotado en base á presenza de fluído e neovascularización foi empregado para entrenar redes convolucionais profundas. A análise deste experimento inicial produciu dúas hipóteses sobre factores que limitan o rendemento: a variabilidade intra-experto e a escaseza de datos. O primeiro foi afrontado mediante o desenvolvemento dun proceso de revisión de anotacións asistido por computadora, baseado na técnica de interpretabilidade Class Activation Mapping (CAM). Un pequeno estudo cego foi favorable á metodoloxía. A segunda hipótese resultou na adaptación dun gran conxunto de datos público para a exploración de transferencia de aprendizaxe específica ó dominio.Traballo fin de grao (UDC.FIC). Enxeñaría Informática. Curso 2021/202