Cascaded Multi-View Canonical Correlation (CaMCCo) for Early Diagnosis of Alzheimer\u27s Disease via Fusion of Clinical, Imaging and Omic Features

The introduction of mild cognitive impairment (MCI) as a diagnostic category adds to the challenges of diagnosing Alzheimer\u27s Disease (AD). No single marker has been proven to accurately categorize patients into their respective diagnostic groups. Thus, previous studies have attempted to develop fused predictors of AD and MCI. These studies have two main limitations. Most do not simultaneously consider all diagnostic categories and provide suboptimal fused representations using the same set of modalities for prediction of all classes. In this work, we present a combined framework, cascaded multiview canonical correlation (CaMCCo), for fusion and cascaded classification that incorporates all diagnostic categories and optimizes classification by selectively combining a subset of modalities at each level of the cascade. CaMCCo is evaluated on a data cohort comprising 149 patients for whom neurophysiological, neuroimaging, proteomic and genomic data were available. Results suggest that fusion of select modalities for each classification task outperforms (mean AUC = 0.92) fusion of all modalities (mean AUC = 0.54) and individual modalities (mean AUC = 0.90, 0.53, 0.71, 0.73, 0.62, 0.68). In addition, CaMCCo outperforms all other multi-class classification methods for MCI prediction (PPV: 0.80 vs. 0.67, 0.63)

Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis

Multimodal language analysis often considers relationships between features based on text and those based on acoustical and visual properties. Text features typically outperform non-text features in sentiment analysis or emotion recognition tasks in part because the text features are derived from advanced language models or word embeddings trained on massive data sources while audio and video features are human-engineered and comparatively underdeveloped. Given that the text, audio, and video are describing the same utterance in different ways, we hypothesize that the multimodal sentiment analysis and emotion recognition can be improved by learning (hidden) correlations between features extracted from the outer product of text and audio (we call this text-based audio) and analogous text-based video. This paper proposes a novel model, the Interaction Canonical Correlation Network (ICCN), to learn such multimodal embeddings. ICCN learns correlations between all three modes via deep canonical correlation analysis (DCCA) and the proposed embeddings are then tested on several benchmark datasets and against other state-of-the-art multimodal embedding algorithms. Empirical results and ablation studies confirm the effectiveness of ICCN in capturing useful information from all three views

Computer extracted gland features from H&E predicts prostate cancer recurrence comparably to a genomic companion diagnostic test: a large multi-site study

Existing tools for post-radical prostatectomy (RP) prostate cancer biochemical recurrence (BCR) prognosis rely on human pathologist-derived parameters such as tumor grade, with the resulting inter-reviewer variability. Genomic companion diagnostic tests such as Decipher tend to be tissue destructive, expensive, and not routinely available in most centers. We present a tissue non-destructive method for automated BCR prognosis, termed "Histotyping", that employs computational image analysis of morphologic patterns of prostate tissue from a single, routinely acquired hematoxylin and eosin slide. Patients from two institutions (n = 214) were used to train Histotyping for identifying high-risk patients based on six features of glandular morphology extracted from RP specimens. Histotyping was validated for post-RP BCR prognosis on a separate set of n = 675 patients from five institutions and compared against Decipher on n = 167 patients. Histotyping was prognostic of BCR in the validation set (p < 0.001, univariable hazard ratio [HR] = 2.83, 95% confidence interval [CI]: 2.03-3.93, concordance index [c-index] = 0.68, median years-to-BCR: 1.7). Histotyping was also prognostic in clinically stratified subsets, such as patients with Gleason grade group 3 (HR = 4.09) and negative surgical margins (HR = 3.26). Histotyping was prognostic independent of grade group, margin status, pathological stage, and preoperative prostate-specific antigen (PSA) (multivariable p < 0.001, HR = 2.09, 95% CI: 1.40-3.10, n = 648). The combination of Histotyping, grade group, and preoperative PSA outperformed Decipher (c-index = 0.75 vs. 0.70, n = 167). These results suggest that a prognostic classifier for prostate cancer based on digital images could serve as an alternative or complement to molecular-based companion diagnostic tests

Discriminative Representations for Heterogeneous Images and Multimodal Data

Histology images of tumor tissue are an important diagnostic and prognostic tool for pathologists. Recently developed molecular methods group tumors into subtypes to further guide treatment decisions, but they are not routinely performed on all patients. A lower cost and repeatable method to predict tumor subtypes from histology could bring benefits to more cancer patients. Further, combining imaging and genomic data types provides a more complete view of the tumor and may improve prognostication and treatment decisions. While molecular and genomic methods capture the state of a small sample of tumor, histological image analysis provides a spatial view and can identify multiple subtypes in a single tumor. This intra-tumor heterogeneity has yet to be fully understood and its quantification may lead to future insights into tumor progression. In this work, I develop methods to learn appropriate features directly from images using dictionary learning or deep learning. I use multiple instance learning to account for intra-tumor variations in subtype during training, improving subtype predictions and providing insights into tumor heterogeneity. I also integrate image and genomic features to learn a projection to a shared space that is also discriminative. This method can be used for cross-modal classification or to improve predictions from images by also learning from genomic data during training, even if only image data is available at test time.Doctor of Philosoph

Multi-Level Canonical Correlation Analysis for Standard-Dose PET Image Estimation

Positron emission tomography (PET) images are widely used in many clinical applications such as tumor detection and brain disorder diagnosis. To obtain PET images of diagnostic quality, a sufficient amount of radioactive tracer has to be injected into a living body, which will inevitably increase the risk of radiation exposure. On the other hand, if the tracer dose is considerably reduced, the quality of the resulting images would be significantly degraded. It is of great interest to estimate a standard-dose PET (S-PET) image from a low-dose one in order to reduce the risk of radiation exposure and preserve image quality. This may be achieved through mapping both standard-dose and low-dose PET data into a common space and then performing patch based sparse representation. However, a one-size-fits-all common space built from all training patches is unlikely to be optimal for each target S-PET patch, which limits the estimation accuracy. In this paper, we propose a data-driven multi-level Canonical Correlation Analysis (mCCA) scheme to solve this problem. Specifically, a subset of training data that is most useful in estimating a target S-PET patch is identified in each level, and then used in the next level to update common space and improve estimation. Additionally, we also use multi-modal magnetic resonance images to help improve the estimation with complementary information. Validations on phantom and real human brain datasets show that our method effectively estimates S-PET images and well preserves critical clinical quantification measures, such as standard uptake value

Multi-views Fusion CNN for Left Ventricular Volumes Estimation on Cardiac MR Images

Left ventricular (LV) volumes estimation is a critical procedure for cardiac disease diagnosis. The objective of this paper is to address direct LV volumes prediction task. Methods: In this paper, we propose a direct volumes prediction method based on the end-to-end deep convolutional neural networks (CNN). We study the end-to-end LV volumes prediction method in items of the data preprocessing, networks structure, and multi-views fusion strategy. The main contributions of this paper are the following aspects. First, we propose a new data preprocessing method on cardiac magnetic resonance (CMR). Second, we propose a new networks structure for end-to-end LV volumes estimation. Third, we explore the representational capacity of different slices, and propose a fusion strategy to improve the prediction accuracy. Results: The evaluation results show that the proposed method outperforms other state-of-the-art LV volumes estimation methods on the open accessible benchmark datasets. The clinical indexes derived from the predicted volumes agree well with the ground truth (EDV: R2=0.974, RMSE=9.6ml; ESV: R2=0.976, RMSE=7.1ml; EF: R2=0.828, RMSE =4.71%). Conclusion: Experimental results prove that the proposed method may be useful for LV volumes prediction task. Significance: The proposed method not only has application potential for cardiac diseases screening for large-scale CMR data, but also can be extended to other medical image research fieldsComment: to appear on Transactions on Biomedical Engineerin

Real-time feature selection technique with concept drift detection using adaptive micro-clusters for data stream mining

Data streams are unbounded, sequential data instances that are generated with high Velocity. Classifying sequential data instances is a very challenging problem in machine learning with applications in network intrusion detection, financial markets and applications requiring real-time sensor-networks-based situation assessment. Data stream classification is concerned with the automatic labelling of unseen instances from the stream in real-time. For this the classifier needs to adapt to concept drifts and can only have a single pass through the data if the stream is fast moving. This research paper presents work on a real-time pre-processing technique, in particular feature tracking. The feature tracking technique is designed to improve Data Stream Mining (DSM) classification algorithms by enabling and optimising real-time feature selection. The technique is based on tracking adaptive statistical summaries of the data and class label distributions, known as Micro-Clusters. Currently the technique is able to detect concept drifts and identify which features have been influential in the drift

Prediction methods for statistical inference in graph signal processing

This thesis studies the problem of inferring topology from signal graphs. For this reason, the Master's Thesis is part of the current of thought, growing in recent years, in which the structure of the network is not assumed to be known. The problem of inferring topology is approached from two angles. The first one, studies how to find the structure of a graph from spectral templates which can be noisy. Thus, from observations of the network, the spectral template of the graph that makes up the network is inferred. In previous works, like my Degree's Thesis, the algorithms for the inference of incomplete spectral templates were studied. In this Master's thesis, we go one step further by demonstrating why the techniques studied do not always work and proposing an algorithm based on LASSO to infer the network topology when the spectral templates are noisy. The proposed algorithm is compared with those previously studied obtaining better results in terms of RMSE and reliability. The second point of view addressed in this thesis is the inference of the network from statistical techniques. It is common to find networks whose nodes have some relation. These techniques are based on, from some observations of the network, trying to find the existing relationships between the different nodes of the graph. These techniques can be used in a more generic way than those based on spectral templates. Statistical methods are studied in more depth in this Master's Thesis. Initially, the Pearson correlation coefficient is explained. After studying it, some limitations are found. Thus, a new approach is proposed based on the conditional covariance. Then, it is assumed that the signals follow a Gaussian distribution which brings us to study the Maximum Likelihood estimator while considering the graph's sparsity. Although, the previous approach was improved, we are interested in finding even a better one. Hence, we study an approach based on linear regression. In this last algorithm, we include a term to promote sparsity when finding the solution. To conclude, the statistical methods studied, are compared by performing some simulations. By performing these simulations, it is observed that the best technique to infer the graph's topology is the one based on linear regression.Esta tesis estudia el problema de inferir la topología de la red a partir de las señales grafo. Por esta razón, la Tesis Final de Máster se inscribe en la corriente actual de pensamiento, creciente en los últimos años, en la que se supone no conocida la estructura de la red. El problema de la inferencia de la topología se aborda desde dos ángulos. El primero, estudia cómo encontrar la estructura de un grafo a partir de plantillas espectrales que pueden ser ruidosas o no. Así, a partir de las observaciones, se infiere la plantilla espectral del grafo que compone la red. En trabajos anteriores, como mi Tesis de Final de Grado, se estudiaron los algoritmos para la inferencia de plantillas espectrales incompletas. En esta trabajo, vamos un paso más allá demostrando por qué las técnicas estudiadas no siempre funcionan. Seguimos, proponiendo un algoritmo basado en LASSO para inferir la topología de la red cuando las plantillas espectrales son ruidosas. El algoritmo propuesto se compara con los anteriormente estudiados obteniendo mejores resultados en términos de RMSE y fiabilidad. El segundo punto de vista abordado en esta tesis es la inferencia de la red a partir de técnicas estadísticas. Es común encontrar redes cuyos nodos tienen alguna relación entre ellos. Estas técnicas se basan, a partir de algunas observaciones de la red, en tratar de encontrar las relaciones existentes entre los diferentes nodos del grafo. Estas técnicas pueden ser utilizadas de manera más genérica que las basadas en plantillas espectrales. Por ese motivo, los métodos estadísticos se estudian con más profundidad en esta Tesis de Máster. Inicialmente, se explica el coeficiente de correlación de Pearson. Después de estudiarlo, se detecta una limitación. Por ese motivo, se propone un nuevo enfoque basado en la covarianza condicional. Luego, se asume que las señales siguen una distribución Gaussiana, lo que nos lleva a estudiar el estimador de Máxima Verosimilitud, también considerando que la matriz solución es dispersa. Aunque con esta técnica se mejora el enfoque anterior, estamos interesados en encontrar una técnica aún mejor. Por lo tanto, estudiamos un enfoque basado en la regresión lineal. En este último algoritmo, incluimos un término para promover la que la solución sea una matriz dispersa. Para concluir, los métodos estadísticos estudiados, se comparan realizando algunas simulaciones. Al realizarlas, se observa que la mejor técnica para inferir la topología del grafo es la que se basa en la regresión lineal.Aquesta tesi estudia el problema d'inferir la topologia d'una xarxa a partir dels senyals graf. Per aquesta raó, la Tesi Final de Màster s'inscriu en el corrent actual de pensament, creixent en els darrers anys, en la que se suposa no coneguda l'estructura de la xarxa. El problema d'inferència de la topologia s'aborda des de dos angles. El primer, estudia com trobar l'estructura d'un graf a partir de plantilles espectrals que poden ser sorolloses o no. Així, a partir de les observacions, s'infereix la plantilla espectral del graf que compon la xarxa. En treballs anteriors, com la meva Tesi de Final de Grau, es van estudiar els algoritmes per a la inferència de plantilles espectrals incompletes. En aquest treball, anem un pas més enllà demostrant per què les tècniques estudiades no sempre funcionen. Seguim, proposant un algoritme basat en LASSO per inferir la topologia de la xarxa quan les plantilles espectrals són sorolloses. L'algoritme proposat es compara amb els anteriorment estudiats obtenint millors resultats en termes de RMSE i fiabilitat. El segon punt de vista abordat en aquesta tesi és la inferència de la xarxa a partir de tècniques estadístiques. És comú trobar xarxes on els nodes tenen alguna relació entre ells. Aquestes tècniques es basen, a partir d'algunes observacions de la xarxa, a tractar de trobar les relacions existents entre els diferents nodes del graf. Aquestes tècniques poden ser utilitzades de manera més genèrica que les basades en plantilles espectrals. Per aquest motiu, els mètodes estadístics s'estudien amb més profunditat en aquesta Tesi de Màster. Inicialment, s'explica el coeficient de correlació de Pearson. Després d'estudiar-lo, es detecta una limitació. Per aquest motiu, es proposa un nou enfocament basat en la covariància condicional. Després, s'assumeix que els senyals segueixen una distribució Gaussiana, el que ens porta a estudiar l'estimador de Màxima Versemblança, també considerant que la matriu solució és dispersa. Encara que amb aquesta tècnica es millora l'enfocament anterior, estem interessats a trobar una tècnica encara millor. Per tant, estudiem un enfocament basat en la regressió lineal. En aquest últim algoritme, incloem un terme per promoure que la solució sigui una matriu dispersa. Per concloure, els mètodes estadístics estudiats, es comparen realitzant algunes simulacions. En realitzar-les, s'observa que la millor tècnica per inferir la topologia del graf és la que es basa en la regressió lineal