961 research outputs found
Inferring Group-Wise Consistent Multimodal Brain Networks via Multi-View Spectral Clustering
Quantitative modeling and analysis of structural and functional brain networks based on diffusion tensor imaging (DTI) and functional MRI (fMRI) data have received extensive interest recently. However, the regularity of these structural and functional brain networks across multiple neuroimaging modalities and also across different individuals is largely unknown. This paper presents a novel approach to inferring group-wise consistent brain sub-networks from multimodal DTI/resting-state fMRI datasets via multi-view spectral clustering of cortical networks, which were constructed upon our recently developed and validated large-scale cortical landmarks - DICCCOL (Dense Individualized and Common Connectivity-based Cortical Landmarks). We applied the algorithms on DTI data of 100 healthy young females and 50 healthy young males, obtained consistent multimodal brain networks within and across multiple groups, and further examined the functional roles of these networks. Our experimental results demonstrated that the derived brain networks have substantially improved inter-modality and inter-subject consistency
Predicting Alzheimer's Disease by Hierarchical Graph Convolution from Positron Emission Tomography Imaging
Imaging-based early diagnosis of Alzheimer Disease (AD) has become an
effective approach, especially by using nuclear medicine imaging techniques
such as Positron Emission Topography (PET). In various literature it has been
found that PET images can be better modeled as signals (e.g. uptake of
florbetapir) defined on a network (non-Euclidean) structure which is governed
by its underlying graph patterns of pathological progression and metabolic
connectivity. In order to effectively apply deep learning framework for PET
image analysis to overcome its limitation on Euclidean grid, we develop a
solution for 3D PET image representation and analysis under a generalized,
graph-based CNN architecture (PETNet), which analyzes PET signals defined on a
group-wise inferred graph structure. Computations in PETNet are defined in
non-Euclidean, graph (network) domain, as it performs feature extraction by
convolution operations on spectral-filtered signals on the graph and pooling
operations based on hierarchical graph clustering. Effectiveness of the PETNet
is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset,
which shows improved performance over both deep learning and other machine
learning-based methods.Comment: Jiaming Guo, Wei Qiu and Xiang Li contribute equally to this wor
Brain Music : Sistema generativo para la creación de música simbólica a partir de respuestas neuronales afectivas
gráficas, tablasEsta tesis de maestrÃa presenta una metodologÃa de aprendizaje profundo multimodal innovadora que fusiona un modelo de clasificación de emociones con un generador musical, con el propósito de crear música a partir de señales de electroencefalografÃa, profundizando asà en la interconexión entre emociones y música. Los resultados alcanzan tres objetivos especÃficos:
Primero, ya que el rendimiento de los sistemas interfaz cerebro-computadora varÃa considerablemente entre diferentes sujetos, se introduce un enfoque basado en la transferencia de conocimiento entre sujetos para mejorar el rendimiento de individuos con dificultades en sistemas de interfaz cerebro-computadora basados en el paradigma de imaginación motora. Este enfoque combina datos de EEG etiquetados con datos estructurados, como cuestionarios psicológicos, mediante un método de "Kernel Matching CKA". Utilizamos una red neuronal profunda (Deep&Wide) para la clasificación de la imaginación motora. Los resultados destacan su potencial para mejorar las habilidades motoras en interfaces cerebro-computadora.
Segundo, proponemos una técnica innovadora llamada "Labeled Correlation Alignment"(LCA) para sonificar respuestas neurales a estÃmulos representados en datos no estructurados, como música afectiva. Esto genera caracterÃsticas musicales basadas en la actividad cerebral inducida por las emociones. LCA aborda la variabilidad entre sujetos y dentro de sujetos mediante el análisis de correlación, lo que permite la creación de envolventes acústicos y la distinción entre diferente información sonora. Esto convierte a LCA en una herramienta prometedora para interpretar la actividad neuronal y su reacción a estÃmulos auditivos.
Finalmente, en otro capÃtulo, desarrollamos una metodologÃa de aprendizaje profundo de extremo a extremo para generar contenido musical MIDI (datos simbólicos) a partir de señales de actividad cerebral inducidas por música con etiquetas afectivas. Esta metodologÃa abarca el preprocesamiento de datos, el entrenamiento de modelos de extracción de caracterÃsticas y un proceso de emparejamiento de caracterÃsticas mediante Deep Centered Kernel Alignment, lo que permite la generación de música a partir de señales EEG.
En conjunto, estos logros representan avances significativos en la comprensión de la relación entre emociones y música, asà como en la aplicación de la inteligencia artificial en la generación musical a partir de señales cerebrales. Ofrecen nuevas perspectivas y herramientas para la creación musical y la investigación en neurociencia emocional. Para llevar a cabo nuestros experimentos, utilizamos bases de datos públicas como GigaScience, Affective Music Listening y Deap Dataset (Texto tomado de la fuente)This master’s thesis presents an innovative multimodal deep learning methodology that combines an emotion classification model with a music generator, aimed at creating music from electroencephalography (EEG) signals, thus delving into the interplay between emotions and music. The results achieve three specific objectives:
First, since the performance of brain-computer interface systems varies significantly among different subjects, an approach based on knowledge transfer among subjects is introduced to enhance the performance of individuals facing challenges in motor imagery-based brain-computer interface systems. This approach combines labeled EEG data with structured information, such as psychological questionnaires, through a "Kernel Matching CKA"method. We employ a deep neural network (Deep&Wide) for motor imagery classification. The results underscore its potential to enhance motor skills in brain-computer interfaces.
Second, we propose an innovative technique called "Labeled Correlation Alignment"(LCA) to sonify neural responses to stimuli represented in unstructured data, such as affective music. This generates musical features based on emotion-induced brain activity. LCA addresses variability among subjects and within subjects through correlation analysis, enabling the creation of acoustic envelopes and the distinction of different sound information. This makes LCA a promising tool for interpreting neural activity and its response to auditory stimuli.
Finally, in another chapter, we develop an end-to-end deep learning methodology for generating MIDI music content (symbolic data) from EEG signals induced by affectively labeled music. This methodology encompasses data preprocessing, feature extraction model training, and a feature matching process using Deep Centered Kernel Alignment, enabling music generation from EEG signals.
Together, these achievements represent significant advances in understanding the relationship between emotions and music, as well as in the application of artificial intelligence in musical generation from brain signals. They offer new perspectives and tools for musical creation and research in emotional neuroscience. To conduct our experiments, we utilized public databases such as GigaScience, Affective Music Listening and Deap DatasetMaestrÃaMagÃster en IngenierÃa - Automatización IndustrialInvestigación en Aprendizaje Profundo y señales BiológicasEléctrica, Electrónica, Automatización Y Telecomunicaciones.Sede Manizale
Fine-Granularity Functional Interaction Signatures for Characterization of Brain Conditions
In the human brain, functional activity occurs at multiple spatial scales. Current studies on functional brain networks and their alterations in brain diseases via resting-state functional magnetic resonance imaging (rs-fMRI) are generally either at local scale (regionally confined analysis and inter-regional functional connectivity analysis) or at global scale (graph theoretic analysis). In contrast, inferring functional interaction at fine-granularity sub-network scale has not been adequately explored yet. Here our hypothesis is that functional interaction measured at fine-granularity subnetwork scale can provide new insight into the neural mechanisms of neurological and psychological conditions, thus offering complementary information for healthy and diseased population classification. In this paper, we derived fine-granularity functional interaction (FGFI) signatures in subjects with Mild Cognitive Impairment (MCI) and Schizophrenia by diffusion tensor imaging (DTI) and rsfMRI, and used patient-control classification experiments to evaluate the distinctiveness of the derived FGFI features. Our experimental results have shown that the FGFI features alone can achieve comparable classification performance compared with the commonly used inter-regional connectivity features. However, the classification performance can be substantially improved when FGFI features and inter-regional connectivity features are integrated, suggesting the complementary information achieved from the FGFI signatures
Data-Driven Representation Learning in Multimodal Feature Fusion
abstract: Modern machine learning systems leverage data and features from multiple modalities to gain more predictive power. In most scenarios, the modalities are vastly different and the acquired data are heterogeneous in nature. Consequently, building highly effective fusion algorithms is at the core to achieve improved model robustness and inferencing performance. This dissertation focuses on the representation learning approaches as the fusion strategy. Specifically, the objective is to learn the shared latent representation which jointly exploit the structural information encoded in all modalities, such that a straightforward learning model can be adopted to obtain the prediction.
We first consider sensor fusion, a typical multimodal fusion problem critical to building a pervasive computing platform. A systematic fusion technique is described to support both multiple sensors and descriptors for activity recognition. Targeted to learn the optimal combination of kernels, Multiple Kernel Learning (MKL) algorithms have been successfully applied to numerous fusion problems in computer vision etc. Utilizing the MKL formulation, next we describe an auto-context algorithm for learning image context via the fusion with low-level descriptors. Furthermore, a principled fusion algorithm using deep learning to optimize kernel machines is developed. By bridging deep architectures with kernel optimization, this approach leverages the benefits of both paradigms and is applied to a wide variety of fusion problems.
In many real-world applications, the modalities exhibit highly specific data structures, such as time sequences and graphs, and consequently, special design of the learning architecture is needed. In order to improve the temporal modeling for multivariate sequences, we developed two architectures centered around attention models. A novel clinical time series analysis model is proposed for several critical problems in healthcare. Another model coupled with triplet ranking loss as metric learning framework is described to better solve speaker diarization. Compared to state-of-the-art recurrent networks, these attention-based multivariate analysis tools achieve improved performance while having a lower computational complexity. Finally, in order to perform community detection on multilayer graphs, a fusion algorithm is described to derive node embedding from word embedding techniques and also exploit the complementary relational information contained in each layer of the graph.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201
Fast Continual Multi-View Clustering with Incomplete Views
Multi-view clustering (MVC) has gained broad attention owing to its capacity
to exploit consistent and complementary information across views. This paper
focuses on a challenging issue in MVC called the incomplete continual data
problem (ICDP). In specific, most existing algorithms assume that views are
available in advance and overlook the scenarios where data observations of
views are accumulated over time. Due to privacy considerations or memory
limitations, previous views cannot be stored in these situations. Some works
are proposed to handle it, but all fail to address incomplete views. Such an
incomplete continual data problem (ICDP) in MVC is tough to solve since
incomplete information with continual data increases the difficulty of
extracting consistent and complementary knowledge among views. We propose Fast
Continual Multi-View Clustering with Incomplete Views (FCMVC-IV) to address it.
Specifically, it maintains a consensus coefficient matrix and updates knowledge
with the incoming incomplete view rather than storing and recomputing all the
data matrices. Considering that the views are incomplete, the newly collected
view might contain samples that have yet to appear; two indicator matrices and
a rotation matrix are developed to match matrices with different dimensions.
Besides, we design a three-step iterative algorithm to solve the resultant
problem in linear complexity with proven convergence. Comprehensive experiments
on various datasets show the superiority of FCMVC-IV
Deep Learning based 3D Segmentation: A Survey
3D object segmentation is a fundamental and challenging problem in computer
vision with applications in autonomous driving, robotics, augmented reality and
medical image analysis. It has received significant attention from the computer
vision, graphics and machine learning communities. Traditionally, 3D
segmentation was performed with hand-crafted features and engineered methods
which failed to achieve acceptable accuracy and could not generalize to
large-scale data. Driven by their great success in 2D computer vision, deep
learning techniques have recently become the tool of choice for 3D segmentation
tasks as well. This has led to an influx of a large number of methods in the
literature that have been evaluated on different benchmark datasets. This paper
provides a comprehensive survey of recent progress in deep learning based 3D
segmentation covering over 150 papers. It summarizes the most commonly used
pipelines, discusses their highlights and shortcomings, and analyzes the
competitive results of these segmentation methods. Based on the analysis, it
also provides promising research directions for the future.Comment: Under review of ACM Computing Surveys, 36 pages, 10 tables, 9 figure
- …