357 research outputs found
Data-Driven Representation Learning in Multimodal Feature Fusion
abstract: Modern machine learning systems leverage data and features from multiple modalities to gain more predictive power. In most scenarios, the modalities are vastly different and the acquired data are heterogeneous in nature. Consequently, building highly effective fusion algorithms is at the core to achieve improved model robustness and inferencing performance. This dissertation focuses on the representation learning approaches as the fusion strategy. Specifically, the objective is to learn the shared latent representation which jointly exploit the structural information encoded in all modalities, such that a straightforward learning model can be adopted to obtain the prediction.
We first consider sensor fusion, a typical multimodal fusion problem critical to building a pervasive computing platform. A systematic fusion technique is described to support both multiple sensors and descriptors for activity recognition. Targeted to learn the optimal combination of kernels, Multiple Kernel Learning (MKL) algorithms have been successfully applied to numerous fusion problems in computer vision etc. Utilizing the MKL formulation, next we describe an auto-context algorithm for learning image context via the fusion with low-level descriptors. Furthermore, a principled fusion algorithm using deep learning to optimize kernel machines is developed. By bridging deep architectures with kernel optimization, this approach leverages the benefits of both paradigms and is applied to a wide variety of fusion problems.
In many real-world applications, the modalities exhibit highly specific data structures, such as time sequences and graphs, and consequently, special design of the learning architecture is needed. In order to improve the temporal modeling for multivariate sequences, we developed two architectures centered around attention models. A novel clinical time series analysis model is proposed for several critical problems in healthcare. Another model coupled with triplet ranking loss as metric learning framework is described to better solve speaker diarization. Compared to state-of-the-art recurrent networks, these attention-based multivariate analysis tools achieve improved performance while having a lower computational complexity. Finally, in order to perform community detection on multilayer graphs, a fusion algorithm is described to derive node embedding from word embedding techniques and also exploit the complementary relational information contained in each layer of the graph.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201
Recommended from our members
Knowledge Discovery and Data Mining for Shared Mobility and Connected and Automated Vehicle Applications
The rapid development of shared mobility and connected and automated vehicles (CAVs) has not only brought new intelligent transportation system (ITS) challenges with the new types of mobility, but also brought a huge opportunity to accelerate the connectivity and informatization of transportation systems, particularly when we consider all the new forms of data that is becoming available. The primary challenge is how to take advantage of the enormous amount of data to discover knowledge, build effective models, and develop impactful applications. With the theoretical and experimental progress being made over the last two decades, data mining and machine learning technologies have become key approaches for parsing data, understanding information, and making informed decisions, especially as the rise of deep learning algorithms bringing new levels of performance to the analysis of large datasets. The combination of data mining and ITS can greatly benefit research and advances in shared mobility and CAVs.This dissertation focuses on knowledge discovery and data mining for shared mobility and CAV applications. When considering big data associated with shared mobility operations and CAV research, data mining techniques can be customized with transportation knowledge to initially parse the data. Then machine learning methods can be used to model the parsed data to elicit hidden knowledge. Finally, the discovered knowledge and extracted information can help in the development of effective shared mobility and CAV applications to achieve the goals of a safer, faster, and more eco-friendly transportation systems.In this dissertation, there are four main sections that are addressed. First, new methodologies are introduced for extracting lane-level road features from rough crowdsourced GPS trajectories via data mining, which is subsequently used as the fundamental information for CAV applications. The proposed method results in decimeter level accuracy, which satisfies the positioning needs for many macroscopic and microscopic shared mobility and CAV applications. Second, macroscopic ride-hailing service big data has been analyzed for demand prediction, vehicle operation, and system efficiency monitoring. The proposed deep learning algorithms increase the ride-hailing demand prediction accuracy to 80% and can help the fleet dispatching system reduce 30% of vacant travel distance. Third, microscopic automated vehicle perception data has been analyzed for a real-time computer vision system that can be used for lane change behavior detection. The proposed deep learning design combines the residual neural network image input with time serious control data and reaches 95% of lane change behavior prediction accuracy. Last but not least, new ride sharing and CAV applications have been simulated in a behavior modeling framework to analyze the impact of mobility and energy consumption, which addresses key barriers by quantifying the transportation system-wide mobility, energy and behavior impacts from new mobility technologies using real-world data
Modeling, Predicting and Capturing Human Mobility
Realistic models of human mobility are critical for modern day applications, specifically for recommendation systems, resource planning and process optimization domains. Given the rapid proliferation of mobile devices equipped with Internet connectivity and GPS functionality today, aggregating large sums of individual geolocation data is feasible. The thesis focuses on methodologies to facilitate data-driven mobility modeling by drawing parallels between the inherent nature of mobility trajectories, statistical physics and information theory. On the applied side, the thesis contributions lie in leveraging the formulated mobility models to construct prediction workflows by adopting a privacy-by-design perspective. This enables end users to derive utility from location-based services while preserving their location privacy. Finally, the thesis presents several approaches to generate large-scale synthetic mobility datasets by applying machine learning approaches to facilitate experimental reproducibility
A step towards Advancing Digital Phenotyping In Mental Healthcare
Smartphones and wrist-wearable devices have infiltrated our lives in recent years. According
to published statistics, nearly 84% of the world’s population owns a smartphone,
and almost 10% own a wearable device today (2022). These devices continuously generate
various data sources from multiple sensors and apps, creating our digital phenotypes.
This opens new research opportunities, particularly in mental health care, which has previously
relied almost exclusively on self-reports of mental health symptoms.
Unobtrusive monitoring using patients’ devices may result in clinically valuable markers
that can improve diagnostic processes, tailor treatment choices, provide continuous
insights into their condition for actionable outcomes, such as early signs of relapse, and
develop new intervention models. However, these data sources must be translated into
meaningful, actionable features related to mental health to achieve their full potential.
In the mental health field, there is a great need and much to be gained from defining a
way to continuously assess the evolution of patients’ mental states, ideally in their everyday
environment, to support the monitoring and treatments by health care providers. A
smartphone-based approach may be valuable in gathering long-term objective data, aside
from the usually used self-ratings, to predict clinical state changes and investigate causal
inferences about state changes in patients (e.g., those with affective disorders).
Being objective does not imply that passive data collection is also perfect. It has several
challenges: some sensors generate vast volumes of data, and others cause significant
battery drain. Furthermore, the analysis of raw passive data is complicated, and collecting
certain types of data may interfere with the phenotype of interest. Nonetheless, machine
learning is predisposed to address these matters and advance psychiatry’s era of personalised
medicine.
This work aimed to advance the research efforts on mobile and wearable sensors for
mental health monitoring. We applied supervised and unsupervised machine learning
methods to model and understand mental disease evolution based on the digital phenotype
of patients and clinician assessments at the follow-up visits, which provide ground
truths. We needed to cope with regularly and irregularly sampled, high-dimensional, and
heterogeneous time series data susceptible to distortion and missingness. Hence, the developed
methods must be robust to these limitations and handle missing data properly.
Throughout the various projects presented here, we used probabilistic latent variable
models for data imputation and feature extraction, namely, mixture models (MM) and hidden
Markov models (HMM). These unsupervised models can learn even in the presence
of missing data by marginalising the missing values in the function of the present observations. Once the generative models are trained on the data set with missing values, they can
be used to generate samples for imputation. First, the most probable component/state has
to be found for each sample. Then, sampling from the most probable distribution yields
valid and robust parameter estimates and explicit imputed values for variables that can
be analysed as outcomes or predictors. The imputation process can be repeated several
times, creating multiple datasets, thereby accounting for the uncertainty in the imputed
values and implicitly augmenting the data. Moreover, they are robust to moderate deviations
of the observed data from the assumed underlying distribution and provide accurate
estimates even when missingness is high.
Depending on the properties of the data at hand, we employed feature extraction
methods combined with classical machine learning algorithms or deep learning-based
techniques for temporal modelling to predict various mental health outcomes - emotional
state, World Health Organisation Disability Assessment Schedule (WHODAS 2.0) functionality
scores and Generalised Anxiety Disorder-7 (GAD-7) scores, of psychiatric outpatients.
We mainly focused on one-size-fits-all models, as the labelled sample size per
patient was limited; however, in the mood prediction case, it was possible to apply personalised
models.
Integrating machines and algorithms into the clinical workflow require interpretability
to increase acceptance. Therefore, we also analysed feature importance by computing
Shapley additive explanations (SHAP) values. SHAP values provide an overview of essential
features in the machine learning models by designating the weight of predictability
of each feature positively or negatively to the target variable.
The provided solutions, as such, are proof of concept, which require further clinical
validation to be deployable in the clinical workflow. Still, the results are promising
and lay some foundations for future research and collaboration among clinicians, patients,
and computer scientists. They set the paths to advance future research prospects in
technology-based mental healthcare.En los últimos años, los smartphones y los dispositivos y pulseras inteligentes, comúnmente
conocidos como wearables, se han infiltrado en nuestras vidas. Según las estadísticas
publicadas a día de hoy (2022), cerca del 84% de la población tiene un smartphone y
aproximadamente un 10% también posee un wearable. Estos dispositivos generan datos
de forma continua en base a distintos sensores y aplicaciones, creando así nuestro fenotipo
digital. Estos datos abren nuevas vías de investigación, particularmente en el área de salud
mental, dónde las fuentes de datos han sido casi exclusivamente autoevaluaciones de síntomas
de salud mental.
Monitorizar de forma no intrusiva a los pacientes mediante sus dispositivos puede dar
lugar a marcadores valiosos en aplicación clínica. Esto permite mejorar los procesos de
diagnóstico, adaptar tratamientos, e incluso proporcionar información continua sobre el
estado de los pacientes, como signos tempranos de recaída, y hasta desarrollar nuevos
modelos de intervención. Aun así, estos datos en crudo han de ser traducidos a datos
interpretables relacionados con la salud mental para conseguir un máximo rendimiento de
los mismos.
En salud mental existe una gran necesidad, y además hay mucho que ganar, de definir
cómo evaluar de forma continuada la evolución del estado mental de los pacientes en su
entorno cotidiano para ayudar en el tratamiento y seguimiento de los mismos por parte
de los profesionales sanitarios. En este ámbito, un enfoque basado en datos recopilados
desde sus smartphones puede ser valioso para recoger datos objetivos a largo plazo al
mismo tiempo que se acompaña de las autoevaluaciones utilizadas habitualmente. La
combinación de ambos tipos de datos puede ayudar a predecir los cambios en el estado
clínico de estos pacientes e investigar las relaciones causales sobre estos cambios (por
ejemplo, en aquellos que padecen trastornos afectivos).
Aunque la recogida de datos de forma pasiva tiene la ventaja de ser objetiva, también
implica varios retos. Por un lado, ciertos sensores generan grandes volúmenes de
datos, provocando un importante consumo de batería. Además, el análisis de los datos
pasivos en crudo es complicado, y la recogida de ciertos tipos de datos puede interferir
con el fenotipo que se quiera analizar. No obstante, el machine learning o aprendizaje
automático, está predispuesto a resolver estas cuestiones y aportar avances en la medicina
personalizada aplicada a psiquiatría.
Esta tesis tiene como objetivo avanzar en la investigación de los datos recogidos por
sensores de smartphones y wearables para la monitorización en salud mental. Para ello,
aplicamos métodos de aprendizaje automático supervisado y no supervisado para modelar y comprender la evolución de las enfermedades mentales basándonos en el fenotipo digital
de los pacientes. Estos resultados se comparan con las evaluaciones de los médicos en
las visitas de seguimiento, que proporcionan las etiquetas reales. Para aplicar estos métodos
hemos lidiado con datos provenientes de series temporales con alta dimensionalidad,
muestreados de forma regular e irregular, heterogéneos y, además, susceptibles a presentar
patrones de datos perdidos y/o distorsionados. Por lo tanto, los métodos desarrollados
deben ser resistentes a estas limitaciones y manejar adecuadamente los datos perdidos.
A lo largo de los distintos proyectos presentados en este trabajo, hemos utilizado
modelos probabilísticos de variables latentes para la imputación de datos y la extracción
de características, como por ejemplo, Mixture Models (MM) y hidden Markov Models
(HMM). Estos modelos no supervisados pueden aprender incluso en presencia de datos
perdidos, marginalizando estos valores en función de las datos que sí han sido observados.
Una vez entrenados los modelos generativos en el conjunto de datos con valores
perdidos, pueden utilizarse para imputar dichos valores generando muestras. En primer
lugar, hay que encontrar el componente/estado más probable para cada muestra. Luego,
se muestrea de la distirbución más probable resultando en estimaciones de parámetros robustos
y válidos. Además, genera imputaciones explícitas que pueden ser tratadas como
resultados. Este proceso de imputación puede repetirse varias veces, creando múltiples
conjuntos de datos, con lo que se tiene en cuenta la incertidumbre de los valores imputados
y aumentándose así, implícitamente, los datos. Además, estas imputaciones son
resistentes a desviaciones que puedan existir en los datos observados con respecto a la
distribución subyacente asumida y proporcionan estimaciones precisas incluso cuando la
falta de datos es elevada.
Dependiendo de las propiedades de los datos en cuestión, hemos usado métodos de extracción
de características combinados con algoritmos clásicos de aprendizaje automático
o técnicas basadas en deep learning o aprendizaje profundo para el modelado temporal.
La finalidad de ambas opciones es ser capaces de predecir varios resultados de salud
mental/estado emocional, como la puntuación sobre el World Health Organisation Disability
Assessment Schedule (WHODAS 2.0), o las puntuaciones del generalised anxiety
disorder-7 (GAD-7) de pacientes psiquiátricos ambulatorios. Nos centramos principalmente
en modelos generalizados, es decir, no personalizados para cada paciente sino
explicativos para la mayoría, ya que el tamaño de muestras etiquetada por paciente es
limitado; sin embargo, en el caso de la predicción del estado de ánimo, puidmos aplicar
modelos personalizados.
Para que la integración de las máquinas y algoritmos dentro del flujo de trabajo clínico
sea aceptada, se requiere que los resultados sean interpretables. Por lo tanto, en este trabajo
también analizamos la importancia de las características sacadas por cada algoritmo
en base a los valores de las explicaciones aditivas de Shapley (SHAP). Estos valores proporcionan
una visión general de las características esenciales en los modelos de aprendizaje
automático designando el peso, positivo o negativo, de cada característica en su
predictibilidad sobre la variable objetivo. Las soluciones aportadas en esta tesis, como tales, son pruebas de concepto, que requieren
una mayor validación clínica para poder ser desplegadas en el flujo de trabajo
clínico. Aun así, los resultados son prometedores y sientan base para futuras investigaciones
y colaboraciones entre clínicos, pacientes y científicos de datos. Éstas establecen
las guías para avanzar en las perspectivas de investigación futuras en la atención sanitaria
mental basada en la tecnología.Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan CarlosPresidente: David Ramírez García.- Secretario: Alfredo Nazábal Rentería.- Vocal: María Luisa Barrigón Estéve
- …