5,843 research outputs found
GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization
Recent research has demonstrated the capability of behavior signals captured
by smartphones and wearables for longitudinal behavior modeling. However, there
is a lack of a comprehensive public dataset that serves as an open testbed for
fair comparison among algorithms. Moreover, prior studies mainly evaluate
algorithms using data from a single population within a short period, without
measuring the cross-dataset generalizability of these algorithms. We present
the first multi-year passive sensing datasets, containing over 700 user-years
and 497 unique users' data collected from mobile and wearable sensors, together
with a wide range of well-being metrics. Our datasets can support multiple
cross-dataset evaluations of behavior modeling algorithms' generalizability
across different users and years. As a starting point, we provide the benchmark
results of 18 algorithms on the task of depression detection. Our results
indicate that both prior depression detection algorithms and domain
generalization techniques show potential but need further research to achieve
adequate cross-dataset generalizability. We envision our multi-year datasets
can support the ML community in developing generalizable longitudinal behavior
modeling algorithms.Comment: Thirty-sixth Conference on Neural Information Processing Systems
Datasets and Benchmarks Trac
Personalized multi-task attention for multimodal mental health detection and explanation
The unprecedented spread of smartphone usage and its various boarding sensors have been garnering increasing interest in automatic mental health detection. However, there are two major barriers to reliable mental health detection applications that can be adopted in real-life: (a)The outputs of the complex machine learning model are not explainable, which reduces the trust of users and thus hinders the application in real-life scenarios. (b)The sensor signal distribution discrepancy across individuals is a major barrier to accurate detection since each individual has their own characteristics. We propose an explainable mental health detection model. Spatial and temporal features of multiple sensory sequences are extracted and fused with different weights generated by the attention mechanism so that the discrepancy of contribution to classifiers across different modalities can be considered in the model. Through a series of experiments on real-life datasets, results show the effectiveness of our model compared to the existing approaches.This research is supported by the National Natural Science Foundation of China (No. 62077027), the Ministry of Science and Technology of the People's Republic of China(No. 2018YFC2002500), the Jilin Province Development and Reform Commission, China (No. 2019C053-1), the Education Department of Jilin Province, China (No. JJKH20200993K), the Department of Science and Technology of Jilin Province, China (No. 20200801002GH), and the European Union's Horizon 2020 FET Proactive project "WeNet-The Internet of us"(No. 823783)
Modern Views of Machine Learning for Precision Psychiatry
In light of the NIMH's Research Domain Criteria (RDoC), the advent of
functional neuroimaging, novel technologies and methods provide new
opportunities to develop precise and personalized prognosis and diagnosis of
mental disorders. Machine learning (ML) and artificial intelligence (AI)
technologies are playing an increasingly critical role in the new era of
precision psychiatry. Combining ML/AI with neuromodulation technologies can
potentially provide explainable solutions in clinical practice and effective
therapeutic treatment. Advanced wearable and mobile technologies also call for
the new role of ML/AI for digital phenotyping in mobile mental health. In this
review, we provide a comprehensive review of the ML methodologies and
applications by combining neuroimaging, neuromodulation, and advanced mobile
technologies in psychiatry practice. Additionally, we review the role of ML in
molecular phenotyping and cross-species biomarker identification in precision
psychiatry. We further discuss explainable AI (XAI) and causality testing in a
closed-human-in-the-loop manner, and highlight the ML potential in multimedia
information extraction and multimodal data fusion. Finally, we discuss
conceptual and practical challenges in precision psychiatry and highlight ML
opportunities in future research
Inferring Mood-While-Eating with Smartphone Sensing and Community-Based Model Personalization
The interplay between mood and eating has been the subject of extensive
research within the fields of nutrition and behavioral science, indicating a
strong connection between the two. Further, phone sensor data have been used to
characterize both eating behavior and mood, independently, in the context of
mobile food diaries and mobile health applications. However, limitations within
the current body of literature include: i) the lack of investigation around the
generalization of mood inference models trained with passive sensor data from a
range of everyday life situations, to specific contexts such as eating, ii) no
prior studies that use sensor data to study the intersection of mood and
eating, and iii) the inadequate examination of model personalization techniques
within limited label settings, as we commonly experience in mood inference. In
this study, we sought to examine everyday eating behavior and mood using two
datasets of college students in Mexico (N_mex = 84, 1843 mood-while-eating
reports) and eight countries (N_mul = 678, 329K mood reports incl. 24K
mood-while-eating reports), containing both passive smartphone sensing and
self-report data. Our results indicate that generic mood inference models
decline in performance in certain contexts, such as when eating. Additionally,
we found that population-level (non-personalized) and hybrid (partially
personalized) modeling techniques were inadequate for the commonly used
three-class mood inference task (positive, neutral, negative). Furthermore, we
found that user-level modeling was challenging for the majority of participants
due to a lack of sufficient labels and data from the negative class. To address
these limitations, we employed a novel community-based approach for
personalization by building models with data from a set of similar users to a
target user
Recommended from our members
Wearables, smartphones, and artificial intelligence for digital phenotyping and health
Ubiquitous progress in wearable sensing and mobile computing technologies, alongside growing diversity in sensor modalities, has created new pathways for the collection of health and well-being data outside of laboratory settings, in a longitudinal fashion. Wearable and mobile devices have the potential to provide low-cost, objective measures of physical activity, clinically relevant data for patient assessment, and scalable behavior monitoring in large populations. These data can be used in both interventional and observational studies to derive insights regarding the links between behavior, health. and disease, as well as to advance the personalization and effectiveness of commercial wellness applications. Today, over 400,000 participants have had their behavior tracked prospectively using accelerometers for epidemiological studies across the globe. Traditionally, epidemiologists and clinicians have relied upon self-report measures of physical activity and sleep which, while valuable in the absence of alternatives, are subject to bias and often provide partial, incomplete information Physical behavior data extracted from wearable devices are being used to derive sensor-assessed, objective measures of physical behaviors, overcoming the limitations of self-report with the aim of relating these to clinical endpoints and eventually applying the findings to preventive and predictive medicine. Moreover, the application of artificial intelligence (AI), sensor fusion, and signal processing to wearable sensor data has led to improved human activity recognition and behavioral phenotyping. Here, we review the state of the art in wearable and mobile sensing technology in epidemiology and clinical medicine and discuss how AI is changing the field
Probabilistic models for human behavior learning
The problem of human behavior learning is a popular interdisciplinary research topic that
has been explored from multiple perspectives, with a principal branch of study in the
context of computer vision systems and activity recognition. However, the statistical methods
used in these frameworks typically assume short time scales, usually of minutes or even
seconds. The emergence of mobile electronic devices, such as smartphones and wearables,
has changed this paradigm as long as we are now able to massively collect digital records
from users. This collection of smartphone-generated data, whose attributes are obtained in
an unobtrusive manner from the devices via multiple sensors and apps, shape the behavioral
footprint that is unique for everyone of us. At an individual level, the data projection also
di ers from person to person, as not all sensors are equal, neither the apps installed, or the
devices used in the real life. This point actually reflects that learning the human behavior
from the digital signature of users is an arduous task, that requires to fuse irregular data.
For instance, collections of samples that are corrupted, heterogeneous, outliers or have shortterm
correlations. The statistical modelling of this sort of objects is one of the principal
contributions of this thesis, that we study from the perspective of Gaussian processes (gp).
In the particular case of humans, as well as many other life species in our world, we are
inherently conditioned to the diurnal and nocturnal cycles that everyday shape our behavior,
and hence, our data. We can study these cycles in our behavioral representation to see that
there exists a perpetual circadian rhytm in everyone of us. This tempo is the 24h periodic
component that shapes the baseline temporal structure of our behavior, not the particular
patterns that change for every person. Looking to the trajectories and variabilities that our
behavior may take in the data, we can appreciate that there is not a single repetitive behavior.
Instead, there are typically several patterns or routines, sampled from our own dictionary,
that we choose for every special situation. At the same time, these routines are arbitrary
combinations of di erents timescales, correlations, levels of mobility, social interaction, sleep
quality or will for working during the same hours on weekdays. Together, the properties of
human behavior already indicate to us how we shall proceed to model its structure, not as
unique functions, but as a dictionary of latent behavioral profiles. To discover them, we have
considered latent variable models.
The main application of the statistical methods developed for human behavior learning
appears as we look to medicine. Having a personalized model that is accurately fitted to
the behavioral patterns of some patient of interest, sudden changes in them could be early
indicators of future relapses. From a technical point of view, the traditional question use to
be if newer observations conform or not to the expected behavior indicated by the already
fitted model. The problem can be analyzed from two perspectives that are interrelated, one
more oriented to the characterization of that single object as outlier, typically named as
anomaly detection, and another focused in refreshing the learning model if no longer fits to
the new sequential data. This last problem, widely known as change-point detection (cpd)
is another pillar of this thesis. These methods are oriented to mental health applications,
and particularly to the passive detection of crisis events. The final goal is to provide an
early detection methodology based on probabilistic modeling for early intervention, e.g. prevent
suicide attempts, on psychiatric outpatients with severe a ective disorders of higher
prevalence, such as depression or bipolar diseases.El problema de aprendizaje del comportamiento humano es un tema de investigación interdisciplinar
que ha sido explorado desde múltiples perspectivas, con una lÃnea de estudio
principal en torno a los sistemas de visión por ordenador y el reconocimiento de actividades.
Sin embargo, los métodos estadÃsticos usados en estos casos suelen asumir escalas de tiempo
cortas, generalmente de minutos o incluso segundos. La aparición de tecnologÃas móviles,
tales como teléfonos o relojes inteligentes, ha cambiado este paradigma, dado que ahora es
posible recolectar ingentes colecciones de datos a partir de los usuarios. Este conjunto de
datos generados a partir de nuestro teléfono, cuyos atributos se obtienen de manera no invasiva
desde múltiples sensores y apps, conforman la huella de comportamiento que es única
para cada uno de nosotros. A nivel individual, la proyección sobre los datos difiere de persona
a persona, dado que no todos los sensores son iguales, ni las apps instaladas asà como
los dispositivos utilizados en la vida real. Esto precisamente refleja que el aprendizaje del
comportamiento humano a partir de la huella digital de los usuarios es una ardua tarea,
que requiere principalmente fusionar datos irregulares. Por ejemplo, colecciones de muestras
corruptas, heterogéneas, con outliers o poseedoras de correlaciones cortas. El modelado estadÃstico de este tipo de objetos es una de las contribuciones principales de esta tesis, que
estudiamos desde la perspectiva de los procesos Gaussianos (gp).
En el caso particular de los humanos, asà como para muchas otras especies en nuestro
planeta, estamos inherentemente condicionados a los ciclos diurnos y nocturnos que cada
dÃa dan forma a nuestro comportamiento, y por tanto, a nuestros datos. Podemos estudiar
estos ciclos en la representación del comportamiento que obtenemos y ver que realmente
existe un ritmo circadiano perpetuo en cada uno de nosotros. Este tempo es en realidad
la componente periódica de 24 horas que construye la base sobre la que se asienta nuestro
comportamiento, no únicamente los patrones que cambian para cada persona. Mirando a las
trayectorias y variabilidades que nuestro comportamiento puede plasmar en los datos, podemos
apreciar que no existe un comportamiento único y repetitivo. En su lugar, hay varios
patrones o rutinas, obtenidas de nuestro propio diccionario, que elegimos para cada situación
especial. Al mismo tiempo, estas rutinas son combinaciones arbitrarias de diferentes escalas
de tiempo, correlaciones, niveles de movilidad, interacción social, calidad del sueño o iniciativa
para trabajar durante las mismas horas cada dÃa laborable. Juntas, estas propiedades
del comportamiento humano nos indican como debemos proceder a modelar su estructura,
no como funciones únicas, sino como un diccionario de perfiles ocultos de comportamiento,
Para descubrirlos, hemos considerado modelos de variables latentes.
La aplicación principal de los modelos estadÃsticos desarrollados para el aprendizaje de
comportamiento humano aparece en cuanto miramos a la medicina. Teniendo un modelo
personalizado que está ajustado de una manera precisa a los patrones de comportamiento
de un paciente, los cambios espontáneos en ellos pueden ser indicadores de futuras recaÃdas.
Desde un punto de vista técnico, la pregunta clásica suele ser si nuevas observaciones encajan
o no con lo indicado por el modelo. Este problema se puede enfocar desde dos perspectivas
que están interrelacionadas, una más orientada a la caracterización de aquellos objetos como
outliers, que usualmente se conoce como detección de anomalÃas, y otro enfocado en refrescar
el modelo de aprendizaje si este deja de ajustarse debidamente a los nuevos datos secuenciales.
Este último problema, ampliamente conocido como detección de puntos de cambio (cpd) es otro de los pilares de esta tesis. Estos métodos se han orientado a aplicaciones de salud
mental, y particularmente, a la detección pasiva de eventos crÃticos. El objetivo final es
proveer de una metodologÃa de detección temprana basada en el modelado probabilÃstico
para intervenciones rápidas. Por ejemplo, de cara a prever intentos de suicidio en pacientes
fuera de hospitales con trastornos afectivos severos de gran prevalencia, como depresión o
sÃndrome bipolar.Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan CarlosPresidente: Pablo MartÃnez Olmos.- Secretario: Daniel Hernández Lobato.- Vocal: Javier González Hernánde
Recommended from our members
Machine learning to model health with multimodal mobile sensor data
The widespread adoption of smartphones and wearables has led to the accumulation of rich datasets, which could aid the understanding of behavior and health in unprecedented detail. At the same time, machine learning and specifically deep learning have reached impressive performance in a variety of prediction tasks, but their use on time-series data appears challenging. Existing models struggle to learn from this unique type of data due to noise, sparsity, long-tailed distributions of behaviors, lack of labels, and multimodality.
This dissertation addresses these challenges by developing new models that leverage multi-task learning for accurate forecasting, multimodal fusion for improved population subtyping, and self-supervision for learning generalized representations. We apply our proposed methods to challenging real-world tasks of predicting mental health and cardio-respiratory fitness through sensor data.
First, we study the relationship of passive data as collected from smartphones (movement and background audio) to momentary mood levels. Our new training pipeline, which combines different sensor data into a low-dimensional embedding and clusters longitudinal user trajectories as outcome, outperforms traditional approaches based solely on psychology questionnaires. Second, motivated by mood instability as a predictor of poor mental health, we propose encoder-decoder models for time-series forecasting which exploit the bi-modality of mood with multi-task learning.
Next, motivated by the success of general-purpose models in vision and language tasks, we propose a self-supervised neural network ready-to-use as a feature extractor for wearable data. To this end, we set the heart rate responses as the supervisory signal for activity data, leveraging their underlying physiological relationship and show that the resulting task-agnostic embeddings can generalize in predicting structurally different downstream outcomes through transfer learning (e.g. BMI, age, energy expenditure), outperforming unsupervised autoencoders and biomarkers. Finally, acknowledging fitness as a strong predictor of overall health, which, however, can only be measured with expensive instruments (e.g., a VO2max test), we develop models that enable accurate prediction of fine-grained fitness levels with wearables in the present, and more importantly, its direction and magnitude almost a decade later.
All proposed methods are evaluated on large longitudinal datasets with tens of thousands of participants in the wild. The models developed and the insights drawn in this dissertation provide evidence for a better understanding of high-dimensional behavioral and physiological data with implications for large-scale health and lifestyle monitoring.The Department of Computer Science and Technology at the University of Cambridge through the EPSRC through Grant DTP (EP/N509620/1), and the Embiricos Trust Scholarship of Jesus College Cambridg
- …