Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition

Human activity recognition (HAR) in wearable computing is typically based on direct processing of sensor data. Sensor readings are translated into representations, either derived through dedicated preprocessing, or integrated into end-to-end learning. Independent of their origin, for the vast majority of contemporary HAR, those representations are typically continuous in nature. That has not always been the case. In the early days of HAR, discretization approaches have been explored - primarily motivated by the desire to minimize computational requirements, but also with a view on applications beyond mere recognition, such as, activity discovery, fingerprinting, or large-scale search. Those traditional discretization approaches, however, suffer from substantial loss in precision and resolution in the resulting representations with detrimental effects on downstream tasks. Times have changed and in this paper we propose a return to discretized representations. We adopt and apply recent advancements in Vector Quantization (VQ) to wearables applications, which enables us to directly learn a mapping between short spans of sensor data and a codebook of vectors, resulting in recognition performance that is generally on par with their contemporary, continuous counterparts - sometimes surpassing them. Therefore, this work presents a proof-of-concept for demonstrating how effective discrete representations can be derived, enabling applications beyond mere activity classification but also opening up the field to advanced tools for the analysis of symbolic sequences, as they are known, for example, from domains such as natural language processing. Based on an extensive experimental evaluation on a suite of wearables-based benchmark HAR tasks, we demonstrate the potential of our learned discretization scheme and discuss how discretized sensor data analysis can lead to substantial changes in HAR

Unsupervised online activity discovery using temporal behaviour assumption

We present a novel unsupervised approach, UnADevs, for discovering activity clusters corresponding to periodic and stationary activities in streaming sensor data. Such activities usually last for some time, which is exploited by our method; it includes mechanisms to regulate sensitivity to brief outliers and can discover multiple clusters overlapping in time to better deal with deviations from nominal behaviour. The method was evaluated on two activity datasets containing large number of activities (14 and 33 respectively) against online agglomerative clustering and DBSCAN. In a multi-criteria evaluation, our approach achieved significantly better performance on majority of the measures, with the advantages that: (i) it does not require to specify the number of clusters beforehand (it is open ended); (ii) it is online and can find clusters in real time; (iii) it has constant time complexity; (iv) and it is memory efficient as it does not keep the data samples in memory. Overall, it has managed to discover 616 of the total 717 activities. Because it discovers clusters of activities in real time, it is ideal to work alongside an active learning system

Early Abstraction of Inertial Sensor Data for Long-Term Deployments

Advances in microelectronics over the last decades have led to miniaturization of computing devices and sensors. A driving force to use these in various application scenarios is the desire to grasp physical phenomena from the environment, objects and living entities. We investigate sensing in two particularly challenging applications: one where small sensor modules are worn by people to detect their activities, and one where wirelessly networked sensors observe events over an area. This thesis takes a data-driven approach, focusing on human motion and vibrations caused by trains that are captured by accelerometer sensors as time series and shall be analyzed for characteristic patterns. For both, the acceleration sensor must be sampled at relatively high rates in order to capture the essence of the phenomena, and remain active for long stretches of time. The large amounts of gathered sensor data demand novel approaches that are able to swiftly process the data while guaranteeing accurate classification results. The following contributions are made in particular: * A data logger that would suit the requirements of long-term deployments is designed and evaluated. In a power profiling study both hardware components and firmware parameters are thoroughly tested, revealing that the sensor is able to log acceleration data at a sampling rate of 100 Hertz for up to 14 full days on a single battery charge. * A technique is proposed that swiftly and accurately abstracts an original signal with a set of linear segments, thus preserving its shape, while being twice as fast as a similar method. This allows for more efficient pattern matching, since for each pattern only a fraction of data points must be considered. A second study shows that this algorithm can perform data abstraction directly on a data logger with limited resources. * The railway monitoring scenario requires streaming vibration data to be analyzed for particular sparse and complex events directly on the sensor node, extracting relevant information such as train type or length from the shape of the vibration footprint. In a study conducted on real-world data, a set of efficient shape features is identified that facilitates train type prediction and length estimation with very high accuracies. * To achieve fast and accurate activity recognition for long-term bipolar patients monitoring scenarios, we present an approach that relies on the salience of motion patterns (motifs) that are characteristic for the target activity. These motifs are accumulated by using a symbolic abstraction that encodes the shape of the original signal. A large-scale study shows that a simple bag-of-words classifier trained with extracted motifs is on par with traditional approaches regarding the accuracy, while being much faster. * Some activities are hard to predict from acceleration data alone with the aforementioned approach. We argue that human-object interactions, captured as human motion and grasped objects through RFID, are an ideal supplement. A custom bracelet-like antenna to detect objects from up to 14 cm is proposed, along with a novel benchmark to evaluate such wearable setups. By aiming for wearable and wirelessly networked sensor systems, these contributions apply for particularly challenging applications that require long-term deployments of miniature sensors in general. They form the basis of a framework towards efficient event detection that relies heavily on early data abstraction and shape-based features for time series, while focusing less on the classification techniques

Activity-Based Human Mobility Patterns Inferred from Mobile Phone Data: A Case Study of Singapore

In this study, with Singapore as an example, we demonstrate how we can use mobile phone call detail record (CDR) data, which contains millions of anonymous users, to extract individual mobility networks comparable to the activity-based approach. Such an approach is widely used in the transportation planning practice to develop urban micro simulations of individual daily activities and travel; yet it depends highly on detailed travel survey data to capture individual activity-based behavior. We provide an innovative data mining framework that synthesizes the state-of-the-art techniques in extracting mobility patterns from raw mobile phone CDR data, and design a pipeline that can translate the massive and passive mobile phone records to meaningful spatial human mobility patterns readily interpretable for urban and transportation planning purposes. With growing ubiquitous mobile sensing, and shrinking labor and fiscal resources in the public sector globally, the method presented in this research can be used as a low-cost alternative for transportation and planning agencies to understand the human activity patterns in cities, and provide targeted plans for future sustainable development.Singapore. National Research Foundation (through the Singapore-MIT Alliance for Research and Technology (SMART) Center for Future Urban Mobility (FM))Center for Complex Engineering Systems at MIT and KACS

Zero-shot learning with matching networks for open-ended human activity recognition.

A real-world solution for Human Activity Recognition (HAR) should cover a variety of activities. However training a model to cover each and every possible activity is not practical. Instead we need a solution that can adapt its learning to unseen activities; referred to as open-ended HAR. Recent advancements in deep learning have increasingly begun to focus on the need to learn from few examples, referred to as k-shot learning and to go beyond this to transfer that learning to situations with unseen classes, referred to as zero-shot learning. The latter is particularly relevant to our research in open-ended HAR; and as yet remains unexplored. This paper presents our preliminary work with Zero-shot Learning (ZSL) with a Matching Network to address openended HAR. A Matching Network has the desirable property of learning with few examples and so is well suited to explorations in ZSL. We evaluate Matching Networks for ZSL with a HAR dataset. We propose the use of a variable length support set at test time to overcome the search for the best support set combination that currently plagues the fixed length support set size used by matching nets. Our results show that the variable length approach to be an effective strategy to maintain accuracy whilst avoiding the combinatorial search for the best class combination to form the support set

Essays on the Network Analysis of Culture

Nelle relazioni economiche, negli accordi internazionali e nel dialogo istituzionale, la parola distanza \ue8 una delle pi\uf9 enunciate. Ci sono distanze esogene da colmare per creare legami, a volte ci sono chiusure necessarie e altre volte rotture inevitabili, ma questo pu\uf2 dipendere, cos\uec come le distanze geografiche e fisiche, e gli interessi impliciti, in gran parte dallo status culturale di gruppi di individui. La valutazione quantitativa della distanza tra due entit\ue0 \ue8 una propriet\ue0 diadica ed in quanto tale, la presenza, intensit\ue0, direzione e segno di un legame rappresenta un modo per catturarla. Poich\ue9 le entit\ue0 possono essere individui, oggetti, societ\ue0, paesi, pianeti, cos\uec come reti che si riferiscono a contesti specifici, e il modo di misurare la somiglianza tra di loro pu\uf2 essere vario, una cosa peculiare delle distanze \ue8 la loro natura mutevole. Mentre le distanze fisiche sono quasi oggettivamente calcolabili, nel caso della cultura (ed anche di altri concetti pi\uf9 o meno ampi) l\u2019utilizzo di un metodo rispetto ad un altro potrebbe cambiare radicalmente la relazione di distanza tra le entit\ue0, soprattutto se esse hanno un alto grado di complessit\ue0. Il bagaglio culturale svolge un ruolo importante nel determinare lo status socio-economico di un paese e la sua caratterizzazione in termini di somiglianza con altri paesi. Il Capitolo 1 - utilizzando i dati della WVS/EVS Joint 2017 - operativizza una definizione di cultura che tiene conto delle interdipendenze tra tratti culturali a livello di paese e propone una nuova misura di distanza culturale. Sfruttando un recente algoritmo Bayesiano di Copula Gaussian graphical models, questo Capitolo stima per ciascuno di 76 paesi inclusi nella WVS/EVS Joint 2017, la rete culturale di interdipendenze tra tratti culturali considerando diversi insiemi di essi: i 6 della prima batteria di domande, i 10 della mappa culturale di Inglehart-Welzel, i 14 della mappa culturale di Inglehart-Welzel, dove per gli indici di \u201cPost-materialism\u201d e \u201cAutonomy\u201d sono state utilizzate le variabili da cui sono ricavate, e 60 tratti culturali dei quali, 14 come definiti in precedenza, 6 fanno riferimento alla prima batteria di domande e i restanti 40 sono selezionati in modo da ottenere un numero di variabili che possa far fronte al trade-off tra il tempo di elaborazione dell\u2019algoritmo e il minimo numero di valori mancanti per paese. Dopo aver definito le distanze tra i paesi considerando sia le reti culturali che le distribuzioni dei tratti culturali, attraverso il metodo DISTATIS, questo Capitolo osserva come l'aggiunta della componente di rete a quella distributiva classica, modifichi sostanzialmente la misura della distanza culturale sia nel caso di pochi tratti culturali (6, 10 e 14) che nel caso di pi\uf9 tratti culturali (60). Infine, esso afferma che la struttura di rete della cultura nazionale \ue8 importante per la definizione della distanza culturale tra i paesi del mondo e trova due misure finali di distanza: il Compromise_Large (da 60 variabili) e il Compromise_IW (dalle variabili della mappa culturale di Inglehart-Welzel). L'effetto delle variabili culturali sulla situazione economica di un paese, o pi\uf9 in generale di un'area geograficamente definita, \ue8 stato negli ultimi anni scandagliato dalla letteratura economica. Le distanze culturali, genetiche, geografiche, climatiche, semantiche, etniche, linguistiche, politiche sono state spesso incluse nei modelli econometrici come variabili indipendenti o di controllo. Il Capitolo 2 segue questa letteratura, prima confrontando individualmente tre misurazioni della distanza culturale calcolate nel Capitolo 1 con altre distanze usate in letteratura assieme alla distanza culturale o come proxy di essa, e poi confrontandole (le misure di distanza culturale e quelle dalla letteratura) congiuntamente tramite DISTATIS. Le tre distanze culturali sono le due nuove misure di cui sopra (Compromise_Large e Compromise_IW) e l'IW index ottenuto come distanza euclidea tra i paesi nella mappa culturale di Inglehart-Welzel, mentre le altre distanze prendono in considerazione la condizione climatica, l'etnia e la lingua, la genetica ed il recente fenomeno di Facebook. Infine, questo Capitolo considera tutte le misure di distanza all\u2019interno di un Social Relations Regression Model (SRRM) che stima la distanza tra i paesi in base al PIL pro capite (anno 2017). Il risultato finale mostra che le distanze culturali sono poco correlate con le distanze prese dalla letteratura, e quando si trova un compromesso tra di loro, di solito la Compromise_Large \ue8 caratterizzata da un peso leggermente superiore. La conclusione principale riguarda l'importante potere esplicativo della distanza Compromise_Large sulla distanza in PIL pro capite rispetto a quello della IW index e della Compromise_IW, la quale ha un significato intermedio tra le due. Ci\uf2 conferma l'importanza di considerare la rete culturale nazionale di interdipendenze tra tratti culturali nella definizione generale della distanza culturale, ed anche che l\u2019aggiunta di un numero maggiore di tratti culturali pu\uf2 influire nella sua specificazione, seppur i tratti culturali considerati da Ronald Inglehart e Christian Welzel nella costruzione della loro mappa culturale sembrano catturare gi\ue0 una buona parte dell\u2019informazione culturale dei paesi. La produzione abnorme di dati nel nostro tempo ha permesso l'osservazione di grandi collezioni di reti all\u2019interno di un campo di analisi specifico, le quali possono essere caratterizzate anche da una diversa dimensione l\u2019una dall\u2019altra (ad esempio si pu\uf2 pensare alla rete commerciale tra paesi di ogni prodotto). Una rete \ue8 un oggetto complesso, per cui un modo comune per analizzare e comparare congiuntamente un set di reti \ue8 ridurne la complessit\ue0 proiettandole in uno spazio ridotto attraverso i descrittori che le caratterizzano. \uc8 qui che sorge il problema analizzato nel Capitolo 3: qual \ue8 il sottoinsieme di descrittori che mantiene le caratteristiche delle reti il pi\uf9 possibile invariate nel processo di mapping, ovvero proietta in punti diversi dello spazio reti non isomorfe e raggruppa vicine reti strutturalmente simili tra di loro e lontano reti dissimili? Attraverso una simulazione di reti da quattro modelli generativi (Random, Scale-free, Small-world e Stochastic block model) e la selezione di un ampio insieme di descrittori riferenti ai livelli micro, meso e macro di analisi della rete, questo Capitolo trova tramite il metodo di Subgroup Discovery un piccolo sottoinsieme di descrittori. Questo sottoinsieme \ue8 composto da 5 descrittori: il momento primo del Coefficiente di Clustering Locale, 3 configurazioni di Motifs e il descrittore di Smallworldness. L'efficacia dei descrittori \ue8 valutata applicandoli all'insieme delle reti culturali binarie con 60 tratti culturali stimate nel Capitolo 1 e confrontando le distanze tra questi punti-rete nello spazio dei descrittori con distanze di reti popolari in letteratura. Le principali innovazioni sono due: la costruzione di un nuovo indice di distanza culturale tra i paesi, in cui \ue8 inclusa la rete culturale di interdipendenze tra tratti culturali; la selezione di un piccolo sottoinsieme efficiente di descrittori per la proiezione nello spazio di insiemi di reti binarie che possono avere grandezza diversa l\u2019una dall\u2019altra.In economic relations, in international agreements and in institutional dialogue, the word distance is one of the most enunciated. There are exogenous distances to be bridged to ignite a bond, sometimes there are necessary cracks and other times unavoidable breaks, but this may depend, as well as geographical and physical distances, and implicit interests, largely on the cultural status of groups of individuals. The quantitative evaluation of the distance between two entities is a dyadic property and as such, the presence, intensity, direction and sign of their tie is a way to undertake it. Since entities can be individuals, objects, companies, countries, planets, as well as networks referring to specific contexts, and the way to measure similarity between them is various, a peculiarity thing of distances is their changeable nature. While physical distances are almost objectively computable, in case of culture (and even other more or less broad concepts) using a method rather than another could radically change the proximity relationship between entities, especially if they have a high degree of complexity. The cultural background plays an important role in determining the socio-economic status of a country and its characterization in terms of similarity to other countries. The Chapter 1 - using data from the WVS/EVS Joint 2017 - operationalizes a definition of culture that takes into account the interdependencies between cultural traits at country level and calculates a new measure of cultural distance. Taking advantage of a recent Bayesian algorithm by Gaussian copula graphical model, this Chapter estimates for each of 76 countries included in the WVS/EVS Joint 2017, the cultural network of interdependencies between cultural traits considering different sets of them: the 6 from the first battery of questions, the 10 of the Inglehart-Welzel Cultural Map, the 14 of the Inglehart-Welzel Cultural Map, where for \u201cPost-materialism\u201d and \u201cAutonomy\u201d indices are used the variables from which they are derived, and 60 cultural traits of which, 14 as previously defined, 6 refer to the first battery of questions and the remaining 40 are selected to get a number that can cope with the trade-off between processing time and the minimum number of missing values per country. After defining the distances between countries considering both cultural networks and distributions of cultural traits, this Chapter observes via DISTATIS how the addition of the network component to the classic distributional one, substantially modifies the measure of cultural distance both in the case of a few cultural traits (6, 10 and 14) and in the case of more cultural traits (60). Finally, it affirms that the network structure of the national culture matters for the definition of the cultural distance among worldwide countries and finds two final distance measures: Compromise_Large (from 60 variables) and Compromise_IW (from the Inglehart-Welzel cultural map variables). The effect of cultural variables on the economic situation of a country or more generally of a geographically definable area, has been scoured in recent years by the economic literature. Cultural, genetic, geographical, climatic, semantic, ethnic, linguistic, political distances have often been included in econometric models as independent or control variables. The Chapter 2 follows this literature, firstly by individually comparing three measurements of cultural distance calculated in Chapter 1 with other distances used in literature together with cultural distance or as a proxy of it, and secondly by jointly comparing them (the measurements of cultural distance and those from literature) via DISTATIS. The three cultural distances are the two new measures mentioned above (Compromise_Large and Compromise_IW) and the IW index obtained as Euclidean distance between countries in the Inglehart-Welzel cultural map, while the other distances take into consideration climatic condition, ethnicity and language, genetics and the recent phenomenon of Facebook. Finally, this Chapter considers these distance measures into a Social Relations Regression Model (SRRM) which estimates the distance between countries in GDP per capita (year 2017). The final result shows that cultural distances are poorly correlated with the distances from the literature, and when a compromise is found between them, usually the Compromise_Large is characterized by a slightly higher weight. The main conclusion concerns the important explanatory power of the Compromise_Large distance on the distance in GDP per capita compared to that of the IW index and the Compromise_IW, which has an intermediate meaning between the two. This confirms the importance of considering the national cultural network of interdependencies between cultural traits in the overall definition of cultural distance, and also that the addition of more cultural traits may influence its specification, although the cultural traits considered by Inglehart and Welzel in the construction of their cultural map seem to capture already a good part of the cultural information of the countries. The abnormal production of data in our time has allowed the observation of large collections of networks within a specific field of analysis, which can also be characterized by a different size from each other, e.g. you can think of the trade network of each product between countries. A network is a complex object, so a common way to analyze and compare a set of networks is to reduce their complexity by mapping them into a space through the descriptors that characterize them. This is where the problem analyzed in Chapter 3 arises: what is the subset of descriptors that keeps the characteristics of networks as much as possible unchanged in the mapping process, namely projects non-isomorphic networks in different points of the space and groups nearby networks structurally similar and distant networks dissimilar? Through a simulation of networks from four generative models (Random, Scale-free, Small-world and Stochastic block model) and the selection of a wide set of descriptors of the micro, meso and macro-level of network analysis, this Chapter finds evidence of a small subset of descriptors via Subgroup Discovery. This subset is composed by 5 descriptors: the first moment of the Local Clustering Coefficient, 3 Motifs configurations and the descriptor of Smallworldness. The effectiveness of descriptors is evaluated by applying them to the set of binary cultural networks with 60 cultural traits estimated in Chapter 1 and comparing distances between these points-network in the space of the descriptors with popular network distances used in literature. Two are the main innovations: the construction of a new index of cultural distance among countries, in which is included the cultural network of interdependencies among cultural traits; the selection of a small efficient subset of descriptors for mapping in the space of sets of binary networks, which can also be characterized by a different size from each other

Lesson Learned from Collecting Quantified Self Information via Mobile and Wearable Devices

The ubiquity and affordability of mobile and wearable devices has enabled us to continually and digitally record our daily life activities. Consequently, we are seeing the growth of data collection experiments in several scientific disciplines. Although these have yielded promising results, mobile and wearable data collection experiments are often restricted to a specific configuration that has been designed for a unique study goal. These approaches do not address all the real-world challenges of “continuous data collection” systems. As a result, there have been few discussions or reports about such issues that are faced when “implementing these platforms” in a practical situation. To address this, we have summarized our technical and user-centric findings from three lifelogging and Quantified Self data collection studies, which we have conducted in real-world settings, for both smartphones and smartwatches. In addition to (i) privacy and (ii) battery related issues; based on our findings we recommend further works to consider (iii) implementing multivariate reflection of the data; (iv) resolving the uncertainty and data loss; and (v) consider to minimize the manual intervention required by users. These findings have provided insights that can be used as a guideline for further Quantified Self or lifelogging studies