Towards streaming gesture recognition

The emergence of low-cost sensors allows more devices to be equipped with various types of sensors. In this way, mobile device such as smartphones or smartwatches now may contain accelerometers, gyroscopes, etc. This offers new possibilities for interacting with the environment and benefits would come to exploit these sensors. As a consequence, the literature on gesture recognition systems that employ such sensors grow considerably. The literature regarding online gesture recognition counts many methods based on Dynamic Time Warping (DTW). However, this method was demonstrated has non-efficient for time series from inertial sensors unit as a lot of noise is present. In this way new methods based on LCSS (Longest Common SubSequence) were introduced. Nevertheless, none of them focus on a class optimization process. In this master thesis, we present and evaluate a new algorithm for online gesture recognition in noisy streams. This technique relies upon the LM-WLCSS (Limited Memory and Warping LCSS) algorithm that has demonstrated its efficiency on gesture recognition. This new method involves a quantization step (via the K-Means clustering algorithm) that transforms new data to a finite set. In this way, each new sample can be compared to several templates (one per class). Gestures are rejected based on a previously trained rejection threshold. Thereafter, an algorithm, called SearchMax, find a local maximum within a sliding window and output whether or not the gesture has been recognized. In order to resolve conflicts that may occur, another classifier (i.e. C4.5) could be completed. As the K-Means clustering algorithm needs to be initialized with the number of clusters to create, we also introduce a straightforward optimization process. Such an operation also optimizes the window size for the SearchMax algorithm. In order to demonstrate the robustness of our algorithm, an experiment has been performed over two different data sets. However, results on tested data sets are only accurate when training data are used as test data. This may be due to the fact that the method is in an overlearning state. L’apparition de nouveaux capteurs à bas prix a permis d’en équiper dans beaucoup plus d’appareils. En effet, dans les appareils mobiles tels que les téléphones et les montres intelligentes nous retrouvons des accéléromètres, gyroscopes, etc. Ces capteurs présents dans notre vie quotidienne offrent de toutes nouvelles possibilités en matière d’interaction avec notre environnement et il serait avantageux de les utiliser. Cela a eu pour conséquence une augmentation considérable du nombre de recherches dans le domaine de reconnaissance de geste basé sur ce type de capteur. La littérature concernant la reconnaissance de gestes en ligne comptabilise beaucoup de méthodes qui se basent sur Dynamic Time Warping (DTW). Cependant, il a été démontré que cette méthode se révèle inefficace en ce qui concerne les séries temporelles provenant d’une centrale à inertie puisqu’elles contiennent beaucoup de bruit. En ce sens de nouvelles méthodes basées sur LCSS (Longest Common SubSequence) sont apparues. Néanmoins, aucune d’entre elles ne s’est focalisée sur un processus d’optimisation par class. Ce mémoire de maîtrise consiste en une présentation et une évaluation d’un nouvel algorithme pour la reconnaissance de geste en ligne avec des données bruitées. Cette technique repose sur l’algorithme LM-WLCSS (Limited Memory and Warping LCSS) qui a d’ores et déjà démontré son efficacité quant à la reconnaissance de geste. Cette nouvelle méthode est donc composée d’une étape dite de quantification (grâce à l’algorithme de regroupement K-Means) qui se charge de convertir les nouvelles données entrantes vers un ensemble de données fini. Chaque nouvelle donnée peut donc être comparée à plusieurs motifs (un par classe) et un geste est reconnu dès lors que son score dépasse un seuil préalablement entrainé. Puis, un autre algorithme appelé SearchMax se charge de trouver un maximum local au sein d’une fenêtre glissant afin de préciser si oui ou non un geste a été reconnu. Cependant des conflits peuvent survenir et en ce sens un autre classifieur (c.-àd. C4.5) est chainé. Étant donné que l’algorithme de regroupement K-Means a besoin d’une valeur pour le nombre de regroupements à faire, nous introduisons également une technique simple d’optimisation à ce sujet. Cette partie d’optimisation se charge également de trouver la meilleure taille de fenêtre possible pour l’algorithme SearchMax. Afin de démontrer l’efficacité et la robustesse de notre algorithme, nous l’avons testé sur deux ensembles de données différents. Cependant, les résultats sur les ensembles de données testées n’étaient bons que lorsque les données d’entrainement étaient utilisées en tant que données de test. Cela peut être dû au fait que la méthode est dans un état de surapprentissage

Unifying interaction across distributed controls in a smart environment using anthropology-based computing to make human-computer interaction "Calm"

Rather than adapt human behavior to suit a life surrounded by computerized systems, is it possible to adapt the systems to suit humans? Mark Weiser called for this fundamental change to the design and engineering of computer systems nearly twenty years ago. We believe it is possible and offer a series of related theoretical developments and practical experiments designed in an attempt to build a system that can meet his challenge without resorting to black box design principles or Wizard of Oz protocols. This culminated in a trial involving 32 participants, each of whom used two different multimodal interactive techniques, based on our novel interaction paradigm, to intuitively control nine distributed devices in a smart home setting. The theoretical work and practical developments have led to our proposal of seven contributions to the state of the art

Recognizing cardiovascular disease patterns with machine learning using NHANES accelerometer determined physical activity data

The relationship between physical activity (PA) and cardiovascular disease (CVD) is well established; however, questions about the appropriate dose of PA to reduce CVD risk still remain (Blair, LaMonte, & Nichaman, 2004; Pate et al., 1995). The optimal dose and the effects of intensity, duration, and frequency of PA are not fully understood (Haskell et al., 2007). This study connects objectively measured PA with a cross-sectional measure of CVD risk for an in-depth analysis of PA patterns that contribute to higher risk of CVD. Specifically, this study applied machine learning algorithms to NHANES accelerometer data from the 2003-2006 cohorts with the Reynolds cardiovascular risk score as the outcome. Using accelerometer data as a proxy for the Reynold's risk score to study cardiovascular disease risk allows the use of cross-sectional data when the longitudinal outcome is not known. A major benefit of using accelerometers to objectively measure of PA is that the data is easy and inexpensive to obtain. Furthermore, most locomotive activities are measured with a high degree of accuracy. Accelerometers can gather highly detailed information about an individual’s PA pattern over extended periods of time. This produces a large amount of data that requires specialized techniques to analyze. The analysis for this study was conducted using a variety of machine learning techniques to identify individual patterns in the data and evaluate what contributes most to high CVD risk. Comparison of machine learning algorithms shows that all classifiers perform well when given appropriate features. Using predefined intensity thresholds to compute average time spent in a PA category yielded good classification results in identifying study participants at high and low risk for CVD (Troiano et al., 2008). Adding PA pattern-related features to the model did not appear to improve classification. Features derived using k-means and the Hidden Markov Model (HMM) performed on the level of using predefined intensity thresholds, indicating that data driven methods may be used for feature extraction without relying on prior knowledge of the data. In general, the lasso regression, support vector machines (SVM) and random forest (RF) classifiers all performed well on large sets of data-driven features, achieving greater than 82% classification accuracy when time spent in PA intensity categories was combined with k-means and HMM-derived inputs. Neural networks performed well on smaller uncorrelated feature sets, and decision trees produced consistent results with the most transparency and interpretability. With respect to physical activity recommendations, the findings indicate that gender and time spent in lifestyle minutes (760-2019 intensity counts) play a key role in classifying CVD risk. Thus, a greater emphasis on gender specific recommendations focusing on lifestyle minutes in addition to moderate and vigorous activity may be necessary. Furthermore, time spent in the activity categories, not how PA is spread throughout the day and week appear to be most important for classification of CVD risk