Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

Identifying evolving multivariate dynamics in individual and cohort time series, with application to physiological control systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 117-125).Physiological control systems involve multiple interacting variables operating in feedback loops that enhance an organism's ability to self-regulate and respond to internal and external disturbances. The resulting multivariate time-series often exhibit rich dynamical patterns, which are altered under pathological conditions. However, model identification for physiological systems is complicated by measurement artifacts and changes between operating regimes. The overall aim of this thesis is to develop and validate computational tools for identification and analysis of structured multivariate models of physiological dynamics in individual and cohort time-series. We first address the identification and stability of the respiratory chemoreflex system, which is key to the pathogenesis of sleep-induced periodic breathing and apnea. Using data from both an animal model of periodic breathing, as well as human recordings from clinical sleep studies, we demonstrate that model-based analysis of the interactions involved in spontaneous breathing can characterize the dynamics of the respiratory control system, and provide a useful tool for quantifying the contribution of various dynamic factors to ventilatory instability. The techniques have suggested novel approaches to titration of combination therapies, and clinical evaluations are now underway. We then study shared multivariate dynamics in physiological cohort time-series, assuming that the time-series are generated by switching among a finite collection of physiologically constrained dynamical models. Patients whose time-series exhibit similar dynamics may be grouped for monitoring and outcome prediction. We develop a novel parallelizable machine-learning algorithm for outcome-discriminative identification of the switching dynamics, using a probabilistic dynamic Bayesian network to initialize a deterministic neural network classifier. In validation studies involving simulated data and human laboratory recordings, the new technique significantly outperforms the standard expectation-maximization approach for identification of switching dynamics. In a clinical application, we show the prognostic value of assessing evolving dynamics in blood pressure time-series to predict mortality in a cohort of intensive care unit patients. A better understanding of the dynamics of physiological systems in both health and disease may enable clinicians to direct therapeutic interventions targeted to specific underlying mechanisms. The techniques developed in this thesis are general, and can be extended to other domains involving multi-dimensional cohort time-series.by Shamim Nemati.Ph.D

Biomechatronics: Harmonizing Mechatronic Systems with Human Beings

This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphasis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level

Contextualizing the Dynamics of Affective Functioning: Conceptual and Statistical Considerations

Aktuelle Affektforschung betont die Bedeutung mikrolängsschnittlicher Daten für das Verstehen täglichen affektiven Funktionierens, da sie es erlauben affektive Dynamiken und potentiell zugrunde liegende Prozesse zu beschreiben. Dynamische Längsschnittmodelle werden entsprechend attraktiver. In dieser Dissertation komme ich Forderungen nach einer Integration kontextueller Informationen in die Untersuchung täglichen affektiven Funktionierens nach. Speziell modifiziere ich populäre dynamische Modelle so, dass sie kontextuelle Variationen einbeziehen. In einem ersten Beitrag werden Personen als in Kontexte eingebettet begriffen. Der vorgeschlagene Ansatz der festen moderierten Zeitreihenanalyse berücksichtigt systemische Reaktionen auf kontextuelle Veränderungen, indem Veränderungen in allen Parametern eines dynamischen Zeitreihenmodells auf kontextuelle Veränderungen bedingt schätzt werden. Kontextuelle Veränderungen werden als bekannt und assoziierte Parameterveränderungen als deterministisch behandelt. Folglich sind Modellspezifikation und -schätzung erleichtert und in kleineren Stichproben praktikabel. Es sind allerdings Informationen über den Einfluss kontextueller Faktoren erforderlich. Anwendbar auf einzelne Personen erlaubt der Ansatz die uneingeschränkte Exploration interindividueller Unterschiede in kontextualisierten affektiven Dynamiken. In einem zweiten Beitrag werden Personen als mit Kontexten interagierend begriffen. Ich implementiere eine Prozessperspektive auf kontextuelle Schwankungen, die die Dynamiken täglicher Ereignisse über autoregressive Modelle mit Poisson Messfehler abbildet. Die Kombination von Poisson und Gaußscher autoregressiver Modellierung erlaubt eine Formalisierung des dynamischen Zusammenspiels kontextueller und affektiver Prozesse. Die Modelle sind hierarchisch aufgesetzt und erfassen so interindividuelle Unterschiede in intraindividuellen Dynamiken. Die Schätzung erfolgt über simulationsbasierte Verfahren der Bayesschen Statistik.Recent affect research stresses the importance of micro-longitudinal data for understanding daily affective functioning, as they allow describing affective dynamics and potentially underlying processes. Accordingly, dynamic longitudinal models get increasingly promoted. In this dissertation, I address calls for an integration of contextual information into the study of daily affective functioning. Specifically, I modify popular dynamic models so that they incorporate contextual changes. In a first contribution, individuals are characterized as embedded in contexts. The proposed approach of fixed moderated time series analysis accounts for systemic reactions to contextual changes by estimating change in all parameters of a dynamic time series model conditional on contextual changes. It thus treats contextual changes as known and related parameter changes as deterministic. Consequently, model specification and estimation are facilitated and feasible in smaller samples, but information on which and how contextual factors matter is required. Applicable to single individuals, the approach permits an unconstrained exploration of inter-individual differences in contextualized affective dynamics. In a second contribution, individuals are characterized as interacting reciprocally with contexts. Implementing a process perspective on contextual changes, I model the dynamics of daily events using autoregressive models with Poisson measurement error. Combining Poisson and Gaussian autoregressive models can formalize the dynamic interplay between contextual and affective processes. It thereby distinguishes not only unique from joint dynamics, but also affective reactivity from situation selection, evocation, or anticipation. The models are set up as hierarchical to capture inter-individual differences in intra-individual dynamics. Estimation is carried out via simulation-based techniques in the Bayesian framework

Low-Cost Sensors and Biological Signals

Many sensors are currently available at prices lower than USD 100 and cover a wide range of biological signals: motion, muscle activity, heart rate, etc. Such low-cost sensors have metrological features allowing them to be used in everyday life and clinical applications, where gold-standard material is both too expensive and time-consuming to be used. The selected papers present current applications of low-cost sensors in domains such as physiotherapy, rehabilitation, and affective technologies. The results cover various aspects of low-cost sensor technology from hardware design to software optimization

Wearable in-ear pulse oximetry: theory and applications

Wearable health technology, most commonly in the form of the smart watch, is employed by millions of users worldwide. These devices generally exploit photoplethysmography (PPG), the non-invasive use of light to measure blood volume, in order to track physiological metrics such as pulse and respiration. Moreover, PPG is commonly used in hospitals in the form of pulse oximetry, which measures light absorbance by the blood at different wavelengths of light to estimate blood oxygen levels (SpO2). This thesis aims to demonstrate that despite its widespread usage over many decades, this sensor still possesses a wealth of untapped value. Through a combination of advanced signal processing and harnessing the ear as a location for wearable sensing, this thesis introduces several novel high impact applications of in-ear pulse oximetry and photoplethysmography. The aims of this thesis are accomplished through a three pronged approach: rapid detection of hypoxia, tracking of cognitive workload and fatigue, and detection of respiratory disease. By means of the simultaneous recording of in-ear and finger pulse oximetry at rest and during breath hold tests, it was found that in-ear SpO2 responds on average 12.4 seconds faster than the finger SpO2. This is likely due in part to the ear being in close proximity to the brain, making it a priority for oxygenation and thus making wearable in-ear SpO2 a good proxy for core blood oxygen. Next, the low latency of in-ear SpO2 was further exploited in the novel application of classifying cognitive workload. It was found that in-ear pulse oximetry was able to robustly detect tiny decreases in blood oxygen during increased cognitive workload, likely caused by increased brain metabolism. This thesis demonstrates that in-ear SpO2 can be used to accurately distinguish between different levels of an N-back memory task, representing different levels of mental effort. This concept was further validated through its application to gaming and then extended to the detection of driver related fatigue. It was found that features derived from SpO2 and PPG were predictive of absolute steering wheel angle, which acts as a proxy for fatigue. The strength of in-ear PPG for the monitoring of respiration was investigated with respect to the finger, with the conclusion that in-ear PPG exhibits far stronger respiration induced intensity variations and pulse amplitude variations than the finger. All three respiratory modes were harnessed through multivariate empirical mode decomposition (MEMD) to produce spirometry-like respiratory waveforms from PPG. It was discovered that these PPG derived respiratory waveforms can be used to detect obstruction to breathing, both through a novel apparatus for the simulation of breathing disorders and through the classification of chronic obstructive pulmonary disease (COPD) in the real world. This thesis establishes in-ear pulse oximetry as a wearable technology with the potential for immense societal impact, with applications from the classification of cognitive workload and the prediction of driver fatigue, through to the detection of chronic obstructive pulmonary disease. The experiments and analysis in this thesis conclusively demonstrate that widely used pulse oximetry and photoplethysmography possess a wealth of untapped value, in essence teaching the old PPG sensor new tricks.Open Acces