Synthesizing Skeletal Motion and Physiological Signals as a Function of a Virtual Human's Actions and Emotions

Round-the-clock monitoring of human behavior and emotions is required in many healthcare applications which is very expensive but can be automated using machine learning (ML) and sensor technologies. Unfortunately, the lack of infrastructure for collection and sharing of such data is a bottleneck for ML research applied to healthcare. Our goal is to circumvent this bottleneck by simulating a human body in virtual environment. This will allow generation of potentially infinite amounts of shareable data from an individual as a function of his actions, interactions and emotions in a care facility or at home, with no risk of confidentiality breach or privacy invasion. In this paper, we develop for the first time a system consisting of computational models for synchronously synthesizing skeletal motion, electrocardiogram, blood pressure, respiration, and skin conductance signals as a function of an open-ended set of actions and emotions. Our experimental evaluations, involving user studies, benchmark datasets and comparison to findings in the literature, show that our models can generate skeletal motion and physiological signals with high fidelity. The proposed framework is modular and allows the flexibility to experiment with different models. In addition to facilitating ML research for round-the-clock monitoring at a reduced cost, the proposed framework will allow reusability of code and data, and may be used as a training tool for ML practitioners and healthcare professionals

Intent Prediction in Human-Human Interactions

The human ability to infer others' intent is innate and crucial to development. Machines ought to acquire this ability for seamless interaction with humans. We propose an agent model for predicting the intent of actors in human-human interactions. This requires simultaneous generation and recognition of an interaction at any time, for which end-to-end models are scarce. The proposed agent actively samples its environment via a sequence of glimpses. At each sampling instant, the model infers the observation class and completes the partially observed body motion. It learns the sequence of body locations to sample by jointly minimizing the classi�cation and generation errors. The model is evaluated on videos of two-skeleton interactions under two settings: (fi�rst person) one skeleton is the modeled agent and the other skeleton's joint movements constitute its visual observation, and (third person) an audience is the modeled agent and the two interacting skeletons' joint movements constitute its visual observation. Three methods for implementing the attention mechanism are analyzed using benchmark datasets. One of them, where attention is driven by sensory prediction error, achieves the highest classi�cation accuracy in both settings by sampling less than 50% of the skeleton joints, while also being the most efficient in terms of model size. This is the �first known attention-based agent to learn end-to-end from two-person interactions for intent prediction, with high accuracy and efficiency

An Attention-Based Predictive Agent for Static and Dynamic Environments

Real-world applications of intelligent agents demand accuracy and efficiency, and seldom provide reinforcement signals. Currently, most agent models are reinforcement-based and concentrate exclusively on accuracy. We propose a general-purpose agent model consisting of proprioceptive and perceptual pathways. The agent actively samples its environment via a sequence of glimpses. It completes the partial propriocept and percept sequences observed till each sampling instant, and learns where and what to sample by minimizing prediction error, without reinforcement or supervision (class labels). The model is evaluated by exposing it to two kinds of stimuli: images of fully-formed handwritten numerals and alphabets, and videos of gradual formation of numerals. It yields state-of-the-art prediction accuracy upon sampling only 22:6% of the scene on average. The model saccades when exposed to images and tracks when exposed to videos. This is the first known attention-based agent to generate realistic handwriting with state-of-the-art accuracy and efficiency by interacting with and learning end-to-end from static and dynamic environments

AttentionMNIST: A Mouse-Click Attention Tracking Dataset for Handwritten Numeral and Alphabet Recognition

Multiple attention-based models that recognize objects via a sequence of glimpses have reported results on handwritten numeral recognition. However, no attentiontracking data for handwritten numeral or alphabet recognition is available. Availability of such data would allow attention-based models to be evaluated in comparison to human performance. We collect mouse-click attention tracking (mcAT) data from 382 participants trying to recognize handwritten numerals and alphabets (upper and lowercase) from images via sequential sampling. Images from benchmark datasets are presented as stimuli. The collected dataset, called AttentionMNIST, consists of a sequence of sample (mouse click) locations, predicted class label(s) at each sampling, and the duration of each sampling. On average, our participants observe only 12.8% of an image for recognition. We propose a baseline model to predict the location and the class(es) a participant will select at the next sampling. When exposed to the same stimuli and experimental conditions as our participants, a highly-cited attention-based reinforcement model falls short of human e�ciency

Non-invasive assessment of liver disease in rats using multiparametric magnetic resonance imaging: a feasibility study

Non-invasive quantitation of liver disease using multiparametric magnetic resonance imaging (MRI) could refine clinical care pathways, trial design and preclinical drug development. The aim of this study was to evaluate the use of multiparametric MRI in experimental models of liver disease. Liver injury was induced in rats using 4 or 12 weeks of carbon tetrachloride (CCl4) intoxication and 4 or 8 weeks on a methionine and choline deficient (MCD) diet. Liver MRI was performed using a 7.0 Tesla small animal scanner at baseline and specified timepoints after liver injury. Multiparametric liver MRI parameters [T1 mapping, T2* mapping and proton density fat fraction (PDFF)] were correlated with gold standard histopathological measures. Mean hepatic T1 increased significantly in rats treated with CCl4 for 12 weeks compared to controls [1122±78 ms versus 959±114 ms; d=162.7, 95% CI (11.92, 313.4), P=0.038] and correlated strongly with histological collagen content (rs=0.717, P=0.037). In MCD diet-treated rats, hepatic PDFF correlated strongly with histological fat content (rs=0.819, P<0.0001), steatosis grade (rs=0.850, P<0.0001) and steatohepatitis score (rs=0.818, P<0.0001). Although there was minimal histological iron, progressive fat accumulation in MCD diet-treated livers significantly shortened T2*. In preclinical models, quantitative MRI markers correlated with histopathological assessments, especially for fatty liver disease. Validation in longitudinal studies is required. This article has an associated First Person interview with the first author of the paper

Pragmatically Framed Cross-Situational Noun Learning Using Computational Reinforcement Models

Cross-situational learning and social pragmatic theories are prominent mechanisms for learning word meanings (i.e., word-object pairs). In this paper, the role of reinforcement is investigated for early word-learning by an artificial agent. When exposed to a group of speakers, the agent comes to understand an initial set of vocabulary items belonging to the language used by the group. Both cross-situational learning and social pragmatic theory are taken into account. As social cues, joint attention and prosodic cues in caregiver's speech are considered. During agent-caregiver interaction, the agent selects a word from the caregiver's utterance and learns the relations between that word and the objects in its visual environment. The “novel words to novel objects” language-specific constraint is assumed for computing rewards. The models are learned by maximizing the expected reward using reinforcement learning algorithms [i.e., table-based algorithms: Q-learning, SARSA, SARSA-λ, and neural network-based algorithms: Q-learning for neural network (Q-NN), neural-fitted Q-network (NFQ), and deep Q-network (DQN)]. Neural network-based reinforcement learning models are chosen over table-based models for better generalization and quicker convergence. Simulations are carried out using mother-infant interaction CHILDES dataset for learning word-object pairings. Reinforcement is modeled in two cross-situational learning cases: (1) with joint attention (Attentional models), and (2) with joint attention and prosodic cues (Attentional-prosodic models). Attentional-prosodic models manifest superior performance to Attentional ones for the task of word-learning. The Attentional-prosodic DQN outperforms existing word-learning models for the same task

A self-organizing auto-associative network for the generalized physical design of microstrip patches

The current work deals with the efficient physical design of patch antennas given the desired parameters like resonant frequency fr, feed point position af, substrate thickness h, relative permittivity εr, input impedance Z (= R + jX), and efficiency η. Based loosely on the analogy of perception of the human brain, a neurocomputing network has been designed, consisting of two distinct phases, namely, the training phase and the application phase. The training phase accepts as input the exhaustive set of the said parameters for patches of different shapes and sizes and determines the optimized processors (processors that adequately define the information topology of the input data set) from the exhaustive training instances using a set of information extracting self-organizing neural networks. The outputs of the training phase are n sets of processors, n being the number of different shapes of patches taken into consideration. The application phase determines the shape and size of a microstrip antenna when its desired parameters are presented to the network as the external input. This is achieved by comparing the external input with each set of processors, hence determining the cost due to each comparison. A cost matrix is thus formed which when passed through an optimization network gives the best match and hence the shape and shape determining attributes of the patch whose parameters had been passed as external input

String tightening as a self-organizing phenomenon

The phenomenon of self-organization has been of special interest to the neural network community throughout the last couple of decades. In this paper, we study a variant of the self- organizing map (SOM) that models the phenomenon of self-organization of the particles forming a string when the string is tightened from one or both of its ends. The proposed variant, called the string tightening self-organizing neural network (STON), can be used to solve certain practical problems, such as computation of shortest homotopic paths, smoothing paths to avoid sharp turns, computation of convex hull, etc. These problems are of considerable interest in computational geometry, robotics path-planning, artificial intelligence (AI) (diagrammatic reasoning), very large scale integration (VLSI) routing, and geographical information systems. Given a set of obstacles and a string with two fixed terminal points in a 2-D space, the STON model continuously tightens the given string until the unique shortest configuration in terms of the Euclidean metric is reached. The STON minimizes the total length of a string on convergence by dynamically creating and selecting feature vectors in a competitive manner. Proof of correctness of this anytime algorithm and experimental results obtained by its deployment have been presented in the paper. © 2007 IEEE