A Data-driven Approach Towards Human-robot Collaborative Problem Solving in a Shared Space

We are developing a system for human-robot communication that enables people to communicate with robots in a natural way and is focused on solving problems in a shared space. Our strategy for developing this system is fundamentally data-driven: we use data from multiple input sources and train key components with various machine learning techniques. We developed a web application that is collecting data on how two humans communicate to accomplish a task, as well as a mobile laboratory that is instrumented to collect data on how two humans communicate to accomplish a task in a physically shared space. The data from these systems will be used to train and fine-tune the second stage of our system, in which the robot will be simulated through software. A physical robot will be used in the final stage of our project. We describe these instruments, a test-suite and performance metrics designed to evaluate and automate the data gathering process as well as evaluate an initial data set.Comment: 2017 AAAI Fall Symposium on Natural Communication for Human-Robot Collaboratio

Injection locking of an electro-optomechanical device

The techniques of cavity optomechanics have enabled significant achievements in precision sensing, including the detection of gravitational waves and the cooling of mechanical systems to their quantum ground state. Recently, the inherent non-linearity in the optomechanical interaction has been harnessed to explore synchronization effects, including the spontaneous locking of an oscillator to a reference injection signal delivered via the optical field. Here, we present the first demonstration of a radiation-pressure driven optomechanical system locking to an inertial drive, with actuation provided by an integrated electrical interface. We use the injection signal to suppress drift in the optomechanical oscillation frequency, strongly reducing phase noise by over 55 dBc/Hz at 2 Hz offset. We further employ the injection tone to tune the oscillation frequency by more than 2 million times its narrowed linewidth. In addition, we uncover previously unreported synchronization dynamics, enabled by the independence of the inertial drive from the optical drive field. Finally, we show that our approach may enable control of the optomechanical gain competition between different mechanical modes of a single resonator. The electrical interface allows enhanced scalability for future applications involving arrays of injection-locked precision sensors.Comment: Main text: 10 pages, 7 figures. Supplementary Information: 5 pages, 4 figure

HP-GAN: Probabilistic 3D human motion prediction via GAN

Predicting and understanding human motion dynamics has many applications, such as motion synthesis, augmented reality, security, and autonomous vehicles. Due to the recent success of generative adversarial networks (GAN), there has been much interest in probabilistic estimation and synthetic data generation using deep neural network architectures and learning algorithms. We propose a novel sequence-to-sequence model for probabilistic human motion prediction, trained with a modified version of improved Wasserstein generative adversarial networks (WGAN-GP), in which we use a custom loss function designed for human motion prediction. Our model, which we call HP-GAN, learns a probability density function of future human poses conditioned on previous poses. It predicts multiple sequences of possible future human poses, each from the same input sequence but a different vector z drawn from a random distribution. Furthermore, to quantify the quality of the non-deterministic predictions, we simultaneously train a motion-quality-assessment model that learns the probability that a given skeleton sequence is a real human motion. We test our algorithm on two of the largest skeleton datasets: NTURGB-D and Human3.6M. We train our model on both single and multiple action types. Its predictive power for long-term motion estimation is demonstrated by generating multiple plausible futures of more than 30 frames from just 10 frames of input. We show that most sequences generated from the same input have more than 50\% probabilities of being judged as a real human sequence. We will release all the code used in this paper to Github

What does not happen: quantifying embodied engagement using NIMI and self-adaptors

Previous research into the quantification of embodied intellectual and emotional engagement using non-verbal movement parameters has not yielded consistent results across different studies. Our research introduces NIMI (Non-Instrumental Movement Inhibition) as an alternative parameter. We propose that the absence of certain types of possible movements can be a more holistic proxy for cognitive engagement with media (in seated persons) than searching for the presence of other movements. Rather than analyzing total movement as an indicator of engagement, our research team distinguishes between instrumental movements (i.e. physical movement serving a direct purpose in the given situation) and non-instrumental movements, and investigates them in the context of the narrative rhythm of the stimulus. We demonstrate that NIMI occurs by showing viewers’ movement levels entrained (i.e. synchronised) to the repeating narrative rhythm of a timed computer-presented quiz. Finally, we discuss the role of objective metrics of engagement in future context-aware analysis of human behaviour in audience research, interactive media and responsive system and interface design

Earth as a Hybrid Planet - The Anthropocene in an Evolutionary Astrobiological Context

We develop a classification scheme for the evolutionary state of planets based on the non-equilibrium thermodynamics of their coupled systems, including the presence of a biosphere and the possibility of what we call an agency-dominated biosphere (i.e. an energy-intensive technological species). The premise is that Earths entry into the Anthropocene represents what might be from an astrobiological perspective a predictable planetary transition. We explore this problem from the perspective of the solar system and exoplanet studies. Our classification discriminates planets by the forms of free energy generation driven from stellar forcing. We then explore how timescales for global evolutionary processes on Earth might be synchronized with ecological transformations driven by increases in energy harvesting and its consequences (which might have reached a turning point with global urbanization). Finally, we describe quantitatively the classification scheme based on the maintenance of chemical disequilibrium in the past and current Earth systems and on other worlds in the solar system. In this perspective, the beginning of the Anthropocene can be seen as the onset of the hybridization of the planet - a transitional stage from one class of planetary systems interaction to another. For Earth, this stage occurs as the effects of human civilization yield not just new evolutionary pressures, but new selected directions for novel planetary ecosystem functions and their capacity to generate disequilibrium and enhance planetary dissipation.Comment: Accepted for publication in the journal Anthropocen