Empirical Evaluation of Contextual Policy Search with a Comparison-based Surrogate Model and Active Covariance Matrix Adaptation

Contextual policy search (CPS) is a class of multi-task reinforcement learning algorithms that is particularly useful for robotic applications. A recent state-of-the-art method is Contextual Covariance Matrix Adaptation Evolution Strategies (C-CMA-ES). It is based on the standard black-box optimization algorithm CMA-ES. There are two useful extensions of CMA-ES that we will transfer to C-CMA-ES and evaluate empirically: ACM-ES, which uses a comparison-based surrogate model, and aCMA-ES, which uses an active update of the covariance matrix. We will show that improvements with these methods can be impressive in terms of sample-efficiency, although this is not relevant any more for the robotic domain.Comment: Supplementary material for poster paper accepted at GECCO 2019; https://doi.org/10.1145/3319619.332193

Learning Dynamic Robot-to-Human Object Handover from Human Feedback

Object handover is a basic, but essential capability for robots interacting with humans in many applications, e.g., caring for the elderly and assisting workers in manufacturing workshops. It appears deceptively simple, as humans perform object handover almost flawlessly. The success of humans, however, belies the complexity of object handover as collaborative physical interaction between two agents with limited communication. This paper presents a learning algorithm for dynamic object handover, for example, when a robot hands over water bottles to marathon runners passing by the water station. We formulate the problem as contextual policy search, in which the robot learns object handover by interacting with the human. A key challenge here is to learn the latent reward of the handover task under noisy human feedback. Preliminary experiments show that the robot learns to hand over a water bottle naturally and that it adapts to the dynamics of human motion. One challenge for the future is to combine the model-free learning algorithm with a model-based planning approach and enable the robot to adapt over human preferences and object characteristics, such as shape, weight, and surface texture.Comment: Appears in the Proceedings of the International Symposium on Robotics Research (ISRR) 201

A Maximum-Entropy Partial Parser for Unrestricted Text

This paper describes a partial parser that assigns syntactic structures to sequences of part-of-speech tags. The program uses the maximum entropy parameter estimation method, which allows a flexible combination of different knowledge sources: the hierarchical structure, parts of speech and phrasal categories. In effect, the parser goes beyond simple bracketing and recognises even fairly complex structures. We give accuracy figures for different applications of the parser.Comment: 9 pages, LaTe

Event Organization 101: Understanding Latent Factors of Event Popularity

The problem of understanding people's participation in real-world events has been a subject of active research and can offer valuable insights for human behavior analysis and event-related recommendation/advertisement. In this work, we study the latent factors for determining event popularity using large-scale datasets collected from the popular Meetup.com EBSN in three major cities around the world. We have conducted modeling analysis of four contextual factors (spatial, group, temporal, and semantic), and also developed a group-based social influence propagation network to model group-specific influences on events. By combining the Contextual features And Social Influence NetwOrk, our integrated prediction framework CASINO can capture the diverse influential factors of event participation and can be used by event organizers to predict/improve the popularity of their events. Evaluations demonstrate that our CASINO framework achieves high prediction accuracy with contributions from all the latent features we capture.Comment: International AAAI Conference on Web and Social Media (ICWSM) 2017 https://www.aaai.org/ocs/index.php/ICWSM/ICWSM17/paper/view/1557