Teaching Task Flow Through Dialog and Observation

In order for robots to act as valuable assistants for non-expert users, they need to be able to learn new abilities and do so through natural methods of communication. Furthermore, it is often desirable that tasks be learned quickly without having to provide multiple demonstrations. Training should also be conducted so that the user has a clear understanding of the manner in which environmental features affect the behavior of the learned activity, so that execution behavior is predictable. We present an interactive framework for teaching a robot the flow of an activity composed of elements from a set of primitive behaviors and previously trained activities. Conditional branching and looping, order-independent activity execution, and contingency (or interrupt) actions can all be captured by our activity structures. Additional convenience functionality to aid in the training process is also provided. By providing a natural method of communicating production rules analogous to rigid programming structures, tasks can be trained quickly and easily. We demonstrate our task training procedure on our CMAssist mobile robot

Teaching procedural flow through dialog and demonstration

Abstract — In order for robots to act as valuable assistants for non-expert users, they need to be able to learn new abilities and do so through natural methods of communication. Furthermore, it is often desirable that tasks be learned quickly without having to provide multiple demonstrations. Training should also be conducted in such a way that the user has a clear understanding of the manner in which environmental features affect the behavior of the learned activity, so that execution behavior is predictable. We present an interactive framework for teaching a robot the flow of an activity composed of elements from a set of primitive behaviors and previously trained activities. Conditional branching and looping, order-independent activity execution, and contingency (or interrupt) actions can all be captured by our activity structures. Additional convenience functionality to aid in the training process is also provided. By providing a natural method of communicating production rules analogous to rigid programming structures, well-defined tasks can be trained easily. We demonstrate our task training procedure on a mobile robot. I

Prioritized Multi-Hypothesis Tracking by a Robot with Limited Sensing

To act intelligently in dynamic environments, mobile robots must estimate object positions using information obtained from a variety of sources. We formally describe the problem of estimating the state of objects where a robot can only task its sensors to view one object at a time. We contribute an object tracking method that generates and maintains multiple hypotheses consisting of probabilistic state estimates that are generated by the individual information sources. These different hypotheses can be generated by the robot’s own prediction model and by communicating robot teammembers. The multiple hypotheses are often spatially disjoint and cannot simultaneously be verified by the robot’s limited sensors. Instead, the robot must decide towards which hypothesis its sensors should be tasked by evaluating each hypothesis on its likelihood of containing the object. Our contributed algorithm prioritizes the different hypotheses, according to rankings set by the expected uncertainty in the object’s motion model, as well as the uncertainties in the sources of information used to track their positions. We describe the algorithm in detail and show extensive empirical results in simulation as well as experiments on actual robots that demonstrate the effectiveness of our approach.

Vision-based bicycle detection and tracking using a deformable part model and an EKF algorithm

Abstract — Bicycles that share the road with intelligent vehicles present particular challenges for automated perception systems. Bicycle detection is important because bicycles share the road with vehicles and can move at comparable speeds in urban environments. From a computer vision standpoint, bicycle detection is challenging as bicycle’s appearance can change dramatically between viewpoints and a person riding on the bicycle is a non-rigid object. In this paper, we present a vision-based framework to detect and track bicycles that takes into account these issues. A mixture model of multiple viewpoints is defined and trained via a Support Vector Machine (SVM) to detect bicycles under a variety of circumstances. Each component of the model uses a part-based representation and known geometric context is used to improve overall detection efficiency. An extended Kalman filter (EKF) is used to estimate the position and velocity of the bicycle in vehicle coordinates. We demonstrate the effectiveness of this approach through a series of experiments run on video data of moving bicycles captured from a vehicle-mounted camera. I

A Human-Assisted Approach for a Mobile Robot to Learn 3D Object Models using Active Vision

Abstract — In this paper we present an algorithm that allows a human to naturally and easily teach a mobile robot how to recognize objects in its environment. The human selects the object by pointing at it using a laser pointer. The robot recognizes the laser reflections with its cameras and uses this data to generate an initial 2D segmentation of the object. The 3D position of SURF feature points are extracted from the designated area using stereo vision. As the robot moves around the object, new views of the object are obtained from which feature points are extracted. These features are filtered using active vision. The complete object representation consists of feature points registered with 3D pose data. We describe the method and show that it works well by performing experiments on real world data collected with our robot. We use an extensive dataset of 21 objects, differing in size, shape and texture. I

Teaching Task Flow Through Dialog and Observation

In order for robots to act as valuable assistants for non-expert users, they need to be able to learn new abilities and do so through natural methods of communication. Furthermore, it is often desirable that tasks be learned quickly without having to provide multiple demonstrations. Training should also be conducted so that the user has a clear understanding of the manner in which environmental features affect the behavior of the learned activity, so that execution behavior is predictable. We present an interactive framework for teaching a robot the flow of an activity composed of elements from a set of primitive behaviors and previously trained activities. Conditional branching and looping, order-independent activity execution, and contingency (or interrupt) actions can all be captured by our activity structures. Additional convenience functionality to aid in the training process is also provided. By providing a natural method of communicating production rules analogous to rigid programming structures, tasks can be trained quickly and easily. We demonstrate our task training procedure on our CMAssist mobile robot.</p

Vision-based 3D Bicycle Tracking using Deformable Part Model and Interacting Multiple Model Filter

Abstract — This paper presents a monocular vision based 3D bicycle tracking framework for intelligent vehicles based on a detection method exploiting a deformable part model and a tracking method using an Interacting Multiple Model (IMM) algorithm. Bicycle tracking is important because bicycles share the road with vehicles and can move at comparable speeds in urban environments. From a computer vision standpoint, bicycle detection is challenging as bicycle’s appearance can change dramatically between viewpoints and a person riding on the bicycle is a non-rigid object. To this end, we present a tracking-by-detection method to detect and track bicycles that takes into account these difficult issues. First, a mixture model of multiple viewpoints is defined and trained via a Latent Support Vector Machine (LSVM) to detect bicycles under a variety of circumstances. Each model uses a partbased representation. This robust bicycle detector provides a series of measurements (i.e., bounding boxes) in the context of the Kalman filter. Second, to exploit the unique characteristics of bicycle tracking, two motion models based on bicycle’s kinematics are fused using an IMM algorithm. For each motion model, an extended Kalman filter (EKF) is used to estimate the position and velocity of a bicycle in the vehicle coordinates. Finally, a single bicycle tracking method using an IMM algorithm is extended to that of multiple bicycle tracking by incorporating a Rao-Blackwellized Particle Filter which runs a particle filter for a data association and an IMM filter for each bicycle tracking. We demonstrate the effectiveness of this approach through a series of experiments run on a new bicycle dataset captured from a vehicle-mounted camera. I

Using Sparse Visual Data to Model Human Activities in Meetings

We have recently engaged on the challenging development of an agent to assist users in everyday office-related tasks. In particular, the agent needs to keep track of the state of their users so it can anticipate the user&apos;s needs and proactively address them. The state of the user may be easily available when the user directly interacts with their agent through a PC or PDA interface. However, when the user attends a meeting and interacts with other people, PC and PDA interfaces are not sufficient to give the agents a general view of the environment in which their users are interacting. In this paper, we introduce the CAMEO, the Camera Assisted Meeting Event Observer, which is a physical awareness system designed for use by an agent-based electronic assistant. We then present a particular aspect of CAMEO and main contribution of the paper, namely how CAMEO addresses the problem of extracting and reasoning about high-level features from real-time and continuous observation of a meeting environment. Contextual information about meetings and the interactions that take place with them is used to define Dynamic Bayesian Network classifiers to effectively infer the state of the users as well as a higher-level state of the meeting. We present and show results of the state inference algorithm