Learning Contact-Rich Manipulation Skills with Guided Policy Search

Autonomous learning of object manipulation skills can enable robots to acquire rich behavioral repertoires that scale to the variety of objects found in the real world. However, current motion skill learning methods typically restrict the behavior to a compact, low-dimensional representation, limiting its expressiveness and generality. In this paper, we extend a recently developed policy search method \cite{la-lnnpg-14} and use it to learn a range of dynamic manipulation behaviors with highly general policy representations, without using known models or example demonstrations. Our approach learns a set of trajectories for the desired motion skill by using iteratively refitted time-varying linear models, and then unifies these trajectories into a single control policy that can generalize to new situations. To enable this method to run on a real robot, we introduce several improvements that reduce the sample count and automate parameter selection. We show that our method can acquire fast, fluent behaviors after only minutes of interaction time, and can learn robust controllers for complex tasks, including putting together a toy airplane, stacking tight-fitting lego blocks, placing wooden rings onto tight-fitting pegs, inserting a shoe tree into a shoe, and screwing bottle caps onto bottles

Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

Reinforcement learning (RL) constitutes a promising solution for alleviating the problem of traffic congestion. In particular, deep RL algorithms have been shown to produce adaptive traffic signal controllers that outperform conventional systems. However, in order to be reliable in highly dynamic urban areas, such controllers need to be robust with the respect to a series of exogenous sources of uncertainty. In this paper, we develop an open-source callback-based framework for promoting the flexible evaluation of different deep RL configurations under a traffic simulation environment. With this framework, we investigate how deep RL-based adaptive traffic controllers perform under different scenarios, namely under demand surges caused by special events, capacity reductions from incidents and sensor failures. We extract several key insights for the development of robust deep RL algorithms for traffic control and propose concrete designs to mitigate the impact of the considered exogenous uncertainties.Comment: 8 page

An on-line training radial basis function neural network for optimum operation of the UPFC

The concept of Flexible A.C. Transmission Systems (FACTS) technology was developed to enhance the performance of electric power networks (both in steady-state and transient-state) and to make better utilization of existing power transmission facilities. The continuous improvement in power ratings and switching performance of power electronic devices together with advances in circuit design and control techniques are making this concept and devices employed in FACTS more commercially attractive. The Unified Power Flow Controller (UPFC) is one of the main FACTS devices that have a wide implication on the power transmission systems and distribution. The purpose of this paper is to explore the use of Radial Basis Function Neural Network (RBFNN) to control the operation of the UPFC in order to improve its dynamic performance. The performance of the proposed controller compares favourably with the conventional PI and the off-line trained controller. The simple structure of the proposed controller reduces the computational requirements and emphasizes its appropriateness for on-line operation. Real-time implementation of the controller is achieved through using dSPACE ds1103 control and data acquisition board. Simulation and experimental results are presented to demonstrate the robustness of the proposed controller against changes in the transmission system operating conditions