Learning Contact-Rich Manipulation Skills with Guided Policy Search

Autonomous learning of object manipulation skills can enable robots to acquire rich behavioral repertoires that scale to the variety of objects found in the real world. However, current motion skill learning methods typically restrict the behavior to a compact, low-dimensional representation, limiting its expressiveness and generality. In this paper, we extend a recently developed policy search method \cite{la-lnnpg-14} and use it to learn a range of dynamic manipulation behaviors with highly general policy representations, without using known models or example demonstrations. Our approach learns a set of trajectories for the desired motion skill by using iteratively refitted time-varying linear models, and then unifies these trajectories into a single control policy that can generalize to new situations. To enable this method to run on a real robot, we introduce several improvements that reduce the sample count and automate parameter selection. We show that our method can acquire fast, fluent behaviors after only minutes of interaction time, and can learn robust controllers for complex tasks, including putting together a toy airplane, stacking tight-fitting lego blocks, placing wooden rings onto tight-fitting pegs, inserting a shoe tree into a shoe, and screwing bottle caps onto bottles

Content based image pose manipulation

This thesis proposes the application of space-frequency transformations to the domain of pose estimation in images. This idea is explored using the Wavelet Transform with illustrative applications in pose estimation for face images, and images of planar scenes. The approach is based on examining the spatial frequency components in an image, to allow the inherent scene symmetry balance to be recovered. For face images with restricted pose variation (looking left or right), an algorithm is proposed to maximise this symmetry in order to transform the image into a fronto-parallel pose. This scheme is further employed to identify the optimal frontal facial pose from a video sequence to automate facial capture processes. These features are an important pre-requisite in facial recognition and expression classification systems. The under lying principles of this spatial-frequency approach are examined with respect to images with planar scenes. Using the Continuous Wavelet Transform, full perspective planar transformations are estimated within a featureless framework. Restoring central symmetry to the wavelet transformed images in an iterative optimisation scheme removes this perspective pose. This advances upon existing spatial approaches that require segmentation and feature matching, and frequency only techniques that are limited to affine transformation recovery. To evaluate the proposed techniques, the pose of a database of subjects portraying varying yaw orientations is estimated and the accuracy is measured against the captured ground truth information. Additionally, full perspective homographies for synthesised and imaged textured planes are estimated. Experimental results are presented for both situations that compare favourably with existing techniques in the literature

Novel control of a high performance rotary wood planing machine

Rotary planing, and moulding, machining operations have been employed within the woodworking industry for a number of years. Due to the rotational nature of the machining process, cuttermarks, in the form of waves, are created on the machined timber surface. It is the nature of these cuttermarks that determine the surface quality of the machined timber. It has been established that cutting tool inaccuracies and vibrations are a prime factor in the form of the cuttermarks on the timber surface. A principal aim of this thesis is to create a control architecture that is suitable for the adaptive operation of a wood planing machine in order to improve the surface quality of the machined timber. In order to improve the surface quality, a thorough understanding of the principals of wood planing is required. These principals are stated within this thesis and the ability to manipulate the rotary wood planing process, in order to achieve a higher surface quality, is shown. An existing test rig facility is utilised within this thesis, however upgrades to facilitate higher cutting and feed speeds, as well as possible future implementations such as extended cutting regimes, the test rig has been modified and enlarged. This test rig allows for the dynamic positioning of the centre of rotation of the cutterhead during a cutting operation through the use of piezo electric actuators, with a displacement range of ±15μm. A new controller for the system has been generated. Within this controller are a number of tuneable parameters. It was found that these parameters were dependant on a high number external factors, such as operating speeds and run‐out of the cutting knives. A novel approach to the generation of these parameters has been developed and implemented within the overall system. Both cutterhead inaccuracies and vibrations can be overcome, to some degree, by the vertical displacement of the cutterhead. However a crucial information element is not known, the particular displacement profile. Therefore a novel approach, consisting of a subtle change to the displacement profile and then a pattern matching approach, has been implemented onto the test rig. Within the pattern matching approach the surface profiles are simplified to a basic form. This basic form allows for a much simplified approach to the pattern matching whilst producing a result suitable for the subtle change approach. In order to compress the data levels a Principal Component Analysis was performed on the measured surface data. Patterns were found to be present in the resultant data matrix and so investigations into defect classification techniques have been carried out using both K‐Nearest Neighbour techniques and Neural Networks. The application of these novel approaches has yielded a higher system performance, for no additional cost to the mechanical components of the wood planing machine, both in terms of wood throughput and machined timber surface quality

Design and Control Integration of a Reactive Distillation Column for Ethyl Lactate Production

ABSTRACT: Nowadays, the worldwide tendency to obtain environmentally friendly products through the use of safe and stable production processes, minimizing the energy consumption (i.e. using energy integration), and avoiding products out of specification, are an important motivation for applying a process design methodology that incorporates controllability issues since the earliest design stages. Although the topic of design-control integration has been a research topic investigated from different fronts for more than thirty-five years, it was in 2005 where a methodology incorporating local practical controllability issues for nonlinear systems was proposed. Such methodology allows designing processes that fulfill some controllability criteria, which assures that the resulted design will be controllable from the modern control theory. The mentioned design-control integration methodology was applied in this work for designing a reactive distillation column for producing ethyl lactate, an important green solvent. Production of this green solvent has gained great attention worldwide since it is seen as an excellent alternative for replacing petroleum-based solvents. As with any green product that intends to replace oil-based products, ethyl lactate production needs to be improved (in terms of its economic feasibility) to have an actual chance for replacing the petroleum-based solvents at a worldwide scale. One of the proposals for improving the economic feasibility of this green solvent, is to produce it in a reactive distillation column system, which would reduce the energy consumption, increasing the process profit. The design-control methodology applied here involved several steps. First, the development of a first principles-based model is required. Unfortunately, experimental data for a reactive distillation system for ethyl lactate production are scarce. Therefore, the model was identified and validated using data generated by running simulations in Aspen Plus. After model validation, simulated data were used in conjunction with knowledge of the process (obtained from technical literature) to select the state variables to be controlled. Then the manipulated and controlled variables were paired by applying digraphs theory, which avoids linearization of the nonlinear model. After this, local practical controllability metrics were formulated for being used as constraints during the optimization step of the design-control methodology. Besides the controllability metrics, physical constraints as well as product specifications constraints were included in the optimization. To compare the integrated design methodology with a traditional design methodology, the optimization was also run but considering only the physical and product specifications as constraints, but not the controllability metrics. Results of the comparison of the integrated design and the traditional design methodologies have shown that the design obtained by using the design control methodology leads to a higher profit while fulfilling all the constraints. A key factor in the design of the reactive distillation column is the ratio between the number of trays in the rectification zone and the stripping zone. Therefore, the optimization was run for several values of this ratio. Then the best case for this ratio was used for finally designing the column under the design–control methodology. Furthermore, as defining a ratio between the column length and column diameter is a common practice in the traditional design of distillation columns, in this work, such ratio was also included as a constraint in the optimization problem, to investigate how it impacted the optimal design results. It was observed that such type of constraint is not suitable for being included in the design of the reactive distillation column for the analyzed case study

On Iterative Learning in Multi-agent Systems Coordination and Control

Ph.DDOCTOR OF PHILOSOPH