Reinforcement Learning for Racecar Control

This thesis investigates the use of reinforcement learning to learn to drive a racecar in the simulated environment of the Robot Automobile Racing Simulator. Real-life race driving is known to be difficult for humans, and expert human drivers use complex sequences of actions. There are a large number of variables, some of which change stochastically and all of which may affect the outcome. This makes driving a promising domain for testing and developing Machine Learning techniques that have the potential to be robust enough to work in the real world. Therefore the principles of the algorithms from this work may be applicable to a range of problems. The investigation starts by finding a suitable data structure to represent the information learnt. This is tested using supervised learning. Reinforcement learning is added and roughly tuned, and the supervised learning is then removed. A simple tabular representation is found satisfactory, and this avoids difficulties with more complex methods and allows the investigation to concentrate on the essentials of learning. Various reward sources are tested and a combination of three are found to produce the best performance. Exploration of the problem space is investigated. Results show exploration is essential but controlling how much is done is also important. It turns out the learning episodes need to be very long and because of this the task needs to be treated as continuous by using discounting to limit the size of the variables stored. Eligibility traces are used with success to make the learning more efficient. The tabular representation is made more compact by hashing and more accurate by using smaller buckets. This slows the learning but produces better driving. The improvement given by a rough form of generalisation indicates the replacement of the tabular method by a function approximator is warranted. These results show reinforcement learning can work within the Robot Automobile Racing Simulator, and lay the foundations for building a more efficient and competitive agent

Wage v/s Efficiency I. Under normal circumstances

wages, efficiency, labour, steel plant, industrial organization

Wage v/s Efficiencies II. Impact of the emergency period

wages, efficiency, labour, steel plant, industrial organization, emergency

Wage v/s Efficiency I. Under normal circumstances

wages, efficiency, labour, steel plant, industrial organization

Wage v/s Efficiencies II. Impact of the emergency period

wages, efficiency, labour, steel plant, industrial organization, emergency

Hybrid Bayesian Eigenobjects: Combining Linear Subspace and Deep Network Methods for 3D Robot Vision

We introduce Hybrid Bayesian Eigenobjects (HBEOs), a novel representation for 3D objects designed to allow a robot to jointly estimate the pose, class, and full 3D geometry of a novel object observed from a single viewpoint in a single practical framework. By combining both linear subspace methods and deep convolutional prediction, HBEOs efficiently learn nonlinear object representations without directly regressing into high-dimensional space. HBEOs also remove the onerous and generally impractical necessity of input data voxelization prior to inference. We experimentally evaluate the suitability of HBEOs to the challenging task of joint pose, class, and shape inference on novel objects and show that, compared to preceding work, HBEOs offer dramatically improved performance in all three tasks along with several orders of magnitude faster runtime performance.Comment: To appear in the International Conference on Intelligent Robots (IROS) - Madrid, 201

The palaeoceanography of the Leeuwin Current : implications for a future world

Long-term progressive changes of the Leeuwin Current are linked to plate and ocean basin ‘geography’ and Cenozoic global climates and palaeoceanography. Suggestions of the presence of a proto-Leeuwin Current as early as late Middle to Late Eocene times (c. 35–42 Ma) cannot be verified by the fossil record of the western margin of Australia. “Leeuwin Current style” circulation around Australia was certainly established by the early Oligocene, in response to palaeogeographic changes in the Tasman Strait. This, followed by tectonic eorganisation of the Indonesian Archipelago throughout the Miocene, provided a palaeogeographic setting, which by the Pliocene was essentially that of today. The subsequent history of the Leeuwin Current comprises climatically-induced changes operating over orbital and sub-orbital temporal scales. Specifically, the advent of Pleistocene-style climates, especially over the last 800 000 years, and their associated interglacial – glacial states provide the two end-member climate-ocean states that have characterised Leeuwin Current activity during that time. Indications of the nature of these contrasting states is provided by: (i) the Last Interglacial (c. 125 Ka) during which sea level was higher by some +4 m, and with higher sea surface temperatures (SSTs) clearly indicating a more ‘active’ Leeuwin Current; and (ii) the Last Glacial Maximum (21 Ka), during which sea level wassome 130 m lower than today, resulting in massive shelf extensions along the coast of Western Australia, ccompanied by reduced Indonesian Throughflow, lower low latitude SSTs and changes in the Western Pacific Warm Water Pool, and with these changes, possibly reduced Leeuwin Current activity. Sub-orbital scale luctuations in current strength are driven by global climate change associated with El Niño – La Niña events as well as regional climatic changes driven by volcanism. These forcing mechanisms operate at time scales well within the reach of human experience, and provide important comparative data for predicting the response of the Leeuwin Current to climate change predicted for this century. Studies of the impact of changes in the vigour of the Leeuwin Current on shallow marine communities are in their infancy. Coupling climate models with geological analogues provide important research agenda for predicting the trajectory of future changes to the Leeuwin Current and their impacts on the marine biota of coastal Western Australia