Discovering Blind Spots in Reinforcement Learning

Agents trained in simulation may make errors in the real world due to mismatches between training and execution environments. These mistakes can be dangerous and difficult to discover because the agent cannot predict them a priori. We propose using oracle feedback to learn a predictive model of these blind spots to reduce costly errors in real-world applications. We focus on blind spots in reinforcement learning (RL) that occur due to incomplete state representation: The agent does not have the appropriate features to represent the true state of the world and thus cannot distinguish among numerous states. We formalize the problem of discovering blind spots in RL as a noisy supervised learning problem with class imbalance. We learn models to predict blind spots in unseen regions of the state space by combining techniques for label aggregation, calibration, and supervised learning. The models take into consideration noise emerging from different forms of oracle feedback, including demonstrations and corrections. We evaluate our approach on two domains and show that it achieves higher predictive performance than baseline methods, and that the learned model can be used to selectively query an oracle at execution time to prevent errors. We also empirically analyze the biases of various feedback types and how they influence the discovery of blind spots.Comment: To appear at AAMAS 201

The systematic allocation of teacher to task in education systems: a thesis presented in partial fulfilment of the requirements for the degree of Master of Arts (Education) at Massey University

This study examines the issues associated with the systematic allocation of instructional personnel resources to tasks within education systems and develops a conceptual framework to organize knowledge and direct research in this area of enquiry. An evaluation of the current status of the field of enquiry concludes that existing mechanisms for the matching of teacher to task on a system-wide basis are manifestly inefficient, educationalists have paid scant attention to the problem and there is no recognizable body of knowledge relating to it. The basic elements of a rational system, the reward structure, selection criteria, and selection procedures are examined and a rational system, based on a consideration of the issues raised in that examination, is outlined. An approach to the analysis and comparison of systems is discussed and some suggestions on the modification of staffing arrangements to bring them more into accord with the 'realities' of the situation are made

Learning With An Insufficient Supply Of Data Via Knowledge Transfer And Sharing

As machine learning methods extend to more complex and diverse set of problems, situations arise where the complexity and availability of data presents a situation where the information source is not adequate to generate a representative hypothesis. Learning from multiple sources of data is a promising research direction as researchers leverage ever more diverse sources of information. Since data is not readily available, knowledge has to be transferred from other sources and new methods (both supervised and un-supervised) have to be developed to selectively share and transfer knowledge. In this dissertation, we present both supervised and un-supervised techniques to tackle a problem where learning algorithms cannot generalize and require an extension to leverage knowledge from different sources of data. Knowledge transfer is a difficult problem as diverse sources of data can overwhelm each individual dataset\u27s distribution and a careful set of transformations has to be applied to increase the relevant knowledge at the risk of biasing a dataset\u27s distribution and inducing negative transfer that can degrade a learner\u27s performance. We give an overview of the issues encountered when the learning dataset does not have a sufficient supply of training examples. We categorize the structure of small datasets and highlight the need for further research. We present an instance-transfer supervised classification algorithm to improve classification performance in a target dataset via knowledge transfer from an auxiliary dataset. The improved classification performance of our algorithm is demonstrated with several real-world experiments. We extend the instance-transfer paradigm to supervised classification with Absolute Rarity\u27 , where a dataset has an insufficient supply of training examples and a skewed class distribution. We demonstrate one solution with a transfer learning approach and another with an imbalanced learning approach and demonstrate the effectiveness of our algorithms with several real world text and demographics classification problems (among others). We present an unsupervised multi-task clustering algorithm where several small datasets are simultaneously clustered and knowledge is transferred between the datasets to improve clustering performance on each individual dataset and we demonstrate the improved clustering performance with an extensive set of experiments

Rugby College of Further Education: report from the Inspectorate (FEFC inspection report; 112/95 and 75/00)

The Further Education Funding Council has a legal duty to make sure further education in England is properly assessed. The FEFC’s inspectorate inspects and reports on each college of further education according to a four-year cycle. This record comprises the reports for periods 1994-95 and 1999-2000