Learning Task Specifications from Demonstrations

Real world applications often naturally decompose into several sub-tasks. In many settings (e.g., robotics) demonstrations provide a natural way to specify the sub-tasks. However, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the sub-tasks can be safely recombined or limit the types of composition available. Motivated by this deficit, we consider the problem of inferring Boolean non-Markovian rewards (also known as logical trace properties or specifications) from demonstrations provided by an agent operating in an uncertain, stochastic environment. Crucially, specifications admit well-defined composition rules that are typically easy to interpret. In this paper, we formulate the specification inference task as a maximum a posteriori (MAP) probability inference problem, apply the principle of maximum entropy to derive an analytic demonstration likelihood model and give an efficient approach to search for the most likely specification in a large candidate pool of specifications. In our experiments, we demonstrate how learning specifications can help avoid common problems that often arise due to ad-hoc reward composition.Comment: NIPS 201

Autonomiset työkoneet ja autonomian vaikutus koneturvallisuuteen

Autonomous machines and vehicles are an increasing part of everyday life and industrial operations. These machines and vehicles have enjoyed rapid technological advancements in recent years, which has led to increasingly sophisticated functions and functionalities. The advancements in autonomous technologies have, however, given rise to questions and concerns relating to the safety of these machines and vehicles, and on how an adequate level of safety can be ensured when no dedicated operator or driver is present. This thesis looks at the main areas that affect the overall safety of autonomous industrial machines and civilian road vehicles, and presents the most prominent challenges faced in ensuring the safety of autonomous applications. The goal of the thesis is to give the reader an overview of the safety-related aspects of autonomy and to show what has to be considered when ensuring an adequate level of safety for autonomous machines or vehicles. This is achieved by an extensive literature review on autonomous applications in both industrial and automotive fields, and on the safety-related aspects of autonomy. Additionally, mining is used in the thesis as an example of autonomous machines in practice and on the challenges autonomy can face in industrial operations. Based on the research carried out, it can be said that the overall safety of machine autonomy is currently hindered by two main aspects: the lack of applicable standards, legislation and guidelines regarding the autonomy of machines and vehicles, and the paradox that arises from balancing the desired level of autonomy with the needed level of safety. This has led to a situation where, in theory, highly complex and sophisticated autonomous machines are possible from a technical standpoint, but they lack a common and thorough method for ensuring an adequate level of safety