Abstractions in Reasoning for Long-Term Autonomy

The path to building adaptive, robust, intelligent agents has led researchers to develop a suite of powerful models and algorithms for agents with a single objective. However, in recent years, attempts to use this monolithic approach to solve an ever-expanding set of complex real-world problems, which increasingly include long-term autonomous deployments, have illuminated challenges in its ability to scale. Consequently, a fragmented collection of hierarchical and multi-objective models were developed. This trend continues into the algorithms as well, as each approximates an optimal solution in a different manner for scalability. These models and algorithms represent an attempt to solve pieces of an overarching problem: how can an agent explicitly model and integrate the necessary aspects of reasoning required to achieve long-term autonomy? This thesis presents a general hierarchical and multi-objective model called a policy network that unifies prior fragmented solutions into a single graphical decision-making structure. Policy networks are broadly useful to solve numerous real-world problems. This thesis focuses on autonomous vehicle (AV) problems: (1) route-planning with multiple objectives; (2) semi-autonomy with proactive transfer of control; and (3) intersection decision-making for reasoning online about any number of other vehicles and pedestrians. Formal models are presented for each of the distinct problems. Solutions are evaluated using real-world map data in simulation and demonstrated on a fully operational AV prototype driving on real public roads. Policy networks serve as a shared underlying framework for all three, enabling their seamless integration as parts of an overall solution for rich, real-world, scalable decision-making in agents with long-term autonomy

Compressing POMDPs Using Locality Preserving Non-Negative Matrix Factorization

Partially Observable Markov Decision Processes (POMDPs) are a well-established and rigorous framework for sequential decision-making under uncertainty. POMDPs are well-known to be intractable to solve exactly, and there has been significant work on finding tractable approximation methods. One well-studied approach is to find a compression of the original POMDP by projecting the belief states to a lower-dimensional space. We present a novel dimensionality reduction method for POMDPs based on locality preserving non-negative matrix factorization. Unlike previous approaches, such as Krylov compression and regular non-negative matrix factorization, our approach preserves the local geometry of the belief space manifold. We present results on standard benchmark POMDPs showing improved performance over previously explored compression algorithms for POMDPs

Lernen von robotischer Wahrnehmung durch Vorwissen

Intelligent robots must be able to learn; they must be able to adapt their behavior based on experience. But generalization from past experience is only possible based on assumptions or prior knowledge (priors for short) about how the world works. I study the role of these priors for learning perception. Although priors play a central role in machine learning, they are often hidden in the details of learning algorithms. By making these priors explicit, we can see that currently used priors describe the world from the perspective of a passive disinterested observer. Such generic AI priors are useful because they apply to perception scenarios where there is no robot, such as image classification. These priors are still useful for learning robotic perception, but they miss an important aspect of the problem: the robot. Robots are neither disinterested nor passive. They are trying to achieve tasks by interacting with the world around them, which adds structure to the problem and affords new kinds of priors, which I call robotic priors. The questions are: What are the right robotic priors and how can they be used to enable learning? I investigate these questions in three different perception problems based on raw visual input: 1. learning object segmentation for picking up objects in the Amazon picking challenge, 2. learning state estimation for localization and tracking, and 3. unsupervised learning of state representations that facilitate reinforcement learning. To solve these problems, I propose three sources of prior knowledge---1. the robot's task, 2. robotic algorithms, and 3. physics---and develop ways to encode these priors for the corresponding learning problems. Some of these priors are best encoded as hard constraints that restrict the space of hypotheses considered during learning. Other priors are more suitable to be encoded as preferences for certain hypotheses in the form of learning objectives. My experiments across these problems consistently show that robotic-specific prior knowledge leads to more efficient learning and improved generalization. Based on these results, I propose to take a prior-centric perspective on machine learning, from which follows that we need robotics-specific machine learning methods that incorporate appropriate priors.Intelligente Roboter müssen in der Lage sein zu lernen, um ihr Verhalten auf Basis von Erfahrung anzupassen. Um aus spezifischen Erfahrungen allgemeine Schlüsse zu ziehen, bedarf es jedoch Annahmen oder Vorwissen über die Welt. Ich untersuche die Bedeutung dieses Vorwissens für das Lernen von Wahrnehmung. Obwohl Vorwissen eine zentrale Rolle im maschinellen Lernen spielt, ist es oft in den Details der Lernalgorithmen verborgen. Wenn wir dieses Vorwissen explizit machen, wird deutlich, dass aktuell benutztes Vorwissen die Welt aus der Sicht eines passiven ziellos Beobachters beschreibt. Solche allgemeinen KI-Annahmen sind hilfreich, weil sie auf Wahrnehmungsprobleme wie Bildklassifizierung anwendbar sind, bei denen es keinen Roboter gibt. Solche Annahmen sind auch für das Lernen robotischer Wahrnehmung hilfreich, aber sie übersehen einen wichtigen Aspekt des Problems: den Roboter. Roboter sind weder ziellos noch passiv. Sie versuchen bestimmte Aufgaben zu lösen, indem sie mit der Welt interagieren. Dadurch ergibt sich zusätzliche Problemstruktur, die in anderen Arten von Vorwissen genutzten werden kann. Es stellen sich daher die Fragen was die richtigen Robotik-Annahmen sind und wie diese genutzt werden können, um Lernen zu ermöglichen. Ich beschäftige mich mit diesen Fragen in drei unterschiedlichen Wahrnehmungsproblemen auf Basis von visuellen Eingaben: 1. Lernen von Objektsegmentierung die es ermöglicht bestimmte Objekte in der Amazon Picking Challenge zu greifen, 2. Lernen von Zustandsschätzung für Lokalisierung und Nachführung und 3. unüberwachtes Lernen von Zustandsrepräsentationen, die bestärkendes Lernen ermöglichen. Um diese Probleme zu lösen, schlage ich drei Quellen für Vorwissen vor -- 1. die Aufgabe des Roboters, 2. Algorithmen aus der Robotik, 3. physikalische Gesetze -- und entwickle Möglichkeiten Annahmen aus diesen Quellen in den entsprechenden Lernproblemen zu nutzen. Manche dieser Annahmen lassen sich am besten als harte Bedingungen kodieren, die den Raum der möglichen Hypothesen einschränken die beim Lernen in Betracht gezogen werden. Andere Annahmen sind besser dazu geeignet mit ihnen konsistente Hypothesen zu bevorzugen, indem diese Annahmen als Lernzielen implementiert werden. Meine Experimente in den drei untersuchten Problemen stimmen darin überein, dass robotikspezifische Annahmen Lernen effizienter machen und Generalisierung verbessern. Aufgrund dieser Ergebnisse argumentiere ich für eine Sicht auf maschinelles Lernen, die Vorwissen ins Zentrum der Untersuchung stellt. Aus dieser Sicht folgt, dass wir robotikspezifische Lernmethoden mit entsprechenden Annahmen benötigen.DFG, 329426068, Maschinelles Lernen für Probleme in der Roboti

Lifted Bayesian filtering in multi-entity systems

This thesis focuses on Bayesian filtering for systems that consist of multiple, interacting entites (e.g. agents or objects), which can naturally be described by Multiset Rewriting Systems (MRSs). The main insight is that the state space that is underling an MRS exhibits a certain symmetry, which can be exploited to increase inference efficiency. We provide an efficient, lifted filtering algorithm, which is able to achieve a factorial reduction in space and time complexity, compared to conventional, ground filtering.Diese Arbeit betrachtet Bayes'sche Filter in Systemen, die aus mehreren, interagierenden Entitäten (z.B. Agenten oder Objekten) bestehen. Die Systemdynamik solcher Systeme kann auf natürliche Art durch Multiset Rewriting Systems (MRS) spezifiziert werden. Die wesentliche Erkenntnis ist, dass der Zustandraum Symmetrien aufweist, die ausgenutzt werden können, um die Effizienz der Inferenz zu erhöhen. Wir führen einen effizienten, gelifteten Filter-Algorithmus ein, dessen Zeit- und Platzkomplexität gegenüber dem grundierten Algorithmus um einen faktoriellen Faktor reduziert ist

Safety and Reliability - Safe Societies in a Changing World

The contributions cover a wide range of methodologies and application areas for safety and reliability that contribute to safe societies in a changing world. These methodologies and applications include: - foundations of risk and reliability assessment and management - mathematical methods in reliability and safety - risk assessment - risk management - system reliability - uncertainty analysis - digitalization and big data - prognostics and system health management - occupational safety - accident and incident modeling - maintenance modeling and applications - simulation for safety and reliability analysis - dynamic risk and barrier management - organizational factors and safety culture - human factors and human reliability - resilience engineering - structural reliability - natural hazards - security - economic analysis in risk managemen