Autonomous exploration of hierarchical scene graphs

L'exploració robòtica autònoma és un camp de recerca actiu, on els mètodes de percepció robòtica hi abunden. Els mètodes basats en grafs, en particular, són una manera de representar l'entorn de forma eficient, i ofereixen una base sobre la que raonar a alt nivell per resoldre tasques de l'àmbit de la robòtica. Proposem un sistema per generar grafs jeràrquics d'escena automàticament a partir d'entorns foto-realistes. En aquest treball emprem un mètode de percepció basat en grafs, Hydra, en combinació amb un simulador 3D anomenat Habitat-Sim, per explorar i generar representacions en forma de grafs d'escena 3D dels entorns tridimensionals simulats. Aquest sistema i les dades que n'han derivat ens donen una base sobre la que establim un mètode general per resoldre tasques d'exploració en entorns tridimensionals mitjançant Xarxes Neuronals per a Grafs i Aprenentatge per Reforç.La exploración robótica autónoma es un campo de investigación activo, donde los métodos de percepción robótica abundan. Los métodos basados en grafos, en particular, son una forma de representar el entorno de forma eficiente, y ofrecen una base sobre la que razonar a alto nivel para resolver tareas del ámbito de la robótica. Proponemos un sistema para generar grafos jerárquicos de escena automáticamente a partir de entornos fotorealistas. En este trabajo usamos un método de percepción basado en grafos, Hydra, en combinación con un simulador 3D llamado Habitat-Sim, para explorar y generar representaciones en forma de grafos de escena 3D de los entornos tridimensionales simulados. Este sistema y los datos que han derivado de él nos dan una base sobre la que establecemos un método general para resolver tareas de exploración en entornos tridimensionales mediante Redes Neuronales para Grafos y Aprendizaje por Refuerzo.Robotic autonomous exploration is an active field of research, where robot perception pipelines abound. Graph-based pipelines, in particular, are a way to represent the environment efficiently, and provide grounds for reasoning on a high level to solve robotics tasks. We propose a framework to generate hierarchical scene graphs automatically from photo-realistic environments. In this thesis, a graph perception pipeline, Hydra, is employed in combination with Habitat-Sim, a 3D simulator, to explore and generate 3D scene graph representations from the simulated 3D maps. This framework and data have provided the grounds to establish a general pipeline for solving exploration tasks in 3D environments using Graph Neural Networks and Reinforcement Learning.Outgoin

A Graph Rewriting Visual Language for Database Programming

Textual database programming languages are computationally complete, but have the disadvantage of giving the user a non-intuitive view of the database information that is being manipulated. Visual languages developed in recent years have allowed naive users access to a direct representation of data, often in a graph form, but have concentrated on user interface rather than complex programming tasks. There is a need for a system which combines the advantages of both these programming methods. We describe an implementation of Spider, an experimental visual database programming language aimed at programmers. It uses a graph rewriting paradigm as a basis for a fully visual, computationally complete language. The graphs it rewrites represent the schema and instances of a database. The unique graph rewriting method used by Spider has syntactic and semantic simplicity. Its form of algorithmic expression allows complex computation to be easily represented in short programs. Furthermore, Spider has greater power than normally provided in textual systems, and we show that queries on the schema and associative queries can be performed easily and without requiring any additions to the language

Massively parallel reasoning in transitive relationship hierarchies

This research focuses on building a parallel knowledge representation and reasoning system for the purpose of making progress in realizing human-like intelligence. To achieve human-like intelligence, it is necessary to model human reasoning processes by programs. Knowledge in the real world is huge in size, complex in structure, and is also constantly changing even in limited domains. Unfortunately, reasoning algorithms are very often intractable, which means that they are too slow for any practical applications. One technique to deal with this problem is to design special-purpose reasoners. Many past Al systems have worked rather nicely for limited problem sizes, but attempts to extend them to realistic subsets of world knowledge have led to difficulties. Even special purpose reasoners are not immune to this impasse. In this work, to overcome this problem, we are combining special purpose reasoners with massive We have developed and implemented a massively parallel transitive closure reasoner, called Hydra, that can dynamically assimilate any transitive, binary relation and efficiently answer queries using the transitive closure of all those relations. Within certain limitations, we achieve constant-time responses for transitive closure queries. Hydra can dynamically insert new concepts or new links into a. knowledge base for realistic problem sizes. To get near human-like reasoning capabilities requires the possibility of dynamic updates of the transitive relation hierarchies. Our incremental, massively parallel, update algorithms can achieve almost constant time updates of large knowledge bases. Hydra expands the boundaries of Knowledge Representation and Reasoning in a number of different directions: (1) Hydra improves the representational power of current systems. We have developed a set-based representation for class hierarchies that makes it easy to represent class hierarchies on arrays of processors. Furthermore, we have developed and implemented two methods for mapping this set-based representation onto the processor space of a Connection Machine. These two representations, the Grid Representation and the Double Strand Representation successively improve transitive closure reasoning in terms of speed and processor utilization. (2) Hydra allows fast rerieval and dynamic update of a large knowledge base. New fast update algorithms are formulated to dynamically insert new concepts or new relations into a knowledge base of thousands of nodes. (3) Hydra provides reasoning based on mixed hierarchical representations. We have designed representational tools and massively parallel reasoning algorithms to model reasoning in combined IS-A, Part-of, and Contained-in hierarchies. (4) Hydra\u27s reasoning facilities have been successfully applied to the Medical Entities Dictionary, a large medical vocabulary of Columbia Presbyterian Medical Center. As a result of (1) - (3), Hydra is more general than many current special-purpose reasoners, faster than currently existing general-purpose reasoners, and its knowledge base can be updated dynamically

Collaborative Dynamic 3D Scene Graphs for Automated Driving

Maps have played an indispensable role in enabling safe and automated driving. Although there have been many advances on different fronts ranging from SLAM to semantics, building an actionable hierarchical semantic representation of urban dynamic scenes from multiple agents is still a challenging problem. In this work, we present Collaborative URBan Scene Graphs (CURB-SG) that enable higher-order reasoning and efficient querying for many functions of automated driving. CURB-SG leverages panoptic LiDAR data from multiple agents to build large-scale maps using an effective graph-based collaborative SLAM approach that detects inter-agent loop closures. To semantically decompose the obtained 3D map, we build a lane graph from the paths of ego agents and their panoptic observations of other vehicles. Based on the connectivity of the lane graph, we segregate the environment into intersecting and non-intersecting road areas. Subsequently, we construct a multi-layered scene graph that includes lane information, the position of static landmarks and their assignment to certain map sections, other vehicles observed by the ego agents, and the pose graph from SLAM including 3D panoptic point clouds. We extensively evaluate CURB-SG in urban scenarios using a photorealistic simulator. We release our code at http://curb.cs.uni-freiburg.de.Comment: Refined manuscript and extended supplementar

Voltage stability analysis of a power system network comprising a nuclear power plant

As recently as 2016, the performance of South Africa’s power utility has shown that it is not resilient enough to withstand the consequences of a power system blackout. Blackouts are defined as being a form of power system instability that can be brought about by a variety of abnormal network scenarios. The most common modes of failure are grouped under the term power system stability. In this dissertation, the different modes of power stability that can affect a nuclear power station will be investigated and discussed. The particular phenomenon that will be focused on, however, is the effect that voltage instability has on the ability of generators and loads to perform their standard functions, thus ensuring a secure power system. To investigate the effect that voltage instability has on a nuclear power station, this dissertation will look at relevant literature on the topic. In addition, by extracting from common examples of national and international occurrences of voltage stability, this dissertation will record the effects that this phenomenon has on the security of a power system, in particular on nuclear power plants. To model the network containing a nuclear power plant for the evaluation of voltage stability, the different mathematical models of the generation plant are presented, which include: the automatic voltage regulator, power system stabilizer, governor, nuclear reactor, and excitation system. Also presented are mathematical models of network equipment such as under voltage tap changers and the dynamic loads that are of interest when evaluating voltage stability. The models used for evaluation of the voltage stability phenomenon affecting a nuclear power plant and the surrounding integrated power system are built in the Digsilent PowerFactory® software. The scenario for evaluation is based on a voltage stability event that occurred around at the Koeberg nuclear power system situated in the Western Cape province on South Africa on 15 October 2003. It is commonly accepted that voltage stability can be evaluated at a steady state level by performing power versus voltage (PV) analysis to determine the voltage buses vulnerable to voltage collapse, and reactive power versus voltage (QV) analysis to determine the critical reactive devices required to avert a voltage instability event. The scenarios that are evaluated for voltage stability are divided into two sections: i) a PV and QV analysis as per the event that occurred on 15 October 2003 and ii) present-day voltage stability indices for PV and QV if mixed with a generation such as renewable energy sources that include wind, solar, biomass and concentrated solar power (CSPs). The result reveals the vulnerabilities of the nuclear power plant and the surrounding integrated power system due to a voltage instability event. Some of the solutions proposed include a review of the typical power system protection schemes — such as under and overvoltage detection scheme — that are used. In the study, PV and QV curves provide v good indications of the state of critical busbars and the reactive power reserve margins available before instability can potentially settle in. Simulations confirmed the effectiveness of critical equipment installed in the Western Grid and the effect on their electrical parameters such as torque and the slip on motors

S-Graphs+: Real-time Localization and Mapping leveraging Hierarchical Representations

In this paper, we present an evolved version of the Situational Graphs, which jointly models in a single optimizable factor graph, a SLAM graph, as a set of robot keyframes, containing its associated measurements and robot poses, and a 3D scene graph, as a high-level representation of the environment that encodes its different geometric elements with semantic attributes and the relational information between those elements. Our proposed S-Graphs+ is a novel four-layered factor graph that includes: (1) a keyframes layer with robot pose estimates, (2) a walls layer representing wall surfaces, (3) a rooms layer encompassing sets of wall planes, and (4) a floors layer gathering the rooms within a given floor level. The above graph is optimized in real-time to obtain a robust and accurate estimate of the robot's pose and its map, simultaneously constructing and leveraging the high-level information of the environment. To extract such high-level information, we present novel room and floor segmentation algorithms utilizing the mapped wall planes and free-space clusters. We tested S-Graphs+ on multiple datasets including, simulations of distinct indoor environments, on real datasets captured over several construction sites and office environments, and on a real public dataset of indoor office environments. S-Graphs+ outperforms relevant baselines in the majority of the datasets while extending the robot situational awareness by a four-layered scene model. Moreover, we make the algorithm available as a docker file.Comment: 8 Pages, 7 Figures, 3 Table