Mobile Robot Object Recognition through the Synergy of Probabilistic Graphical Models and Semantic Knowledge

J.R. Ruiz-Sarmiento and C. Galindo and J. Gonzalez-Jimenez, Mobile Robot Object Recognition through the Synergy of Probabilistic Graphical Models and Semantic Knowledge, in European Conf. of Artificial Intelligence, CogRob workshop, 2014.Mobile robots intended to perform high-level tasks have to recognize objects in their workspace. In order to increase the success of the recognition process, recent works have studied the use of contextual information. Probabilistic Graphical Models (PGMs) and Semantic Knowledge (SK) are two well-known approaches for dealing with contextual information, although they exhibit some drawbacks: the PGMs complexity exponentially increases with the number of objects in the scene, while SK are unable to handle uncertainty. In this work we combine both approaches to address the object recognition problem. We propose the exploitation of SK to reduce the complexity of the probabilistic inference, while we rely on PGMs to enhance SK with a mechanism to manage uncertainty. The suitability of our method is validated through a set of experiments, in which a mobile robot endowed with a Kinect-like sensor captured 3D data from 25 real environments, achieving a promising result of ~94% of success.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. This work has been funded by the Spanish grant program FPU-MICINN 2010 and the Spanish project "TAROTH: New developments toward a robot at home"

Improvement of the sensory and autonomous capability of robots through olfaction: the IRO Project

Proyecto de Excelencia Junta de Andalucía TEP2012-530Olfaction is a valuable source of information about the environment that has not been su ciently exploited in mobile robotics yet. Certainly, odor information can contribute to other sensing modalities, e.g. vision, to successfully accomplish high-level robot activities, such as task planning or execution in human environments. This paper describes the developments carried out in the scope of the IRO project, which aims at making progress in this direction by investigating mechanisms that exploit odor information (usually coming in the form of the type of volatile and its concentration) in problems like object recognition and scene-activity understanding. A distinctive aspect of this research is the special attention paid to the role of semantics within the robot perception and decisionmaking processes. The results of the IRO project have improved the robot capabilities in terms of efciency, autonomy and usefulness.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tec

Online Context-based Object Recognition for Mobile Robots

This work proposes a robotic object recognition system that takes advantage of the contextual information latent in human-like environments in an online fashion. To fully leverage context, it is needed perceptual information from (at least) a portion of the scene containing the objects of interest, which could not be entirely covered by just an one-shot sensor observation. Information from a larger portion of the scenario could still be considered by progressively registering observations, but this approach experiences difficulties under some circumstances, e.g. limited and heavily demanded computational resources, dynamic environments, etc. Instead of this, the proposed recognition system relies on an anchoring process for the fast registration and propagation of objects’ features and locations beyond the current sensor frustum. In this way, the system builds a graphbased world model containing the objects in the scenario (both in the current and previously perceived shots), which is exploited by a Probabilistic Graphical Model (PGM) in order to leverage contextual information during recognition. We also propose a novel way to include the outcome of local object recognition methods in the PGM, which results in a decrease in the usually high CRF learning complexity. A demonstration of our proposal has been conducted employing a dataset captured by a mobile robot from restaurant-like settings, showing promising results.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Probabilistic techniques in semantic mapping for mobile robotics

Los mapas semánticos son representaciones del mundo que permiten a un robot entender no sólo los aspectos espaciales de su lugar de trabajo, sino también el significado de sus elementos (objetos, habitaciones, etc.) y como los humanos interactúan con ellos (e.g. funcionalidades, eventos y relaciones). Para conseguirlo, un mapa semántico añade a las representaciones puramente espaciales, tales como mapas geométricos o topológicos, meta-información sobre los tipos de elementos y relaciones que pueden encontrarse en el entorno de trabajo. Esta meta-información, denominada conocimiento semántico o de sentido común, se codifica típicamente en Bases de Conocimiento. Un ejemplo de este tipo de información podría ser: "los frigoríficos son objetos grandes, con forma rectangular, colocados normalmente en las cocinas, y que pueden contener comida perecedera y medicación". Codificar y manejar este conocimiento semántico permite al robot razonar acerca de la información obtenida de un cierto lugar de trabajo, así como inferir nueva información con el fin de ejecutar eficientemente tareas de alto nivel como "¡hola robot! llévale la medicación a la abuela, por favor". La presente tesis propone la utilización de técnicas probabilísticas para construir y mantener mapas semánticos, lo cual presenta tres ventajas principales en comparación con los enfoques tradicionales: i) permite manejar incertidumbre (proveniente de los sensores imprecisos del robot y de los modelos empleados), ii) provee representaciones del entorno coherentes por medio del aprovechamiento de las relaciones contextuales entre los elementos observados (e.g. los frigoríficos usualmente se encuentran en las cocinas) desde un punto de vista holístico, y iii) produce valores de certidumbre que reflejan el grado de exactitud de la comprensión del robot acerca de su entorno. Específicamente, las contribuciones presentadas pueden agruparse en dos temas principales. El primer conjunto de contribuciones se basa en el problema del reconocimiento de objetos y/o habitaciones, ya que los sistemas de mapeo semántico deben contar con algoritmos de reconocimiento fiables para la construcción de representaciones válidas. Para ello se ha explorado la utilización de Modelos Gráficos Probabilísticos (Probabilistic Graphical Models o PGMs en inglés) con el fin de aprovechar las relaciones de contexto entre objetos y/o habitaciones a la vez que se maneja la incertidumbre inherente al problema de reconocimiento, y el empleo de Bases de Conocimiento para mejorar su desempeño de distintos modos, e.g., detectando resultados incoherentes, proveyendo información a priori, reduciendo la complejidad de los algoritmos de inferencia probabilística, generando ejemplos de entrenamiento sintéticos, habilitando el aprendizaje a partir de experiencias pasadas, etc. El segundo grupo de contribuciones acomoda los resultados probabilísticos provenientes de los algoritmos de reconocimiento desarrollados en una nueva representación semántica, denominada Multiversal Semantic Map (MvSmap). Este mapa gestiona múltiples interpretaciones del espacio de trabajo del robot, llamadas universos, los cuales son anotados con la probabilidad de ser los correctos de acuerdo con el conocimiento actual del robot. Así, este enfoque proporciona una creencia fundamentada sobre la exactitud de la comprensión del robot sobre su entorno, lo que le permite operar de una manera más eficiente y coherente. Los algoritmos probabilísticos propuestos han sido testeados concienzudamente y comparados con otros enfoques actuales e innovadores empleando conjuntos de datos del estado del arte. De manera adicional, esta tesis también contribuye con dos conjuntos de datos, UMA-Offices and Robot@Home, los cuales contienen información sensorial capturada en distintos entornos de oficinas y casas, así como dos herramientas software, la librería Undirected Probabilistic Graphical Models in C++ (UPGMpp), y el conjunto de herramientas Object Labeling Toolkit (OLT), para el trabajo con Modelos Gráficos Probabilísticos y el procesamiento de conjuntos de datos respectivamente

Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic

UPGMpp: a Software Library for Contextual Object Recognition

Object recognition is a cornerstone task towards the scene understanding problem. Recent works in the field boost their perfor- mance by incorporating contextual information to the traditional use of the objects’ geometry and/or appearance. These contextual cues are usually modeled through Conditional Random Fields (CRFs), a partic- ular type of undirected Probabilistic Graphical Model (PGM), and are exploited by means of probabilistic inference methods. In this work we present the Undirected Probabilistic Graphical Models in C++ library (UPGMpp), an open source solution for representing, training, and per- forming inference over undirected PGMs in general, and CRFs in par- ticular. The UPGMpp library supposes a reliable and comprehensive workbench for recognition systems exploiting contextual information, in- cluding a variety of inference methods based on local search, graph cuts, and message passing approaches. This paper illustrates the virtues of the library, i.e. it is efficient, comprehensive, versatile, and easy to use, by presenting a use-case applied to the object recognition problem in home scenes from the challenging NYU2 dataset.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. Spanish grant program FPU-MICINN 2010 and the Spanish projects “TAROTH: New developments toward a robot at home” (Ref. DPI2011-25483) and “PROMOVE: Advances in mobile robotics for promoting independent life of elders” (Ref. DPI2014-55826-R

Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age

Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications, and witnessing a steady transition of this technology to industry. We survey the current state of SLAM. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors' take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved