Towards generalization of semi-supervised place classification over generalized Voronoi graph

With the progress of human-robot interaction (HRI), the ability of a robot to perform high-level tasks in complex environments is fast becoming an essential requirement. To this end, it is desirable for a robot to understand the environment at both geometric and semantic levels. Therefore in recent years, research towards place classification has been gaining in popularity. After the era of heuristic and rule-based approaches, supervised learning algorithms have been extensively used for this purpose, showing satisfactory performance levels. However, most of those approaches have only been trained and tested in the same environments and thus impede a generalized solution. In this paper, we have proposed a semi-supervised place classification over a generalized Voronoi graph (SPCoGVG) which is a semi-supervised learning framework comprised of three techniques: support vector machine (SVM), conditional random field (CRF) and generalized Voronoi graph (GVG), in order to improve the generalizability. The inherent problem of training CRF with partially labeled data has been solved using a novel parameter estimation algorithm. The effectiveness of the proposed algorithm is validated through extensive analysis of data collected in international university environments. © 2013 Elsevier B.V. All rights reserved

Space Subdivision of Indoor Mobile Laser Scanning Data Based on the Scanner Trajectory

State-of-the-art indoor mobile laser scanners are now lightweight and portable enough to be carried by humans. They allow the user to map challenging environments such as multi-story buildings and staircases while continuously walking through the building. The trajectory of the laser scanner is usually discarded in the analysis, although it gives insight about indoor spaces and the topological relations between them. In this research, the trajectory is used in conjunction with the point cloud to subdivide the indoor space into stories, staircases, doorways, and rooms. Analyzing the scanner trajectory as a standalone dataset is used to identify the staircases and to separate the stories. Also, the doors that are traversed by the operator during the scanning are identified by processing only the interesting spots of the point cloud with the help of the trajectory. Semantic information like different space labels is assigned to the trajectory based on the detected doors. Finally, the point cloud is semantically enriched by transferring the labels from the annotated trajectory to the full point cloud. Four real-world datasets with a total of seven stories are used to evaluate the proposed methods. The evaluation items are the total number of correctly detected rooms, doors, and staircases

Semantic Interpretation of Mobile Laser Scanner Point Clouds in Indoor Scenes Using Trajectories

The data acquisition with Indoor Mobile Laser Scanners (IMLS) is quick, low-cost and accurate for indoor 3D modeling. Besides a point cloud, an IMLS also provides the trajectory of the mobile scanner. We analyze this trajectory jointly with the point cloud to support the labeling of noisy, highly reflected and cluttered points in indoor scenes. An adjacency-graph-based method is presented for detecting and labeling of permanent structures, such as walls, floors, ceilings, and stairs. Through occlusion reasoning and the use of the trajectory as a set of scanner positions, gaps are discriminated from real openings in the data. Furthermore, a voxel-based method is applied for labeling of navigable space and separating them from obstacles. The results show that 80% of the doors and 85% of the rooms are correctly detected, and most of the walls and openings are reconstructed. The experimental outcomes indicate that the trajectory of MLS systems plays an essential role in the understanding of indoor scene

Hardware dedicado para sistemas empotrados de visión

La constante evolución de las Tecnologías de la Información y las Comunicaciones no solo ha permitido que más de la mitad de la población mundial esté actualmente interconectada a través de Internet, sino que ha sido el caldo de cultivo en el que han surgido nuevos paradigmas, como el ‘Internet de las cosas’ (IoT) o la ‘Inteligencia ambiental’ (AmI), que plantean la necesidad de interconectar objetos con distintas funcionalidades para lograr un entorno digital, sensible y adaptativo, que proporcione servicios de muy distinta índole a sus usuarios. La consecución de este entorno requiere el desarrollo de dispositivos electrónicos de bajo coste que, con tamaño y peso reducido, sean capaces de interactuar con el medio que los rodea, operar con máxima autonomía y proporcionar un elevado nivel de inteligencia. La funcionalidad de muchos de estos dispositivos incluirá la capacidad para adquirir, procesar y transmitir imágenes, extrayendo, interpretando o modificando la información visual que resulte de interés para una determinada aplicación. En el marco de este desafío surge la presente Tesis Doctoral, cuyo eje central es el desarrollo de hardware dedicado para la implementación de algoritmos de procesamiento de imágenes y secuencias de vídeo usados en sistemas empotrados de visión. El trabajo persigue una doble finalidad. Por una parte, la búsqueda de soluciones que, por sus prestaciones y rendimiento, puedan ser incorporadas en sistemas que satisfagan las estrictas exigencias de funcionalidad, tamaño, consumo de energía y velocidad de operación demandadas por las nuevas aplicaciones. Por otra, el diseño de una serie de bloques funcionales implementados como módulos de propiedad intelectual, que permitan aliviar la carga computacional de las unidades de procesado de los sistemas en los que se integren. En la Tesis se proponen soluciones específicas para la implementación de dos tipos de operaciones habitualmente presentes en muchos sistemas de visión artificial: la sustracción de fondo y el etiquetado de componentes conexos. Las distintas alternativas surgen como consecuencia de aplicar una adecuada relación de compromiso entre funcionalidad y coste, entendiendo este último criterio en términos de recursos de cómputo, velocidad de operación y potencia consumida, lo que permite cubrir un amplio espectro de aplicaciones. En algunas de las soluciones propuestas se han utilizado además, técnicas de inferencia basadas en Lógica Difusa con idea de mejorar la calidad de los sistemas de visión resultantes. Para la realización de los diferentes bloques funcionales se ha seguido una metodología de diseño basada en modelos, que ha permitido la realización de todo el ciclo de desarrollo en un único entorno de trabajo. Dicho entorno combina herramientas informáticas que facilitan las etapas de codificación algorítmica, diseño de circuitos, implementación física y verificación funcional y temporal de las distintas alternativas, acelerando con ello todas las fases del flujo de diseño y posibilitando una exploración más eficiente del espacio de posibles soluciones. Asimismo, con el objetivo de demostrar la funcionalidad de las distintas aportaciones de esta Tesis Doctoral, algunas de las soluciones propuestas han sido integradas en sistemas de vídeo reales, que emplean buses estándares de uso común. Los dispositivos seleccionados para llevar a cabo estos demostradores han sido FPGAs y SoPCs de Xilinx, ya que sus excelentes propiedades para el prototipado y la construcción de sistemas que combinan componentes software y hardware, los convierten en candidatos ideales para dar soporte a la implementación de este tipo de sistemas.The continuous evolution of the Information and Communication Technologies (ICT), not only has allowed more than half of the global population to be currently interconnected through Internet, but it has also been the breeding ground for new paradigms such as Internet of Things (ioT) or Ambient Intelligence (AmI). These paradigms expose the need of interconnecting elements with different functionalities in order to achieve a digital, sensitive, adaptive and responsive environment that provides services of distinct nature to the users. The development of low cost devices, with small size, light weight and a high level of autonomy, processing power and ability for interaction is required to obtain this environment. Attending to this last feature, many of these devices will include the capacity to acquire, process and transmit images, extracting, interpreting and modifying the visual information that could be of interest for a certain application. This PhD Thesis, focused on the development of dedicated hardware for the implementation of image and video processing algorithms used in embedded systems, attempts to response to this challenge. The work has a two-fold purpose: on one hand, the search of solutions that, for its performance and properties, could be integrated on systems with strict requirements of functionality, size, power consumption and speed of operation; on the other hand, the design of a set of blocks that, packaged and implemented as IP-modules, allow to alleviate the computational load of the processing units of the systems where they could be integrated. In this Thesis, specific solutions for the implementation of two kinds of usual operations in many computer vision systems are provided. These operations are background subtraction and connected component labelling. Different solutions are created as the result of applying a good performance/cost trade-off (approaching this last criteria in terms of area, speed and consumed power), able to cover a wide range of applications. Inference techniques based on Fuzzy Logic have been applied to some of the proposed solutions in order to improve the quality of the resulting systems. To obtain the mentioned solutions, a model based-design methodology has been applied. This fact has allowed us to carry out all the design flow from a single work environment. That environment combines CAD tools that facilitate the stages of code programming, circuit design, physical implementation and functional and temporal verification of the different algorithms, thus accelerating the overall processes and making it possible to explore the space of solutions. Moreover, aiming to demonstrate the functionality of this PhD Thesis’s contributions, some of the proposed solutions have been integrated on real video systems that employ common and standard buses. The devices selected to perform these demonstrators have been FPGA and SoPCs (manufactured by Xilinx) since, due to their excellent properties for prototyping and creating systems that combine software and hardware components, they are ideal to develop these applications

Multimodal Shared-Control Interaction for Mobile Robots in AAL Environments

This dissertation investigates the design, development and implementation of cognitively adequate, safe and robust, spatially-related, multimodal interaction between human operators and mobile robots in Ambient Assisted Living environments both from the theoretical and practical perspectives. By focusing on different aspects of the concept Interaction, the essential contribution of this dissertation is divided into three main research packages; namely, Formal Interaction, Spatial Interaction and Multimodal Interaction in AAL. As the principle package, in Formal Interaction, research effort is dedicated to developing a formal language based interaction modelling and management solution process and a unified dialogue modelling approach. This package aims to enable a robust, flexible, and context-sensitive, yet formally controllable and tractable interaction. This type of interaction can be used to support the interaction management of any complex interactive systems, including the ones covered in the other two research packages. In the second research package, Spatial Interaction, a general qualitative spatial knowledge based multi-level conceptual model is developed and proposed. The goal is to support a spatially-related interaction in human-robot collaborative navigation. With a model-based computational framework, the proposed conceptual model has been implemented and integrated into a practical interactive system which has been evaluated by empirical studies. It has been particularly tested with respect to a set of high-level and model-based conceptual strategies for resolving the frequent spatially-related communication problems in human-robot interaction. Last but not least, in Multimodal Interaction in AAL, attention is drawn to design, development and implementation of multimodal interaction for elderly persons. In this elderly-friendly scenario, ageing-related characteristics are carefully considered for an effective and efficient interaction. Moreover, a standard model based empirical framework for evaluating multimodal interaction is provided. This framework was especially applied to evaluate a minutely developed and systematically improved elderly-friendly multimodal interactive system through a series of empirical studies with groups of elderly persons

Exploring the role of system operation modes in failure analysis in the context of first generation cyber-physical systems

Typically, emerging system failures have a strong impact on the performance of industrial systems as well as on the efficiency of their operational and servicing processes. Being aware of these, maintenance and repair researchers have developed multiple failure detection and diagnosis techniques that allow early recognition of system or component failures and maintaining continuous system operation in a cost-effective way. However, these techniques have many deficiencies in the case of self-tuning first generation cyber-physical systems (1G-CPSs). The reason is that these systems compensate for the effects of emerging system failures until their resources are exhausted, and the compensatory actions not only mask the failures, but also make their recognition difficult. Late recognition of failures is however in contrast with the principles of preventive maintenance. Therefore, the promotion research concentrated on the issue of recognizing and forecasting failures under dynamic and adaptive behavior of 1G-CPSs. CPSs are enabled to compensate for failure symptoms by changing their system operation modes (SOMs). It was also observed that transitions of SOMs reduce the reliability of a signal-based failure diagnosis. It was hypothesized that the frequency and the duration of the changes of the operational states of the 1G-CPS may be strong indicators of the failure emergence phenomenon and that investigation of SOMs facilitates early detection of failures. Therefore, the completed exploratory studies were aimed at exploring how the frequency and duration of transitions of SOMs can be brought into correlation with specific types of failures, and how they can be computed as measures of failure occurrence. The obtained results revealed that system failures tend to induce unusual system operation modes that can be used as basis for failure characterization, and even for failure forecasting. The empirical research made use of a cyber-physical greenhouse testbed to get experimental data and was completed by the development of computational model. A failure injection strategy was implemented in order to induce failure occurrence in a controlled manner. The proposed approach can be applied as a basis of forecasting system failures of 1G-CPSs, but additional research seems to be necessary

Semantic labeling of places with mobile robots

During the last years there has been an increasing interest in the area of service robots. Under this category we find robots working in tasks such as elderly care, guiding, office and domestic assistance, inspection, and many more. Service robots usually work in indoor environments designed for humans, with offices and houses being some of the most typical examples. These environments are typically divided into places with different functionalities like corridors, rooms or doorways. The ability to learn such semantic categories from sensor data enables a mobile robot to extend its representation of the environment, and to improve its capabilities. As an example, natural language terms like corridor or room can be used to indicate the position of the robot in a more intuitive way when communicating with humans. This book presents several approaches to enable a mobile robot to categorize places in indoor environments. The categories are indicated by terms which represent the different regions in these environments. The objective of this work is to enable mobile robots to perceive the spatial divisions in indoor environments in a similar way as people do. This is an interesting step forward to the problem of moving the perception of robots closer to the perception of humans. Many approaches introduced in this book come from the area of pattern recognition and classification. The applied methods have been adapted to solve the specific problem of place recognition. In this regard, this work is a useful reference to students and researchers who want to introduce classification techniques to help solve similar problems in mobile robotics