A Robust Localization System for Inspection Robots in Sewer Networks †

Sewers represent a very important infrastructure of cities whose state should be monitored periodically. However, the length of such infrastructure prevents sensor networks from being applicable. In this paper, we present a mobile platform (SIAR) designed to inspect the sewer network. It is capable of sensing gas concentrations and detecting failures in the network such as cracks and holes in the floor and walls or zones were the water is not flowing. These alarms should be precisely geo-localized to allow the operators performing the required correcting measures. To this end, this paper presents a robust localization system for global pose estimation on sewers. It makes use of prior information of the sewer network, including its topology, the different cross sections traversed and the position of some elements such as manholes. The system is based on a Monte Carlo Localization system that fuses wheel and RGB-D odometry for the prediction stage. The update step takes into account the sewer network topology for discarding wrong hypotheses. Additionally, the localization is further refined with novel updating steps proposed in this paper which are activated whenever a discrete element in the sewer network is detected or the relative orientation of the robot over the sewer gallery could be estimated. Each part of the system has been validated with real data obtained from the sewers of Barcelona. The whole system is able to obtain median localization errors in the order of one meter in all cases. Finally, the paper also includes comparisons with state-of-the-art Simultaneous Localization and Mapping (SLAM) systems that demonstrate the convenience of the approach.Unión Europea ECHORD ++ 601116Ministerio de Ciencia, Innovación y Universidades de España RTI2018-100847-B-C2

A RGBD-Based System for Real-Time Robotic Defects Detection on Sewer Networks

In this paper we summarize the automatic defect inspec- tion onboard the sewer inspection ground platform SIAR. We include a general overview of the software and hardware characteristics of our platform, making a special emphasis on the sensing devices and software systems that are used for defect inspection. The main detection algo- rithm makes use of the a priori knowledge of ideal sections of the sewers that can be found in the Geographic Information Systems (GIS), and uses a variant of the Iterative Closest Point (ICP) algorithm for nd- ing structural and serviceability defects. Then, we describe the software modules that are in charge of storing the alerts found by the detection system and of displaying them to the operator. The whole system has been tested in two eld scenarios on di erent locations of the real sewer network of Barcelona, Spain

A robust method for approximate visual robot localization in feature-sparse sewer pipes

Buried sewer pipe networks present many challenges for robot localization systems, which require non-standard solutions due to the unique nature of these environments: they cannot receive signals from global positioning systems (GPS) and can also lack visual features necessary for standard visual odometry algorithms. In this paper, we exploit the fact that pipe joints are equally spaced and develop a robot localization method based on pipe joint detection that operates in one degree-of-freedom along the pipe length. Pipe joints are detected in visual images from an on-board forward facing (electro-optical) camera using a bag-of-keypoints visual categorization algorithm, which is trained offline by unsupervised learning from images of sewer pipe joints. We augment the pipe joint detection algorithm with drift correction using vision-based manhole recognition. We evaluated the approach using real-world data recorded from three sewer pipes (of lengths 30, 50 and 90 m) and benchmarked against a standard method for visual odometry (ORB-SLAM3), which demonstrated that our proposed method operates more robustly and accurately in these feature-sparse pipes: ORB-SLAM3 completely failed on one tested pipe due to a lack of visual features and gave a mean absolute error in localization of approximately 12%–20% on the other pipes (and regularly lost track of features, having to re-initialize multiple times), whilst our method worked successfully on all tested pipes and gave a mean absolute error in localization of approximately 2%–4%. In summary, our results highlight an important trade-off between modern visual odometry algorithms that have potentially high precision and estimate full six degree-of-freedom pose but are potentially fragile in feature sparse pipes, versus simpler, approximate localization methods that operate in one degree-of-freedom along the pipe length that are more robust and can lead to substantial improvements in accuracy

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications

Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark

Perhaps surprisingly sewerage infrastructure is one of the most costly infrastructures in modern society. Sewer pipes are manually inspected to determine whether the pipes are defective. However, this process is limited by the number of qualified inspectors and the time it takes to inspect a pipe. Automatization of this process is therefore of high interest. So far, the success of computer vision approaches for sewer defect classification has been limited when compared to the success in other fields mainly due to the lack of public datasets. To this end, in this work we present a large novel and publicly available multi-label classification dataset for image-based sewer defect classification called Sewer-ML. The Sewer-ML dataset consists of 1.3 million images annotated by professional sewer inspectors from three different utility companies across nine years. Together with the dataset, we also present a benchmark algorithm and a novel metric for assessing performance. The benchmark algorithm is a result of evaluating 12 state-of-the-art algorithms, six from the sewer defect classification domain and six from the multi-label classification domain, and combining the best performing algorithms. The novel metric is a class-importance weighted F2 score, $\text{F}2_{\text{CIW}}$, reflecting the economic impact of each class, used together with the normal pipe F1 score, $\text{F}1_{\text{Normal}}$. The benchmark algorithm achieves an $\text{F}2_{\text{CIW}}$ score of 55.11% and $\text{F}1_{\text{Normal}}$ score of 90.94%, leaving ample room for improvement on the Sewer-ML dataset. The code, models, and dataset are available at the project page https://vap.aau.dk/sewer-ml/Comment: CVPR 2021. Project webpage: https://vap.aau.dk/sewer-ml

Simultaneous localization and mapping for inspection robots in water and sewer pipe networks: a review

At the present time, water and sewer pipe networks are predominantly inspected manually. In the near future, smart cities will perform intelligent autonomous monitoring of buried pipe networks, using teams of small robots. These robots, equipped with all necessary computational facilities and sensors (optical, acoustic, inertial, thermal, pressure and others) will be able to inspect pipes whilst navigating, selflocalising and communicating information about the pipe condition and faults such as leaks or blockages to human operators for monitoring and decision support. The predominantly manual inspection of pipe networks will be replaced with teams of autonomous inspection robots that can operate for long periods of time over a large spatial scale. Reliable autonomous navigation and reporting of faults at this scale requires effective localization and mapping, which is the estimation of the robot’s position and its surrounding environment. This survey presents an overview of state-of-the-art works on robot simultaneous localization and mapping (SLAM) with a focus on water and sewer pipe networks. It considers various aspects of the SLAM problem in pipes, from the motivation, to the water industry requirements, modern SLAM methods, map-types and sensors suited to pipes. Future challenges such as robustness for long term robot operation in pipes are discussed, including how making use of prior knowledge, e.g. geographic information systems (GIS) can be used to build map estimates, and improve the multi-robot SLAM in the pipe environmen

Robot Localization in Tunnel-like Environments.

Los entornos confinados como tuberías, túneles o minas constituyen infraestructuras clave para el desarrollo de las economías de los diferentes países. La existencia de estas infraestructuras conlleva la necesidad de llevar a cabo una serie de tareas de mantenimiento mediante inspecciones regulares para asegurar la integridad estructural de las mismas. Así mismo, existen otras tareas que se tienen que realizar en estos entornos como pueden ser misiones de rescate en caso de accidentes e incluso las propias tareas derivadas de la construcción de los mismos. La duras condiciones de este tipo de entornos, ausencia de luz, polvo, presencia de fluidos e incluso de sustancias tóxicas, hace que la ejecución de las mismas suponga un trabajo tedioso e incluso peligroso para las personas. Todo esto, unido a los continuos avances en las tecnologías robóticas, hacen que los robots sean los dispositivos más adecuados para la realización de estas tareas.Para que un robot pueda desempeñar su cometido de manera autónoma, es fundamental que pueda localizarse de manera precisa, no sólo para poder decidir las acciones a llevar a cabo sino también para poder ubicar de manera inequívoca los posibles daños que se puedan detectar durante las labores de inspección. El problema de la localización ha sido ampliamente estudiado en el mundo de la robótica, existiendo multitud de soluciones tanto para interiores como para exteriores mediante el uso de diferentes sensores y tecnologías. Sin embargo, los entornos tipo túnel presentan una serie de características específicas que hacen que la tarea de localización se convierta en todo un reto. La ausencia de iluminación y de características distinguibles tanto visuales como estructurales, hacen que los métodos tradicionales de localización basados en sensores láser y cámaras no funcionen correctamente. Además, al tratarse de entornos confinados, no es posible utilizar sensores típicos de exteriores como es el caso del GPS. La presencia de fluidos e incluso de superficies irregulares hacen poco fiables los métodos basados en odometría utilizando encoders en las ruedas del robot.Por otra parte, estos entornos presentan un comportamiento peculiar en lo que a la propagación de la señal de radiofrecuencia se refiere. Por un lado, a determinadas frecuencias, se comportan como guías de onda extendiendo el alcance de la comunicación, pero por otro, la señal radio sufre fuertes desvanecimientos o fadings. Trabajos previos han demostrado que es posible obtener fadings periódicos bajo una configuración determinada.Partiendo de estos estudios, en esta tesis se aborda el problema de la localización en tuberías y túneles reaprovechando esta naturaleza periódica de la señal radio. Inicialmente, se propone un método de localización para tuberías metálicas basado en técnicas probabilísticas, utilizando el modelo de propagación de la señal como un mapa de radiofrecuencia. Posteriormente, se aborda la localización en túneles siguiendo una estrategia similar de reaprovechar la naturaleza periódica de la señal y se presenta un método de localización discreta. Yendo un paso más allá, y con el objetivo de mejorar la localización a lo largo del túnel incluyendo otras fuentes de información, se desarrolla un método inspirado en el paradigma del graph-SLAM donde se incorporan los resultados obtenidos de la detección de características discretas proporcionadas por el propio túnel.Para ello, se implementa un sistema de detección que proporciona la posición absoluta de características relevantes de la señal periódica radio. Del mismo modo, se desarrolla un método de detección de características estructurales del túnel (galerías) que devuelve la posición conocida de las mismas. Todos estos resultados se incorporan al grafo como fuentes de información.Los métodos de localización desarrollados a lo largo de la tesis han sido validados con datos recolectados durante experimentos llevados a cabo con plataformas robóticas en escenarios reales: la tubería de Santa Ana en Castillonroy y el túnel ferroviario de Somport.<br /

Object-Proposal Evaluation Protocol is 'Gameable'

Object proposals have quickly become the de-facto pre-processing step in a number of vision pipelines (for object detection, object discovery, and other tasks). Their performance is usually evaluated on partially annotated datasets. In this paper, we argue that the choice of using a partially annotated dataset for evaluation of object proposals is problematic -- as we demonstrate via a thought experiment, the evaluation protocol is 'gameable', in the sense that progress under this protocol does not necessarily correspond to a "better" category independent object proposal algorithm. To alleviate this problem, we: (1) Introduce a nearly-fully annotated version of PASCAL VOC dataset, which serves as a test-bed to check if object proposal techniques are overfitting to a particular list of categories. (2) Perform an exhaustive evaluation of object proposal methods on our introduced nearly-fully annotated PASCAL dataset and perform cross-dataset generalization experiments; and (3) Introduce a diagnostic experiment to detect the bias capacity in an object proposal algorithm. This tool circumvents the need to collect a densely annotated dataset, which can be expensive and cumbersome to collect. Finally, we plan to release an easy-to-use toolbox which combines various publicly available implementations of object proposal algorithms which standardizes the proposal generation and evaluation so that new methods can be added and evaluated on different datasets. We hope that the results presented in the paper will motivate the community to test the category independence of various object proposal methods by carefully choosing the evaluation protocol.Comment: 15 pages, 11 figures, 4 table

Detección automática de puntos característicos en el sistema de alcantarillado basado en redes neuronales convolucionales

Desarrollo de un sistema de inteligencia artificial basado en redes neuronales convolucionales (CNNs) para detectar puntos característicos en redes de saneamiento publica. El desarrollo incluye el procesamiento de las bases de datos necesarias, el diseño de las CNNs y el ajuste de los parámetros. Como resultado final se desea obtener un clasificador robusto y con alta precisión que permita su uso sin necesidad de recurrir a imágenes tridimensionales.Development of an artificial intelligent system based on convolutional neural networks (CNNs) in order to detect key points in the sewer net. The development includes processing data bases, design of CNNs and tuninig the parameters. Is expected as final result a robust and high accuracy classifier, which operates independently from depth images.Universidad de Sevilla. Grado en Ingeniería de las Tecnologías Industriale