Capturing 3D textured inner pipe surfaces for sewer inspection

Inspection robots equipped with TV camera technology are commonly used to detect defects in sewer systems. Currently, these defects are predominantly identified by human assessors, a process that is not only time-consuming and costly but also susceptible to errors. Furthermore, existing systems primarily offer only information from 2D imaging for damage assessment, limiting the accurate identification of certain types of damage due to the absence of 3D information. Thus, the necessary solid quantification and characterisation of damage, which is needed to evaluate remediation measures and the associated costs, is limited from the sensory side. In this paper, we introduce an innovative system designed for acquiring multimodal image data using a camera measuring head capable of capturing both color and 3D images with high accuracy and temporal availability based on the single-shot principle. This sensor head, affixed to a carriage, continuously captures the sewer's inner wall during transit. The collected data serves as the basis for an AI-based automatic analysis of pipe damages as part of the further assessment and monitoring of sewers. Moreover, this paper is focused on the fundamental considerations about the design of the multimodal measuring head and elaborates on some application-specific implementation details. These include data pre-processing, 3D reconstruction, registration of texture and depth images, as well as 2D-3D registration and 3D image fusion

Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

A bayesian approach to simultaneously recover camera pose and non-rigid shape from monocular images

© . This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/In this paper we bring the tools of the Simultaneous Localization and Map Building (SLAM) problem from a rigid to a deformable domain and use them to simultaneously recover the 3D shape of non-rigid surfaces and the sequence of poses of a moving camera. Under the assumption that the surface shape may be represented as a weighted sum of deformation modes, we show that the problem of estimating the modal weights along with the camera poses, can be probabilistically formulated as a maximum a posteriori estimate and solved using an iterative least squares optimization. In addition, the probabilistic formulation we propose is very general and allows introducing different constraints without requiring any extra complexity. As a proof of concept, we show that local inextensibility constraints that prevent the surface from stretching can be easily integrated. An extensive evaluation on synthetic and real data, demonstrates that our method has several advantages over current non-rigid shape from motion approaches. In particular, we show that our solution is robust to large amounts of noise and outliers and that it does not need to track points over the whole sequence nor to use an initialization close from the ground truth.Peer ReviewedPostprint (author's final draft

A Comparative Study of Registration Methods for RGB-D Video of Static Scenes

The use of RGB-D sensors for mapping and recognition tasks in robotics or, in general, for virtual reconstruction has increased in recent years. The key aspect of these kinds of sensors is that they provide both depth and color information using the same device. In this paper, we present a comparative analysis of the most important methods used in the literature for the registration of subsequent RGB-D video frames in static scenarios. The analysis begins by explaining the characteristics of the registration problem, dividing it into two representative applications: scene modeling and object reconstruction. Then, a detailed experimentation is carried out to determine the behavior of the different methods depending on the application. For both applications, we used standard datasets and a new one built for object reconstruction.This work has been supported by a grant from the Spanish Government, DPI2013-40534-R, University of Alicante projects GRE11-01 and a grant from the Valencian Government, GV/2013/005

An Approach Of Features Extraction And Heatmaps Generation Based Upon Cnns And 3D Object Models

The rapid advancements in artificial intelligence have enabled recent progress of self-driving vehicles. However, the dependence on 3D object models and their annotations collected and owned by individual companies has become a major problem for the development of new algorithms. This thesis proposes an approach of directly using graphics models created from open-source datasets as the virtual representation of real-world objects. This approach uses Machine Learning techniques to extract 3D feature points and to create annotations from graphics models for the recognition of dynamic objects, such as cars, and for the verification of stationary and variable objects, such as buildings and trees. Moreover, it generates heat maps for the elimination of stationary/variable objects in real-time images before working on the recognition of dynamic objects. The proposed approach helps to bridge the gap between the virtual and physical worlds and to facilitate the development of new algorithms for self-driving vehicles

Comparison of Methods for Matching of Images with Weakly Textured Areas

Tato bakalářská práce se zabývá srovnáním metod pro hledání korespondencí mezi snímky, kteréžto metody jsou potom využity v 3D rekonstrukci. Především se práce zaměřuje na jejich schopnost vypořádat se s těžko rekonstruovatelnými fakotry a oblastmi, které snižují kvalitu rekonstrukce na daných datasetech.This thesis contains a closer look at comparison correspondence search methods used in 3D image reconstructions. Primarily at their ability to tackle reconstruction-problematic factors and areas that are decreasing the quality of reconstruction of given datasets

Using Surfaces and Surface Relations in an Early Cognitive Vision System

The final publication is available at Springer via http://dx.doi.org/10.1007/s00138-015-0705-yWe present a deep hierarchical visual system with two parallel hierarchies for edge and surface information. In the two hierarchies, complementary visual information is represented on different levels of granularity together with the associated uncertainties and confidences. At all levels, geometric and appearance information is coded explicitly in 2D and 3D allowing to access this information separately and to link between the different levels. We demonstrate the advantages of such hierarchies in three applications covering grasping, viewpoint independent object representation, and pose estimation.European Community’s Seventh Framework Programme FP7/IC

Automatically Controlled Morphing of 2D Shapes with Textures

This paper deals with 2D image transformations from a perspective of a 3D heterogeneous shape modeling and computer animation. Shape and image morphing techniques have attracted a lot of attention in artistic design, computer animation, and interactive and streaming applications. We present a novel method for morphing between two topologically arbitrary 2D shapes with sophisticated textures (raster color attributes) using a metamorphosis technique called space-time blending (STB) coupled with space-time transfinite interpolation. The method allows for a smooth transition between source and target objects by generating in-between shapes and associated textures without setting any correspondences between boundary points or features. The method requires no preprocessing and can be applied in 2D animation when position and topology of source and target objects are significantly different. With the conversion of given 2D shapes to signed distance fields, we have detected a number of problems with directly applying STB to them. We propose a set of novel and mathematically substantiated techniques, providing automatic control of the morphing process with STB and an algorithm of applying those techniques in combination. We illustrate our method with applications in 2D animation and interactive applications