Efficient Semantic Segmentation on Edge Devices

Semantic segmentation works on the computer vision algorithm for assigning each pixel of an image into a class. The task of semantic segmentation should be performed with both accuracy and efficiency. Most of the existing deep FCNs yield to heavy computations and these networks are very power hungry, unsuitable for real-time applications on portable devices. This project analyzes current semantic segmentation models to explore the feasibility of applying these models for emergency response during catastrophic events. We compare the performance of real-time semantic segmentation models with non-real-time counterparts constrained by aerial images under oppositional settings. Furthermore, we train several models on the Flood-Net dataset, containing UAV images captured after Hurricane Harvey, and benchmark their execution on special classes such as flooded buildings vs. non-flooded buildings or flooded roads vs. non-flooded roads. In this project, we developed a real-time UNet based model and deployed that network on Jetson AGX Xavier module

Photogrammetric techniques for across-scale soil erosion assessment: Developing methods to integrate multi-temporal high resolution topography data at field plots

Soil erosion is a complex geomorphological process with varying influences of different impacts at different spatio-temporal scales. To date, measurement of soil erosion is predominantly realisable at specific scales, thereby detecting separate processes, e.g. interrill erosion contrary to rill erosion. It is difficult to survey soil surface changes at larger areal coverage such as field scale with high spatial resolution. Either net changes at the system outlet or remaining traces after the erosional event are usually measured. Thus, either quasi-point measurements are extrapolated to the corresponding area without knowing the actual sediment source as well as sediment storage behaviour on the plot or erosion rates are estimated disrupting the area of investigation during the data acquisition impeding multi-temporal assessment. Furthermore, established methods of soil erosion detection and quantification are typically only reliable for large event magnitudes, very labour and time intense, or inflexible. To better observe soil erosion processes at field scale and under natural conditions, the development of a method is necessary, which identifies and quantifies sediment sources and sinks at the hillslope with high spatial resolution and captures single precipitation events as well as allows for longer observation periods. Therefore, an approach is introduced, which measures soil surface changes for multi-spatio-temporal scales without disturbing the area of interest. Recent advances regarding techniques to capture high resolution topography (HiRT) data led to several promising tools for soil erosion measurement with corresponding advantages but also disadvantages. The necessity exists to evaluate those methods because they have been rarely utilised in soil surface studies. On the one hand, there is terrestrial laser scanning (TLS), which comprises high error reliability and retrieves 3D information directly. And on the other hand, there is unmanned aerial vehicle (UAV) technology in combination with structure from motion (SfM) algorithms resulting in UAV photogrammetry, which is very flexible in the field and depicts a beneficial perspective. Evaluation of the TLS feasibility reveals that this method implies a systematic error that is distance-related and temporal constant for the investigated device and can be corrected transferring calibration values retrieved from an estimated lookup table. However, TLS still reaches its application limits quickly due to an unfavourable (almost horizontal) scanning view at the soil surface resulting in a fast decrease of point density and increase of noise with increasing distance from the device. UAV photogrammetry allows for a better perspective (birds-eye view) onto the area of interest, but possesses more complex error behaviour, especially in regard to the systematic error of a DEM dome, which depends on the method for 3D reconstruction from 2D images (i.e. options for additional implementation of observations) and on the image network configuration (i.e. parallel-axes and control point configuration). Therefore, a procedure is developed that enables flexible usage of different cameras and software tools without the need of additional information or specific camera orientations and yet avoiding this dome error. Furthermore, the accuracy potential of UAV photogrammetry describing rough soil surfaces is assessed because so far corresponding data is missing. Both HiRT methods are used for multi-temporal measurement of soil erosion processes resulting in surface changes of low magnitudes, i.e. rill and especially interrill erosion. Thus, a reference with high accuracy and stability is a requirement. A local reference system with sub-cm and at its best 1 mm accuracy is setup and confirmed by control surveys. TLS and UAV photogrammetry data registration with these targets ensures that errors due to referencing are of minimal impact. Analysis of the multi-temporal performance of both HiRT methods affirms TLS to be suitable for the detection of erosion forms of larger magnitudes because of a level of detection (LoD) of 1.5 cm. UAV photogrammetry enables the quantification of even lower magnitude changes (LoD of 1 cm) and a reliable observation of the change of surface roughness, which is important for runoff processes, at field plots due to high spatial resolution (1 cm²). Synergetic data fusion as a subsequent post-processing step is necessary to exploit the advantages of both HiRT methods and potentially further increase the LoD. The unprecedented high level of information entails the need for automatic geomorphic feature extraction due to the large amount of novel content. Therefore, a method is developed, which allows for accurate rill extraction and rill parameter calculation with high resolution enabling new perspectives onto rill erosion that has not been possible before due to labour and area access limits. Erosion volume and cross sections are calculated for each rill revealing a dominant rill deepening. Furthermore, rill shifting in dependence of the rill orientation towards the dominant wind direction is revealed. Two field plots are installed at erosion prone positions in the Mediterranean (1,000 m²) and in the European loess belt (600 m²) to ensure the detection of surface changes, permitting the evaluation of the feasibility, potential and limits of TLS and UAV photogrammetry in soil erosion studies. Observations are made regarding sediment connectivity at the hillslope scale. Both HiRT methods enable the identification of local sediment sources and sinks, but still exhibiting some degree of uncertainty due to the comparable high LoD in regard to laminar accumulation and interrill erosion processes. At both field sites wheel tracks and erosion rills increase hydrological and sedimentological connectivity. However, at the Mediterranean field plot especially dis-connectivity is obvious. At the European loess belt case study a triggering event could be captured, which led to high erosion rates due to high soil moisture contents and yet further erosion increase due to rill amplification after rill incision. Estimated soil erosion rates range between 2.6 tha-1 and 121.5 tha-1 for single precipitation events and illustrate a large variability due to very different site specifications, although both case studies are located in fragile landscapes. However, the susceptibility to soil erosion has different primary causes, i.e. torrential precipitation at the Mediterranean site and high soil erodibility at the European loess belt site. The future capability of the HiRT methods is their potential to be applicable at yet larger scales. Hence, investigations of the importance of gullys for sediment connectivity between hillslopes and channels are possible as well as the possible explanation of different erosion rates observed at hillslope and at catchment scales because local sediment sink and sources can be quantified. In addition, HiRT data can be a great tool for calibrating, validating and enhancing soil erosion models due to the unprecedented level of detail and the flexible multi-spatio-temporal application

Robust perceptual organization techniques for analysis of color images

Esta tesis aborda el desarrollo de nuevas técnicas de análisis robusto de imágenes estrechamente relacionadas con el comportamiento del sistema visual humano. Uno de los pilares de la tesis es la votación tensorial, una técnica robusta que propaga y agrega información codificada en tensores mediante un proceso similar a la convolución. Su robustez y adaptabilidad han sido claves para su uso en esta tesis. Ambas propiedades han sido verificadas en tres nuevas aplicaciones de la votación tensorial: estimación de estructura, detección de bordes y segmentación de imágenes adquiridas mediante estereovisión.El mayor problema de la votación tensorial es su elevado coste computacional. En esta línea, esta tesis propone dos nuevas implementaciones eficientes de la votación tensorial derivadas de un análisis en profundidad de esta técnica.A pesar de su capacidad de adaptación, esta tesis muestra que la formulación original de la votación tensorial (a partir de aquí, votación tensorial clásica) no es adecuada para algunas aplicaciones, dado que las hipótesis en las que se basa no se ajustan a todas ellas. Esto ocurre particularmente en el filtrado de imágenes en color. Así, esta tesis muestra que, más que un método, la votación tensorial es una metodología en la que la codificación y el proceso de votación pueden ser adaptados específicamente para cada aplicación, manteniendo el espíritu de la votación tensorial.En esta línea, esta tesis propone un marco unificado en el que se realiza a la vez el filtrado de imágenes y la detección robusta de bordes. Este marco de trabajo es una extensión de la votación tensorial clásica en la que el color y la probabilidad de encontrar un borde en cada píxel se codifican mediante tensores, y en el que el proceso de votación se basa en un conjunto de criterios perceptuales relacionados con el modo en que el sistema visual humano procesa información. Los avances recientes en la percepción del color han sido esenciales en el diseño de dicho proceso de votación.Este nuevo enfoque ha sido efectivo, obteniendo excelentes resultados en ambas aplicaciones. En concreto, el nuevo método aplicado al filtrado de imágenes tiene un mejor rendimiento que los métodos del estado del arte para ruido real. Esto lo hace más adecuado para aplicaciones reales, donde los algoritmos de filtrado son imprescindibles. Además, el método aplicado a detección de bordes produce resultados más robustos que las técnicas del estado del arte y tiene un rendimiento competitivo con relación a la completitud, discriminabilidad, precisión y rechazo de falsas alarmas.Además, esta tesis demuestra que este nuevo marco de trabajo puede combinarse con otras técnicas para resolver el problema de segmentación robusta de imágenes. Los tensores obtenidos mediante el nuevo método se utilizan para clasificar píxeles como probablemente homogéneos o no homogéneos. Ambos tipos de píxeles se segmentan a continuación por medio de una variante de un algoritmo eficiente de segmentación de imágenes basada en grafos. Los experimentos muestran que el algoritmo propuesto obtiene mejores resultados en tres de las cinco métricas de evaluación aplicadas en comparación con las técnicas del estado del arte, con un coste computacional competitivo.La tesis también propone nuevas técnicas de evaluación en el ámbito del procesamiento de imágenes. En concreto, se proponen dos métricas de filtrado de imágenes con el fin de medir el grado en que un método es capaz de preservar los bordes y evitar la introducción de defectos. Asimismo, se propone una nueva metodología para la evaluación de detectores de bordes que evita posibles sesgos introducidos por el post-procesado. Esta metodología se basa en cinco métricas para estimar completitud, discriminabilidad, precisión, rechazo de falsas alarmas y robustez. Por último, se proponen dos nuevas métricas no paramétricas para estimar el grado de sobre e infrasegmentación producido por los algoritmos de segmentación de imágenes.This thesis focuses on the development of new robust image analysis techniques more closely related to the way the human visual system behaves. One of the pillars of the thesis is the so called tensor voting technique. This is a robust perceptual organization technique that propagates and aggregates information encoded by means of tensors through a convolution like process. Its robustness and adaptability have been one of the key points for using tensor voting in this thesis. These two properties are verified in the thesis by applying tensor voting to three applications where it had not been applied so far: image structure estimation, edge detection and image segmentation of images acquired through stereo vision.The most important drawback of tensor voting is that its usual implementations are highly time consuming. In this line, this thesis proposes two new efficient implementations of tensor voting, both derived from an in depth analysis of this technique.Despite its adaptability, this thesis shows that the original formulation of tensor voting (hereafter, classical tensor voting) is not adequate for some applications, since the hypotheses from which it is based are not suitable for all applications. This is particularly certain for color image denoising. Thus, this thesis shows that, more than a method, tensor voting can be thought of as a methodology in which the encoding and voting process can be tailored for every specific application, while maintaining the tensor voting spirit.By following this reasoning, this thesis proposes a unified framework for both image denoising and robust edge detection.This framework is an extension of the classical tensor voting in which both color and edginess the likelihood of finding an edge at every pixel of the image are encoded through tensors, and where the voting process takes into account a set of plausible perceptual criteria related to the way the human visual system processes visual information. Recent advances in the perception of color have been essential for designing such a voting process.This new approach has been found effective, since it yields excellent results for both applications. In particular, the new method applied to image denoising has a better performance than other state of the art methods for real noise. This makes it more adequate for real applications, in which an image denoiser is indeed required. In addition, the method applied to edge detection yields more robust results than the state of the art techniques and has a competitive performance in recall, discriminability, precision, and false alarm rejection.Moreover, this thesis shows how the results of this new framework can be combined with other techniques to tackle the problem of robust color image segmentation. The tensors obtained by applying the new framework are utilized to classify pixels into likely homogeneous and likely inhomogeneous. Those pixels are then sequentially segmented through a variation of an efficient graph based image segmentation algorithm. Experiments show that the proposed segmentation algorithm yields better scores in three of the five applied evaluation metrics when compared to the state of the art techniques with a competitive computational cost.This thesis also proposes new evaluation techniques in the scope of image processing. First, two new metrics are proposed in the field of image denoising: one to measure how an algorithm is able to preserve edges, and the second to measure how a method is able not to introduce undesirable artifacts. Second, a new methodology for assessing edge detectors that avoids possible bias introduced by post processing is proposed. It consists of five new metrics for assessing recall, discriminability, precision, false alarm rejection and robustness. Finally, two new non parametric metrics are proposed for estimating the degree of over and undersegmentation yielded by image segmentation algorithms

Autonomous Navigation for Unmanned Aerial Systems - Visual Perception and Motion Planning

L'abstract è presente nell'allegato / the abstract is in the attachmen

Deep Learning Methods for Remote Sensing

Remote sensing is a field where important physical characteristics of an area are exacted using emitted radiation generally captured by satellite cameras, sensors onboard aerial vehicles, etc. Captured data help researchers develop solutions to sense and detect various characteristics such as forest fires, flooding, changes in urban areas, crop diseases, soil moisture, etc. The recent impressive progress in artificial intelligence (AI) and deep learning has sparked innovations in technologies, algorithms, and approaches and led to results that were unachievable until recently in multiple areas, among them remote sensing. This book consists of sixteen peer-reviewed papers covering new advances in the use of AI for remote sensing

Exploiting Spatio-Temporal Coherence for Video Object Detection in Robotics

This paper proposes a method to enhance video object detection for indoor environments in robotics. Concretely, it exploits knowledge about the camera motion between frames to propagate previously detected objects to successive frames. The proposal is rooted in the concepts of planar homography to propose regions of interest where to find objects, and recursive Bayesian filtering to integrate observations over time. The proposal is evaluated on six virtual, indoor environments, accounting for the detection of nine object classes over a total of ∼ 7k frames. Results show that our proposal improves the recall and the F1-score by a factor of 1.41 and 1.27, respectively, as well as it achieves a significant reduction of the object categorization entropy (58.8%) when compared to a two-stage video object detection method used as baseline, at the cost of small time overheads (120 ms) and precision loss (0.92).</p

EG-ICE 2021 Workshop on Intelligent Computing in Engineering

The 28th EG-ICE International Workshop 2021 brings together international experts working at the interface between advanced computing and modern engineering challenges. Many engineering tasks require open-world resolutions to support multi-actor collaboration, coping with approximate models, providing effective engineer-computer interaction, search in multi-dimensional solution spaces, accommodating uncertainty, including specialist domain knowledge, performing sensor-data interpretation and dealing with incomplete knowledge. While results from computer science provide much initial support for resolution, adaptation is unavoidable and most importantly, feedback from addressing engineering challenges drives fundamental computer-science research. Competence and knowledge transfer goes both ways

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC

Robot Manipulators

Robot manipulators are developing more in the direction of industrial robots than of human workers. Recently, the applications of robot manipulators are spreading their focus, for example Da Vinci as a medical robot, ASIMO as a humanoid robot and so on. There are many research topics within the field of robot manipulators, e.g. motion planning, cooperation with a human, and fusion with external sensors like vision, haptic and force, etc. Moreover, these include both technical problems in the industry and theoretical problems in the academic fields. This book is a collection of papers presenting the latest research issues from around the world