TRACKING THE URBAN CHAMELEON – TOWARDS A HYBRID CHANGE DETECTION OF GRAFFITI

Colourful and ever-changing: Graffiti can be considered the urban chameleon skin. At the Donaukanal (Eng. Danube Channel), Vienna's central waterway and one of the largest and most active graffiti-scapes worldwide, this metaphor applies like hardly anywhere else. Every day a multitude of graffiti is destroyed by the creation of new works. Recently, efforts have been made to mitigate this constant loss of cultural heritage along the Donaukanal by systematically documenting the graffiti, mainly using photography and photogrammetry. However, keeping track of the newly added works is very time-consuming and often like finding needles in a haystack, considering the large extent and high volatility of the monitored area. Thus, an automated graffiti change detection would significantly reduce the effort and avoid overlooking graffiti.This contribution outlines the main challenges in image-based change detection for cultural heritage and proposes a hybrid graffiti change detection method. The investigated method exploits and combines an established pixel-based change detection algorithm, the Iteratively Multivariate Alteration Detection, with a novel descriptor-based method. The latter relies on image features, rather than pixels as analysis unit and can robustly filter false alarms from the high-performing but noise-prone pixel-based approach. Overall, the results indicate that the proposed method can largely automate image-based change detection of graffiti-scapes. It can uncover graffiti-related changes and robustly distinguish them from other image differences such as shadows but tends to overlook small-scale graffiti, indicating the need for further fine-tuning.</p

Code-Aligned Autoencoders for Unsupervised Change Detection in Multimodal Remote Sensing Images

Image translation with convolutional autoencoders has recently been used as an approach to multimodal change detection (CD) in bitemporal satellite images. A main challenge is the alignment of the code spaces by reducing the contribution of change pixels to the learning of the translation function. Many existing approaches train the networks by exploiting supervised information of the change areas, which, however, is not always available. We propose to extract relational pixel information captured by domain-specific affinity matrices at the input and use this to enforce alignment of the code spaces and reduce the impact of change pixels on the learning objective. A change prior is derived in an unsupervised fashion from pixel pair affinities that are comparable across domains. To achieve code space alignment, we enforce pixels with similar affinity relations in the input domains to be correlated also in code space. We demonstrate the utility of this procedure in combination with cycle consistency. The proposed approach is compared with the state-of-the-art machine learning and deep learning algorithms. Experiments conducted on four real and representative datasets show the effectiveness of our methodology

Fotogrametría de rango cercano aplicada a la Ingeniería Agroforestal

Tesis por compendio de publicaciones[EN]Since the late twentieth century, Geotechnologies are being applied in different research lines in Agroforestry Engineering aimed at advancing in the modeling of biophysical parameters in order to improve the productivity. In this study, low-cost and close range photogrammetry has been used in different agroforestry scenarios to solve identified gaps in the results and improve procedures and technology hitherto practiced in this field. Photogrammetry offers the advantage of being a non-destructive and non-invasive technique, never changing physical properties of the studied element, providing rigor and completeness to the captured information. In this PhD dissertation, the following contributions are presented divided into three research papers: • A methodological proposal to acquire georeferenced multispectral data of high spatial resolution using a low-cost manned aerial platform, to monitor and sustainably manage extensive áreas of crops. The vicarious calibration is exposed as radiometric calibration method of the multispectral sensor embarked on a paraglider. Low-cost surfaces are performed as control coverages. • The development of a method able to determine crop productivity under field conditions, from the combination of close range photogrammetry and computer vision, providing a constant operational improvement and a proactive management in the crop monitoring. An innovate methodology in the sector is proposed, ensuring flexibility and simplicity in the data collection by non-invasive technologies, automation in processing and quality results with low associated cost. • A low cost, efficient and accurate methodology to obtain Digital Height Models of vegatal cover intended for forestry inventories by integrating public data from LiDAR into photogrammetric point clouds coming from low cost flights. This methodology includes the potentiality of LiDAR to register ground points in areas with high density of vegetation and the better spatial, radiometric and temporal resolution from photogrammetry for the top of vegetal covers.[ES]Desde finales del siglo XX se están aplicando Geotecnologías en diferentes líneas de investigación en Ingeniería Agroforestal orientadas a avanzar en la modelización de parámetros biofísicos con el propósito de mejorar la productividad. En este estudio se ha empleado fotogrametría de bajo coste y rango cercano en distintos escenarios agroforestales para solventar carencias detectadas en los resultados obtenidos y mejorar los procedimientos y la tecnología hasta ahora usados en este campo. La fotogrametría ofrece como ventaja el ser una técnica no invasiva y no destructiva, por lo que no altera en ningún momento las propiedades físicas del elemento estudiado, dotando de rigor y exhaustividad a la información capturada. En esta Tesis Doctoral se presentan las siguientes contribuciones, divididas en tres artículos de investigación: • Una propuesta metodológica de adquisición de datos multiespectrales georreferenciados de alta resolución espacial mediante una plataforma aérea tripulada de bajo coste, para monitorizar y gestionar sosteniblemente amplias extensiones de cultivos. Se expone la calibración vicaria como método de calibración radiométrico del sensor multiespectral embarcado en un paramotor empleando como coberturas de control superficies de bajo coste. • El desarrollo de un método capaz de determinar la productividad del cultivo en condiciones de campo, a partir de la combinación de fotogrametría de rango cercano y visión computacional, facilitando una mejora operativa constante así como una gestión proactiva en la monitorización del cultivo. Se propone una metodología totalmente novedosa en el sector, garantizando flexibilidad y sencillez en la toma de datos mediante tecnologías no invasivas, automatismo en el procesado, calidad en los resultados y un bajo coste asociado. • Una metodología de bajo coste, eficiente y precisa para la obtención de Modelos Digitales de Altura de Cubierta Vegetal destinados al inventario forestal mediante la integración de datos públicos procedentes del LiDAR en las nubes de puntos fotogramétricas obtenidas con un vuelo de bajo coste. Esta metodología engloba la potencialidad del LiDAR para registrar el terreno en zonas con alta densidad de vegetación y una mejor resolución espacial, radiométrica y temporal procedente de la fotogrametría para la parte superior de las cubiertas vegetales

Urban Detection From Hyperspectral Images Using Dimension-Reduction Model and Fusion of Multiple Segmentations Based on Stuctural and Textural Features

Ce mémoire de maîtrise présente une nouvelle approche non supervisée pour détecter et segmenter les régions urbaines dans les images hyperspectrales. La méthode proposée n ́ecessite trois étapes. Tout d’abord, afin de réduire le coût calculatoire de notre algorithme, une image couleur du contenu spectral est estimée. A cette fin, une étape de réduction de dimensionalité non-linéaire, basée sur deux critères complémentaires mais contradictoires de bonne visualisation; à savoir la précision et le contraste, est réalisée pour l’affichage couleur de chaque image hyperspectrale. Ensuite, pour discriminer les régions urbaines des régions non urbaines, la seconde étape consiste à extraire quelques caractéristiques discriminantes (et complémentaires) sur cette image hyperspectrale couleur. A cette fin, nous avons extrait une série de paramètres discriminants pour décrire les caractéristiques d’une zone urbaine, principalement composée d’objets manufacturés de formes simples g ́eométriques et régulières. Nous avons utilisé des caractéristiques texturales basées sur les niveaux de gris, la magnitude du gradient ou des paramètres issus de la matrice de co-occurrence combinés avec des caractéristiques structurelles basées sur l’orientation locale du gradient de l’image et la détection locale de segments de droites. Afin de réduire encore la complexité de calcul de notre approche et éviter le problème de la ”malédiction de la dimensionnalité” quand on décide de regrouper des données de dimensions élevées, nous avons décidé de classifier individuellement, dans la dernière étape, chaque caractéristique texturale ou structurelle avec une simple procédure de K-moyennes et ensuite de combiner ces segmentations grossières, obtenues à faible coût, avec un modèle efficace de fusion de cartes de segmentations. Les expérimentations données dans ce rapport montrent que cette stratégie est efficace visuellement et se compare favorablement aux autres méthodes de détection et segmentation de zones urbaines à partir d’images hyperspectrales.This master’s thesis presents a new approach to urban area detection and segmentation in hyperspectral images. The proposed method relies on a three-step procedure. First, in order to decrease the computational complexity, an informative three-colour composite image, minimizing as much as possible the loss of information of the spectral content, is computed. To this end, a non-linear dimensionality reduction step, based on two complementary but contradictory criteria of good visualization, namely accuracy and contrast, is achieved for the colour display of each hyperspectral image. In order to discriminate between urban and non-urban areas, the second step consists of extracting some complementary and discriminant features on the resulting (three-band) colour hyperspectral image. To attain this goal, we have extracted a set of features relevant to the description of different aspects of urban areas, which are mainly composed of man-made objects with regular or simple geometrical shapes. We have used simple textural features based on grey-levels, gradient magnitude or grey-level co-occurence matrix statistical parameters combined with structural features based on gradient orientation, and straight segment detection. In order to also reduce the computational complexity and to avoid the so-called “curse of dimensionality” when clustering high-dimensional data, we decided, in the final third step, to classify each individual feature (by a simple K-means clustering procedure) and to combine these multiple low-cost and rough image segmentation results with an efficient fusion model of segmentation maps. The experiments reported in this report demonstrate that the proposed segmentation method is efficient in terms of visual evaluation and performs well compared to existing and automatic detection and segmentation methods of urban areas from hyperspectral images

Desarrollo de geotecnologías aplicadas a la inspección y monitorización de entornos industriales

Tesis por compendio de publicaciones[ES]El desarrollo tecnológico de las últimas dos décadas ha supuesto un cambio radical que está llevando a un nuevo paradigma en el que se entremezclan el mundo físico y el digital. Estos cambios han influido enormemente en la sociedad, modificando las formas de comunicación, acceso a información, ocio, trabajo, etc. Asimismo, la industria ha adoptado estas tecnologías disruptivas, las cuales están contribuyendo a lograr un mayor control y automatización del proceso productivo. En el ámbito industrial, las tareas de mantenimiento son críticas para garantizar el correcto funcionamiento de una planta o instalación, ya que influyen directamente en la productividad y pueden suponer un elevado costo adicional. Las nuevas tecnologías están posibilitando la monitorización continua y a la inspección automatizada, proporcionando herramientas auxiliares a los inspectores que mejoran la detección de fallos y permiten anticipar y optimizar la planificación de las tareas de mantenimiento. Con el objetivo de desarrollar herramientas que aporten mejoras en las tareas de mantenimiento en industria, la presente tesis doctoral se basa en el estudio de como las geotecnologías pueden aportar soluciones óptimas en la monitorización e inspección. Debido a la gran variedad de entornos industriales, las herramientas de apoyo al mantenimiento deben adaptarse a cada caso en concreto. En este aspecto, y con el fin de demostrar la adaptabilidad de la geomática y las geotecnologías, se han estudiado instalaciones industriales de ámbitos muy diversos, como una sala de máquinas (escenario interior), plantas fotovoltaicas (escenario exterior) y soldaduras (interior y exterior). La escala de los escenarios objeto de estudio ha sido muy variada, desde las escalas más pequeñas, para el estudio de las soldaduras y la sala de máquinas, a las escalas más grandes, en los estudios de evolución de la vegetación y presencia de masas de agua en plantas fotovoltaicas. Las geotecnologías demuestran su versatilidad para trabajar a distintas escalas, con soluciones que permiten un gran detalle y precisión, como la fotogrametría de rango cercano y el sistema de escaneado portátil (Portable Mobile Mapping System - PMMS), y otras que pueden abarcar zonas más amplias del territorio, como es el caso de la teledetección o la fotogrametría con drones. Según lo expuesto anteriormente, el enfoque de la tesis ha sido el estudio de elementos o instalaciones industriales a diferentes escalas. En el primer caso se desarrolló una herramienta para el control de calidad externo de soldaduras utilizando fotogrametría de rango cercano y algoritmos para la detección automática de defectos. En el segundo caso se propuso el uso de un PMMS para optimizar la toma de datos en las tareas de inspección en instalaciones fluidomecánicas. En el tercer caso se utilizó la fotogrametría con drones y la combinación de imágenes RGB y térmicas con algoritmos de visión computacional para la detección de patologías en paneles fotovoltaicos. Finalmente, para la monitorización de la vegetación y la detección de masas de agua en el entorno de plantas fotovoltaicas, se empleó la teledetección mediante el cálculo de índices espectrales. [EN]The technological development of the last two decades has brought about a radical change that is leading to a new paradigm in which the physical and digital worlds are intertwined. These changes have had a great impact on society, modifying communication methods, access to information, leisure, work, etc. In addition, the industry has adopted these disruptive technologies, which are contributing to achieving greater control and automation of the production process. In the industrial sector, maintenance tasks are critical to ensuring the proper operation of a plant or facility, as they directly influence productivity and can involve high additional costs. New technologies are making continuous monitoring and automated inspection possible, providing auxiliary tools to inspectors that improve fault detection and allow for the anticipation and optimization of maintenance task planning. With the aim of developing tools that provide improvements in maintenance tasks in industry, this doctoral thesis is based on the study of how geotechnologies can provide optimal solutions in monitoring and inspection. Due to the great variety of industrial environments, maintenance support tools must adapt to each specific case. In this regard, and in order to demonstrate the adaptability of geomatics and geotechnologies, industrial installations from very diverse areas have been studied, such as a machine room (indoor scenario), photovoltaic plants (outdoor scenario), and welding (indoor and outdoor scenarios). The scale of the studied scenarios has been very varied, ranging from smaller scales for the study of welds and machine rooms, to larger scales in the studies of vegetation evolution and the presence of bodies of water in photovoltaic plants. Geotechnologies demonstrate their versatility to work at different scales, with solutions that allow for great detail and precision, such as close-range photogrammetry and the Portable Mobile Mapping System (PMMS), as well as others that can cover larger areas of the territory, such as remote sensing or photogrammetry with drones. The focus of the thesis has been the study of industrial elements or installations at different scales. In the first case, a tool was developed for external quality control of welding, using close-range photogrammetry and algorithms for automatic defect detection. In the second case, the use of a PMMS is proposed to optimize data collection in fluid-mechanical installation inspection tasks. In the third case, drone photogrammetry and the combination of RGB and thermal images with computer vision algorithms were used for the detection of pathologies in photovoltaic panels. Finally, for the monitoring of vegetation and the detection of water masses in the environment of photovoltaic plants, remote sensing was employed through the calculation of spectral indices

Exploiting Spatio-Temporal Coherence for Video Object Detection in Robotics

This paper proposes a method to enhance video object detection for indoor environments in robotics. Concretely, it exploits knowledge about the camera motion between frames to propagate previously detected objects to successive frames. The proposal is rooted in the concepts of planar homography to propose regions of interest where to find objects, and recursive Bayesian filtering to integrate observations over time. The proposal is evaluated on six virtual, indoor environments, accounting for the detection of nine object classes over a total of ∼ 7k frames. Results show that our proposal improves the recall and the F1-score by a factor of 1.41 and 1.27, respectively, as well as it achieves a significant reduction of the object categorization entropy (58.8%) when compared to a two-stage video object detection method used as baseline, at the cost of small time overheads (120 ms) and precision loss (0.92).</p