COMPARITIVE STUDY OF TREE COUNTING ALGORITHMS IN DENSE AND SPARSE VEGETATIVE REGIONS

Tree counting can be a challenging and time consuming task, especially if done manually. This study proposes and compares three different approaches for automatic detection and counting of trees in different vegetative regions. First approach is to mark extended minima’s, extended maxima’s along with morphological reconstruction operations on an image for delineation and tree crown segmentation. To separate two touching crowns, a marker controlled watershed algorithm is used. For second approach, the color segmentation method for tree identification is used. Starting with the conversion of an RGB image to HSV color space then filtering, enhancing and thresholding to isolate trees from non-trees elements followed by watershed algorithm to separate touching tree crowns. Third approach involves deep learning method for classification of tree and non-tree, using approximately 2268 positive and 1172 negative samples each. Each segment of an image is then classified and sliding window algorithm is used to locate each tree crown. Experimentation shows that the first approach is well suited for classification of trees is dense vegetation, whereas the second approach is more suitable for detecting trees in sparse vegetation. Deep learning classification accuracy lies in between these two approaches and gave an accuracy of 92% on validation data. The study shows that deep learning can be used as a quick and effective tool to ascertain the count of trees from airborne optical imagery

Efficient Dense Registration, Segmentation, and Modeling Methods for RGB-D Environment Perception

One perspective for artificial intelligence research is to build machines that perform tasks autonomously in our complex everyday environments. This setting poses challenges to the development of perception skills: A robot should be able to perceive its location and objects in its surrounding, while the objects and the robot itself could also be moving. Objects may not only be composed of rigid parts, but could be non-rigidly deformable or appear in a variety of similar shapes. Furthermore, it could be relevant to the task to observe object semantics. For a robot acting fluently and immediately, these perception challenges demand efficient methods. This theses presents novel approaches to robot perception with RGB-D sensors. It develops efficient registration, segmentation, and modeling methods for scene and object perception. We propose multi-resolution surfel maps as a concise representation for RGB-D measurements. We develop probabilistic registration methods that handle rigid scenes, scenes with multiple rigid parts that move differently, and scenes that undergo non-rigid deformations. We use these methods to learn and perceive 3D models of scenes and objects in both static and dynamic environments. For learning models of static scenes, we propose a real-time capable simultaneous localization and mapping approach. It aligns key views in RGB-D video using our rigid registration method and optimizes the pose graph of the key views. The acquired models are then perceived in live images through detection and tracking within a Bayesian filtering framework. An assumption frequently made for environment mapping is that the observed scene remains static during the mapping process. Through rigid multi-body registration, we take advantage of releasing this assumption: Our registration method segments views into parts that move independently between the views and simultaneously estimates their motion. Within simultaneous motion segmentation, localization, and mapping, we separate scenes into objects by their motion. Our approach acquires 3D models of objects and concurrently infers hierarchical part relations between them using probabilistic reasoning. It can be applied for interactive learning of objects and their part decomposition. Endowing robots with manipulation skills for a large variety of objects is a tedious endeavor if the skill is programmed for every instance of an object class. Furthermore, slight deformations of an instance could not be handled by an inflexible program. Deformable registration is useful to perceive such shape variations, e.g., between specific instances of a tool. We develop an efficient deformable registration method and apply it for the transfer of robot manipulation skills between varying object instances. On the object-class level, we segment images using random decision forest classifiers in real-time. The probabilistic labelings of individual images are fused in 3D semantic maps within a Bayesian framework. We combine our object-class segmentation method with simultaneous localization and mapping to achieve online semantic mapping in real-time. The methods developed in this thesis are evaluated in experiments on publicly available benchmark datasets and novel own datasets. We publicly demonstrate several of our perception approaches within integrated robot systems in the mobile manipulation context.Effiziente Dichte Registrierungs-, Segmentierungs- und Modellierungsmethoden für die RGB-D Umgebungswahrnehmung In dieser Arbeit beschäftigen wir uns mit Herausforderungen der visuellen Wahrnehmung für intelligente Roboter in Alltagsumgebungen. Solche Roboter sollen sich selbst in ihrer Umgebung zurechtfinden, und Wissen über den Verbleib von Objekten erwerben können. Die Schwierigkeit dieser Aufgaben erhöht sich in dynamischen Umgebungen, in denen ein Roboter die Bewegung einzelner Teile differenzieren und auch wahrnehmen muss, wie sich diese Teile bewegen. Bewegt sich ein Roboter selbständig in dieser Umgebung, muss er auch seine eigene Bewegung von der Veränderung der Umgebung unterscheiden. Szenen können sich aber nicht nur durch die Bewegung starrer Teile verändern. Auch die Teile selbst können ihre Form in nicht-rigider Weise ändern. Eine weitere Herausforderung stellt die semantische Interpretation von Szenengeometrie und -aussehen dar. Damit intelligente Roboter unmittelbar und flüssig handeln können, sind effiziente Algorithmen für diese Wahrnehmungsprobleme erforderlich. Im ersten Teil dieser Arbeit entwickeln wir effiziente Methoden zur Repräsentation und Registrierung von RGB-D Messungen. Zunächst stellen wir Multi-Resolutions-Oberflächenelement-Karten (engl. multi-resolution surfel maps, MRSMaps) als eine kompakte Repräsentation von RGB-D Messungen vor, die unseren effizienten Registrierungsmethoden zugrunde liegt. Bilder können effizient in dieser Repräsentation aggregiert werde, wobei auch mehrere Bilder aus verschiedenen Blickpunkten integriert werden können, um Modelle von Szenen und Objekte aus vielfältigen Ansichten darzustellen. Für die effiziente, robuste und genaue Registrierung von MRSMaps wird eine Methode vorgestellt, die Rigidheit der betrachteten Szene voraussetzt. Die Registrierung schätzt die Kamerabewegung zwischen den Bildern und gewinnt ihre Effizienz durch die Ausnutzung der kompakten multi-resolutionalen Darstellung der Karten. Die Registrierungsmethode erzielt hohe Bildverarbeitungsraten auf einer CPU. Wir demonstrieren hohe Effizienz, Genauigkeit und Robustheit unserer Methode im Vergleich zum bisherigen Stand der Forschung auf Vergleichsdatensätzen. In einem weiteren Registrierungsansatz lösen wir uns von der Annahme, dass die betrachtete Szene zwischen Bildern statisch ist. Wir erlauben nun, dass sich rigide Teile der Szene bewegen dürfen, und erweitern unser rigides Registrierungsverfahren auf diesen Fall. Unser Ansatz segmentiert das Bild in Bereiche einzelner Teile, die sich unterschiedlich zwischen Bildern bewegen. Wir demonstrieren hohe Segmentierungsgenauigkeit und Genauigkeit in der Bewegungsschätzung unter Echtzeitbedingungen für die Verarbeitung. Schließlich entwickeln wir ein Verfahren für die Wahrnehmung von nicht-rigiden Deformationen zwischen zwei MRSMaps. Auch hier nutzen wir die multi-resolutionale Struktur in den Karten für ein effizientes Registrieren von grob zu fein. Wir schlagen Methoden vor, um aus den geschätzten Deformationen die lokale Bewegung zwischen den Bildern zu berechnen. Wir evaluieren Genauigkeit und Effizienz des Registrierungsverfahrens. Der zweite Teil dieser Arbeit widmet sich der Verwendung unserer Kartenrepräsentation und Registrierungsmethoden für die Wahrnehmung von Szenen und Objekten. Wir verwenden MRSMaps und unsere rigide Registrierungsmethode, um dichte 3D Modelle von Szenen und Objekten zu lernen. Die räumlichen Beziehungen zwischen Schlüsselansichten, die wir durch Registrierung schätzen, werden in einem Simultanen Lokalisierungs- und Kartierungsverfahren (engl. simultaneous localization and mapping, SLAM) gegeneinander abgewogen, um die Blickposen der Schlüsselansichten zu schätzen. Für das Verfolgen der Kamerapose bezüglich der Modelle in Echtzeit, kombinieren wir die Genauigkeit unserer Registrierung mit der Robustheit von Partikelfiltern. Zu Beginn der Posenverfolgung, oder wenn das Objekt aufgrund von Verdeckungen oder extremen Bewegungen nicht weiter verfolgt werden konnte, initialisieren wir das Filter durch Objektdetektion. Anschließend wenden wir unsere erweiterten Registrierungsverfahren für die Wahrnehmung in nicht-rigiden Szenen und für die Übertragung von Objekthandhabungsfähigkeiten von Robotern an. Wir erweitern unseren rigiden Kartierungsansatz auf dynamische Szenen, in denen sich rigide Teile bewegen. Die Bewegungssegmente in Schlüsselansichten werden zueinander in Bezug gesetzt, um Äquivalenz- und Teilebeziehungen von Objekten probabilistisch zu inferieren, denen die Segmente entsprechen. Auch hier liefert unsere Registrierungsmethode die Bewegung der Kamera bezüglich der Objekte, die wir in einem SLAM Verfahren optimieren. Aus diesen Blickposen wiederum können wir die Bewegungssegmente in dichten Objektmodellen vereinen. Objekte einer Klasse teilen oft eine gemeinsame Topologie von funktionalen Elementen, die durch Formkorrespondenzen ermittelt werden kann. Wir verwenden unsere deformierbare Registrierung, um solche Korrespondenzen zu finden und die Handhabung eines Objektes durch einen Roboter auf neue Objektinstanzen derselben Klasse zu übertragen. Schließlich entwickeln wir einen echtzeitfähigen Ansatz, der Kategorien von Objekten in RGB-D Bildern erkennt und segmentiert. Die Segmentierung basiert auf Ensemblen randomisierter Entscheidungsbäume, die Geometrie- und Texturmerkmale zur Klassifikation verwenden. Wir fusionieren Segmentierungen von Einzelbildern einer Szene aus mehreren Ansichten in einer semantischen Objektklassenkarte mit Hilfe unseres SLAM-Verfahrens. Die vorgestellten Methoden werden auf öffentlich verfügbaren Vergleichsdatensätzen und eigenen Datensätzen evaluiert. Einige unserer Ansätze wurden auch in integrierten Robotersystemen für mobile Objekthantierungsaufgaben öffentlich demonstriert. Sie waren ein wichtiger Bestandteil für das Gewinnen der RoboCup-Roboterwettbewerbe in der RoboCup@Home Liga in den Jahren 2011, 2012 und 2013

Chromatic monitoring of transformer oil condition using CCD camera technology

Power transformers are essential components within the power distribution system. Transformer failures having a high economic impact on the distribution operators and the industrial and domestic customers. Dielectric mineral oil is used in transformers for electrical insulation between live parts, cooling and protection of the insulation papers in the transformer. Oil contamination and changes in the chemical structure of the oil result in the decay of insulation paper and reduced insulation and cooling which can lead to a transformer failure. The general approach to oil monitoring has been for an operator to examine the colour index (ASTM) of the oil, electrical strength, acidity, water contents and dissolved gas analysis results and form an opinion as to the extent of oil degradation. Chromatic techniques enable data from di↵erent sources to be combined to give an overall evaluation about the condition of a system being monitored. One of the main goals for this work was to use chromatic techniques for integrating the oil data from the di↵erent sources and sensors. In addition the chromatic approach enables liquids to be monitored optically so a second aim was to apply chromatic optical oil monitoring using portable system by transmitting polychro- matic light through the oil sample, which is contained in a transparent cuvette and imaged using a mobile phone camera. A number of oil samples were optically analysed with portable chromatic sys- tem and the optical data was compared with the colour index and chromatically companied with the dissolved gas and other oil data to give overall evaluation of oil degradation. The chromatic optical result compared favourably with the colour index. It was also possible to classify the oil samples chromatically into categories of low, medium and high degradation. This enabled the chromatic data combination approach to be implemented as a prototype system in Matlab software that an operator could use to get a classification of an oil sample. Essential experiment was introduced to monitor di↵erent oil particles by obtaining the result of di↵erent filtered samples through the filter paper. Beside the ability to analyse data and distinguish between fresh and contam- inated oil samples the chromatic technique has the ability to track the history of di↵erent degraded oil samples which can give an indication about failure faults and it could give a prediction of any future faults. Therefore a commercially viable reliable system can be developed to extend the service life and extend the maintenance schedules. These monitoring systems could lead to extending the service life of the transformers, making the electricity supply more reliable and giving the consumer a better quality of life

Bildegjenkjenning og Autonom

Bildebehandling og autonom kjøring.Image recognition and autonomous driving

In vitro and in vivo evaluation of antitumor activity of NO-modified compounds, Saquinavir-NO and GIT-27NO in colon cancer

Iz poznate uzročno-posledične veze infekcije i inflamacije, s jedne strane i karcinogeneze i progresije tumora, s druge strane, proistekla je ideja o upotrebi anti-inflamatornih i anti-infektivnih agenasa u terapiji kancera. Međutim, problem gastrotoksičnosti anti-inflamatornih, kao i generalne toksičnosti anti-viralnih lekova uz činjenicu da je reč o agensima niske biološke iskoristljivosti, ozbiljne su barijere u razmatranju njihove primene u terapiji bolesti za koje inicijalno nisu namenjeni. Kako bi se prevazišla farmakološka ograničenja pomenutih jedinjenja, a imajući u vidu poznato svojstvo azot-monoksida (NO) da neutrališe toksičnost supstanci na različitim nivoima, anti-inflamatornom agensu VGX-1027 i anti-retroviralnom agensu sakvinaviru (Saq) adekvatnom hemijskom intervencijom je dodata grupa dizajnirana da donira NO. Prednost novodobijenih jedinjenja GIT-27NO i Saq-NO nad poznatim nesteroidnim anti-inflamatornim agensima modifikovanim kovalentnim vezivanjem NO (NO-NSAID) je odsustvo molekula “nosača”, koji je inače odgovoran za genotoksičnost NO-NSAID. Strukturna promena ova dva jedinjenja doprinela je njihovom snažnom antitumorskom svojstvu, što je potvđeno brojnim in vitro i in vivo studijama, uz umanjenu toksičnost u slučaju GIT-27NO ili potpunom gubitku toksičnosti Saq-NO tretmana.U ovoj studiji je po prvi put ispitano antitumorsko delovanje GIT-27NO i Saq-NO in vitro i in vivo na modelu kancera kolona, jednog od najtežih formi maligniteta. Kako su oba agensa NO-derivati, definisana je uloga NO u njihovom antitumorskom delovanju. Na molekularnom nivou, determinisani su glavni unutarćelijski događaji pokrenuti u odgovoru na tretman pomenutim jedinjenjima. Na kraju, imajući u vidu značaj mikrosredine za rast i progresiju tumora, ispitana je sposobnost GIT-27NO i Saq-NO da povrate osetljivost ćelija kancera kolona na antitumorski imunski odgovor posredovan TRAIL molekulom.Concept of using anti-inflammatory and anti-infectious agents in cancer therapy is based on a known relationship between inflammation and infection on one side and carcinogenesis and tumor progression on the other side. However, the problem of gastrotoxicity of anti-inflammatory and general toxicity of antiviral drugs, along with their low bioavailability, marks a strong barrier when considering them in therapies for which they were not initially intended for. In order to overcome the pharmacological limits of these compounds, nitric-oxide (NO) with its documented feature to neutralize toxicity of drugs on various levels was added to anti-inflammatory agent VGX-1027 and antiretroviral agent saquinavir (Saq). The advantage of new compounds, GIT-27NO and Saq-NO, over known non-steroid anti-inflammatory agents modified by covalent binding NO moiety (NO-NSAID) is the absence of “carrier” molecule, which is responsible for genotoxicity of NO-NSAID. Structural change in these compounds resulted in potentiation of antitumor action, confirmed by numerous in vitro and in vivo studies, along with reduced toxicity of GIT-27NO or complete loss of toxicity of Saq-NO treatment. In this study, antitumor potential of GIT-27NO and Saq-NO was tested in vitro and in vivo for the first time in colon cancer model, one of the most severe forms of malignancy. As both agents are NO-derivates, the role of NO in their antitumor action was defined. On molecular level, main intercellular events triggered by the treatment were determined. Finally, considering the importance of microenvironment for tumor growth and progression, the ability of GIT-27NO and Saq-NO to re-establish colon cancer cell sensitivity to TRAIL-mediated antitumor immune response was tested. GIT-27NO and Saq-NO reduced the viability of mouse CT26CL25 and human HCT116 colon cancer cell lines, both in vitro and in vivo. The importance of NO release for antitumor action was quite different. While cell viability reduction under GIT-27NO treatment was due to accumulation of high intracellular concentration of NO and consecutively generated oxidative and nitrosative stress, antitumor action of Saq-NO was not mediated by a release of quantitatively relevant amount of this free radical. GIT-27NO induced the accumulation of p53 tumor suppressor, changed pro- and antiapoptotic molecule ratio and triggered mitochondrial membrane depolarization which resulted in cell death via caspase-dependent apoptosis