Evaluation of Tracking Confidence Indicators and Feature Extractors on a Visual Tracking Algorithm

For visual tracking, a radial basis function neural network algorithm will be used. Coupled with a feature extraction algorithm, the neural network has advantages for pattern recognition, including practical implementation in parallel hardware for real-time operation and low power requirements. Targets vary in terms of texture, contrast, sharpness of edge, relative speed, and size. Various feature extractors exhibit tradeoffs in terms of sensitivity and processing requirements as related to the characteristics of candidate target classes. An analysis of feature extractors based on the horizontal and vertical profile has been provided. A comparison of the distance traveled computed from vision to wheel encoders is presented to observe slipping. Feedback from the network can offer an indication of tracking confidence which will be useful in determining if the estimated position is correct. An attempt has been made to look at the various confidence factors to determine if the position estimated is correct.Computer Science Departmen

Visually guided obstacle detection and avoidance for legged robot.

Chow Ying-ho.Thesis (M.Phil.)--Chinese University of Hong Kong, 2000.Includes bibliographical references (leaves 78-83).Abstracts in English and Chinese.Chapter Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Objectives - Visual Navigation for Legged Robots --- p.1Chapter 1.2 --- Summary of Results --- p.3Chapter 1.3 --- Hardware Issues --- p.4Chapter 1.4 --- Contributions --- p.4Chapter 1.5 --- Organization of the Thesis --- p.4Chapter Chapter 2 --- Previous Work --- p.6Chapter 2.1 --- Vision Based Navigation --- p.6Chapter 2.1.1 --- Homography --- p.7Chapter 2.1.2 --- Ground Plane Obstacle Detection --- p.9Chapter 2.1.3 --- Regression --- p.12Chapter 2.2 --- Control Strategy --- p.13Chapter Chapter 3 --- System Overview --- p.16Chapter Chapter 4 --- Obstacle Detection by Fast Homography Estimation --- p.20Chapter 4.1 --- Ground Feature Extraction --- p.21Chapter 4.2 --- Ground Feature Correspondence --- p.21Chapter 4.3 --- Ground Homography Estimation --- p.24Chapter 4.3.1 --- Input point transformation --- p.24Chapter 4.3.2 --- Initial estimation --- p.26Chapter 4.3.3 --- Robust estimation --- p.27Chapter 4.4 --- Obstacle Detection --- p.29Chapter 4.5 --- Local Obstacle Map (LOM) on Ground --- p.33Chapter 4.5.1 --- Extraction from accumulative evidence --- p.34Chapter 4.5.2 --- Time-delay compensation --- p.34Chapter Chapter 5 --- Obstacle Avoidance by a Fuzzy Controller --- p.36Chapter 5.1 --- Gait Pattern --- p.38Chapter 5.2 --- Fuzzy Logic Controller --- p.42Chapter 5.2.1 --- Controller Inputs --- p.42Chapter 5.2.2 --- Controller Outputs --- p.43Chapter 5.2.3 --- Inference mechanism --- p.46Chapter Chapter 6 --- Implementation --- p.49Chapter 6.1 --- Hardware components --- p.49Chapter 6.1.1 --- VisionBug --- p.49Chapter 6.1.2 --- RF transmitter / receiver modules: --- p.52Chapter 6.2 --- Perception --- p.55Chapter 6.3 --- Image Calibration --- p.56Chapter 6.4 --- Motion Calibration: --- p.58Chapter 6.5 --- Software Programs --- p.66Chapter 6.5.1 --- Computational complexity --- p.68Chapter Chapter 7 --- Experimental Results --- p.69Chapter 7.1 --- Real Navigation Experiments --- p.70Chapter 7.2 --- Error Analysis of LOM --- p.73Chapter Chapter 8 --- Conclusion and future work --- p.7

FPGA-plattform för bildbehandling

I denna licentiatavhandling i automationsteknik planeras, förverkligas och testas en FPGA-plattform för bildbehandling, som fungerar som en bildbehandlingsserver på Ethernet och Internet. Plattformen kan utföra ett stort antal databehandlingsmetoder och -tillämpningar inom höghastighetskommunikation i realtid. Med hjälp av Altera- och Eclipse-verktygen, Terasics ALTERA DE3-utvecklingskort med Alteras Stratix III FPGA och HSMC-NET- och minneskort och VHDL-, Verilog-, C- och Assembler-programmeringsspråket skapas en 1 Gbps FPGA-plattform för bildbehandling. Vidare behandlas till lösningen hörande begrepp, som en FPGA-plattform som ett inbyggt system, orsak till val av FPGA-hårdvara och förväntningarna på ett utvecklingskort. Viktiga verktyg, hjälpmedel och komponenter vid konstruktion av en bildbehandlingsplattform samt möjligheter för pipeline och parallellism kartläggs. Konstruktions- och implementeringsmetoder vid planering och konstruktion av hårdvara och mjukvara presenteras speciellt gränssnitt mellan hårdvara och mjukvara och deras verktygs roller i ett HW/SW Co-Design-system. Implementering av hårdvara och mjukvara, hårdvaran, moder-, dotter- och minneskortet med sammankopplingar och implementerade funktioner beskrivs. Mjukvaran beskrivs med implementerade mjukvarufunktionsgrupper såsom system start-up-, operativsystem-, bildbehandlings- och avbrottsrutiner. Det utfördes manuella och prestandatester med plattformen. De manuella TCP- och UDP-testerna visar att alla kommandon och operationer fungerar korrekt i alla lager och på alla nivåer. Prestandatesterna visar att plattformen kan hantera både låg- och högbelastande TCP- och UDP-trafik med stigande och sjunkande längd på testdata. Alla tester visar samma struktur och trend för genomströmning. Maximigenomströmningen för plattformen är ca 7,5 Mbps med en Nios II/f-processor och arbetsfrekvens på 50 MHz. Mitt bidrag har varit att bygga en mer omfattande funktionell mjukvara med hjälp av basprogramvara samt att bygga en omfattande funktionell hårdvara i IPS. Dessutom att bygga en omfattande funktionell testprogramvara för PC – alla med nödvändiga funktioner och komponenter.fi=Opinnäytetyö kokotekstinä PDF-muodossa.|en=Thesis fulltext in PDF format.|sv=Lärdomsprov tillgängligt som fulltext i PDF-format

THREE-DIMENSIONAL VISION FOR STRUCTURE AND MOTION ESTIMATION

1997/1998Questa tesi, intitolata Visione Tridimensionale per la stima di Struttura e Moto, tratta di tecniche di Visione Artificiale per la stima delle proprietà geometriche del mondo tridimensionale a partire da immagini numeriche. Queste proprietà sono essenziali per il riconoscimento e la classificazione di oggetti, la navigazione di veicoli mobili autonomi, il reverse engineering e la sintesi di ambienti virtuali. In particolare, saranno descritti i moduli coinvolti nel calcolo della struttura della scena a partire dalle immagini, e verranno presentati contributi originali nei seguenti campi. Rettificazione di immagini steroscopiche. Viene presentato un nuovo algoritmo per la rettificazione, il quale trasforma una coppia di immagini stereoscopiche in maniera che punti corrispondenti giacciano su linee orizzontali con lo stesso indice. Prove sperimentali dimostrano il corretto comportamento del metodo, come pure la trascurabile perdita di accuratezza nella ricostruzione tridimensionale quando questa sia ottenuta direttamente dalle immagini rettificate. Calcolo delle corrispondenze in immagini stereoscopiche. Viene analizzato il problema della stereovisione e viene presentato un un nuovo ed efficiente algoritmo per l'identificazione di coppie di punti corrispondenti, capace di calcolare in modo robusto la disparità stereoscopica anche in presenza di occlusioni. L'algoritmo, chiamato SMW, usa uno schema multi-finestra adattativo assieme al controllo di coerenza destra-sinistra per calcolare la disparità e l'incertezza associata. Gli esperimenti condotti con immagini sintetiche e reali mostrano che SMW sortisce un miglioramento in accuratezza ed efficienza rispetto a metodi simili Inseguimento di punti salienti. L'inseguitore di punti salienti di Shi-Tomasi- Kanade viene migliorato introducendo uno schema automatico per lo scarto di punti spuri basato sulla diagnostica robusta dei campioni periferici ( outliers ). Gli esperimenti con immagini sintetiche e reali confermano il miglioramento rispetto al metodo originale, sia qualitativamente che quantitativamente. Ricostruzione non calibrata. Viene presentata una rassegna ragionata dei metodi per la ricostruzione di un modello tridimensionale della scena, a partire da una telecamera che si muove liberamente e di cui non sono noti i parametri interni. Il contributo consiste nel fornire una visione critica e unificata delle più recenti tecniche. Una tale rassegna non esiste ancora in letterarura. Moto tridimensionale. Viene proposto un algoritmo robusto per registrate e calcolare le corrispondenze in due insiemi di punti tridimensionali nei quali vi sia un numero significativo di elementi mancanti. Il metodo, chiamato RICP, sfrutta la stima robusta con la Minima Mediana dei Quadrati per eliminare l'effetto dei campioni periferici. Il confronto sperimentale con una tecnica simile, ICP, mostra la superiore robustezza e affidabilità di RICP.This thesis addresses computer vision techniques estimating geometrie properties of the 3-D world /rom digital images. Such properties are essential for object recognition and classification, mobile robots navigation, reverse engineering and synthesis of virtual environments. In particular, this thesis describes the modules involved in the computation of the structure of a scene given some images, and offers original contributions in the following fields. Stereo pairs rectification. A novel rectification algorithm is presented, which transform a stereo pair in such a way that corresponding points in the two images lie on horizontal lines with the same index. Experimental tests prove the correct behavior of the method, as well as the negligible decrease oLthe accuracy of 3-D reconstruction if performed from the rectified images directly. Stereo matching. The problem of computational stereopsis is analyzed, and a new, efficient stereo matching algorithm addressing robust disparity estimation in the presence of occlusions is presented. The algorithm, called SMW, is an adaptive, multi-window scheme using left-right consistency to compute disparity and its associated uncertainty. Experiments with both synthetic and real stereo pairs show how SMW improves on closely related techniques for both accuracy and efficiency. Features tracking. The Shi-Tomasi-Kanade feature tracker is improved by introducing an automatic scheme for rejecting spurious features, based on robust outlier diagnostics. Experiments with real and synthetic images confirm the improvement over the original tracker, both qualitatively and quantitatively. 111 Uncalibrated vision. A review on techniques for computing a three-dimensional model of a scene from a single moving camera, with unconstrained motion and unknown parameters is presented. The contribution is to give a critical, unified view of some of the most promising techniques. Such review does not yet exist in the literature. 3-D motion. A robust algorithm for registering and finding correspondences in two sets of 3-D points with significant percentages of missing data is proposed. The method, called RICP, exploits LMedS robust estimation to withstand the effect of outliers. Experimental comparison with a closely related technique, ICP, shows RICP's superior robustness and reliability.XI Ciclo1968Versione digitalizzata della tesi di dottorato cartacea

Pervasive handheld computing systems

The technological role of handheld devices is fundamentally changing. Portable computers were traditionally application specific. They were designed and optimised to deliver a specific task. However, it is now commonly acknowledged that future handheld devices need to be multi-functional and need to be capable of executing a range of high-performance applications. This thesis has coined the term pervasive handheld computing systems to refer to this type of mobile device. Portable computers are faced with a number of constraints in trying to meet these objectives. They are physically constrained by their size, their computational power, their memory resources, their power usage, and their networking ability. These constraints challenge pervasive handheld computing systems in achieving their multi-functional and high-performance requirements. This thesis proposes a two-pronged methodology to enable pervasive handheld computing systems meet their future objectives. The methodology is a fusion of two independent and yet complementary concepts. The first step utilises reconfigurable technology to enhance the physical hardware resources within the environment of a handheld device. This approach recognises that reconfigurable computing has the potential to dynamically increase the system functionality and versatility of a handheld device without major loss in performance. The second step of the methodology incorporates agent-based middleware protocols to support handheld devices to effectively manage and utilise these reconfigurable hardware resources within their environment. The thesis asserts the combined characteristics of reconfigurable computing and agent technology can meet the objectives of pervasive handheld computing systems

Real-time 2-D feature detection on a reconfigurable computer

We have designed and implemented a system for real-time detection of 2-D features on a reconfigurable computer based on Field Programmable Gate Arrays (FPGA's). We envision this device as the front-end of a system able to track image features in real-time control applications like autonomous vehicle navigation. The algorithm employed to select good features is inspired by Tomasi and Kanade's method. Compared to the original method, the algorithm that we have devised does not require any floating point or transcendental operations, and can be implemented either in hardware or in software. Moreover, it maps efficiently into a highly pipelined architecture, well suited to implementation in FPGA technology. We have implemented the algorithm on a low-cost reconfigurable computer and have observed reliable operation on an image stream generated by a standard NTSC video camera at 30 Hz

Autonomous vision-based terrain-relative navigation for planetary exploration

Abstract: The interest of major space agencies in the world for vision sensors in their mission designs has been increasing over the years. Indeed, cameras offer an efficient solution to address the ever-increasing requirements in performance. In addition, these sensors are multipurpose, lightweight, proven and a low-cost technology. Several researchers in vision sensing for space application currently focuse on the navigation system for autonomous pin-point planetary landing and for sample and return missions to small bodies. In fact, without a Global Positioning System (GPS) or radio beacon around celestial bodies, high-accuracy navigation around them is a complex task. Most of the navigation systems are based only on accurate initialization of the states and on the integration of the acceleration and the angular rate measurements from an Inertial Measurement Unit (IMU). This strategy can track very accurately sudden motions of short duration, but their estimate diverges in time and leads normally to high landing error. In order to improve navigation accuracy, many authors have proposed to fuse those IMU measurements with vision measurements using state estimators, such as Kalman filters. The first proposed vision-based navigation approach relies on feature tracking between sequences of images taken in real time during orbiting and/or landing operations. In that case, image features are image pixels that have a high probability of being recognized between images taken from different camera locations. By detecting and tracking these features through a sequence of images, the relative motion of the spacecraft can be determined. This technique, referred to as Terrain-Relative Relative Navigation (TRRN), relies on relatively simple, robust and well-developed image processing techniques. It allows the determination of the relative motion (velocity) of the spacecraft. Despite the fact that this technology has been demonstrated with space qualified hardware, its gain in accuracy remains limited since the spacecraft absolute position is not observable from the vision measurements. The vision-based navigation techniques currently studied consist in identifying features and in mapping them into an on-board cartographic database indexed by an absolute coordinate system, thereby providing absolute position determination. This technique, referred to as Terrain-Relative Absolute Navigation (TRAN), relies on very complex Image Processing Software (IPS) having an obvious lack of robustness. In fact, these software depend often on the spacecraft attitude and position, they are sensitive to illumination conditions (the elevation and azimuth of the Sun when the geo-referenced database is built must be similar to the ones present during mission), they are greatly influenced by the image noise and finally they hardly manage multiple varieties of terrain seen during the same mission (the spacecraft can fly over plain zone as well as mountainous regions, the images may contain old craters with noisy rims as well as young crater with clean rims and so on). At this moment, no real-time hardware-in-the-loop experiment has been conducted to demonstrate the applicability of this technology to space mission. The main objective of the current study is to develop autonomous vision-based navigation algorithms that provide absolute position and surface-relative velocity during the proximity operations of a planetary mission (orbiting phase and landing phase) using a combined approach of TRRN and TRAN technologies. The contributions of the study are: (1) reference mission definition, (2) advancements in the TRAN theory (image processing as well as state estimation) and (3) practical implementation of vision-based navigation.Résumé: L’intérêt des principales agences spatiales envers les technologies basées sur la vision artificielle ne cesse de croître. En effet, les caméras offrent une solution efficace pour répondre aux exigences de performance, toujours plus élevées, des missions spatiales. De surcroît, ces capteurs sont multi-usages, légers, éprouvés et peu coûteux. Plusieurs chercheurs dans le domaine de la vision artificielle se concentrent actuellement sur les systèmes autonomes pour l’atterrissage de précision sur des planètes et sur les missions d’échantillonnage sur des astéroïdes. En effet, sans système de positionnement global « Global Positioning System (GPS) » ou de balises radio autour de ces corps célestes, la navigation de précision est une tâche très complexe. La plupart des systèmes de navigation sont basés seulement sur l’intégration des mesures provenant d’une centrale inertielle. Cette stratégie peut être utilisée pour suivre les mouvements du véhicule spatial seulement sur une courte durée, car les données estimées divergent rapidement. Dans le but d’améliorer la précision de la navigation, plusieurs auteurs ont proposé de fusionner les mesures provenant de la centrale inertielle avec des mesures d’images du terrain. Les premiers algorithmes de navigation utilisant l’imagerie du terrain qui ont été proposés reposent sur l’extraction et le suivi de traits caractéristiques dans une séquence d’images prises en temps réel pendant les phases d’orbite et/ou d’atterrissage de la mission. Dans ce cas, les traits caractéristiques de l’image correspondent à des pixels ayant une forte probabilité d’être reconnus entre des images prises avec différentes positions de caméra. En détectant et en suivant ces traits caractéristiques, le déplacement relatif du véhicule (la vitesse) peut être déterminé. Ces techniques, nommées navigation relative, utilisent des algorithmes de traitement d’images robustes, faciles à implémenter et bien développés. Bien que cette technologie a été éprouvée sur du matériel de qualité spatiale, le gain en précision demeure limité étant donné que la position absolue du véhicule n’est pas observable dans les mesures extraites de l’image. Les techniques de navigation basées sur la vision artificielle actuellement étudiées consistent à identifier des traits caractéristiques dans l’image pour les apparier avec ceux contenus dans une base de données géo-référencées de manière à fournir une mesure de position absolue au filtre de navigation. Cependant, cette technique, nommée navigation absolue, implique l’utilisation d’algorithmes de traitement d’images très complexes souffrant pour le moment des problèmes de robustesse. En effet, ces algorithmes dépendent souvent de la position et de l’attitude du véhicule. Ils sont très sensibles aux conditions d’illuminations (l’élévation et l’azimut du Soleil présents lorsque la base de données géo-référencée est construite doit être similaire à ceux observés pendant la mission). Ils sont grandement influencés par le bruit dans l’image et enfin ils supportent mal les multiples variétés de terrain rencontrées pendant la même mission (le véhicule peut survoler autant des zones de plaine que des régions montagneuses, les images peuvent contenir des vieux cratères avec des contours flous aussi bien que des cratères jeunes avec des contours bien définis, etc.). De plus, actuellement, aucune expérimentation en temps réel et sur du matériel de qualité spatiale n’a été réalisée pour démontrer l’applicabilité de cette technologie pour les missions spatiales. Par conséquent, l’objectif principal de ce projet de recherche est de développer un système de navigation autonome par imagerie du terrain qui fournit la position absolue et la vitesse relative au terrain d’un véhicule spatial pendant les opérations à basse altitude sur une planète. Les contributions de ce travail sont : (1) la définition d’une mission de référence, (2) l’avancement de la théorie de la navigation par imagerie du terrain (algorithmes de traitement d’images et estimation d’états) et (3) implémentation pratique de cette technologie