The characterisation and simulation of 3D vision sensors for measurement optimisation

The use of 3D Vision is becoming increasingly common in a range of industrial applications including part identification, reverse engineering, quality control and inspection. To facilitate this increased usage, especially in autonomous applications such as free-form assembly and robotic metrology, the capability to deploy a sensor to the optimum pose for a measurement task is essential to reduce cycle times and increase measurement quality. Doing so requires knowledge of the 3D sensor capabilities on a material specific basis, as the optical properties of a surface, object shape, pose and even the measurement itself have severe implications for the data quality. This need is not reflected in the current state of sensor haracterisation standards which commonly utilise optically compliant artefacts and therefore can not inform the user of a 3D sensor the realistic expected performance on non-ideal objects.This thesis presents a method of scoring candidate viewpoints for their ability to perform geometric measurements on an object of arbitrary surface finish. This is achieved by first defining a technology independent, empirical sensor characterisation method which implements a novel variant of the commonly used point density point cloud quality metric, which is normalised to isolate the effect of surface finish on sensor performance, as well as the more conventional assessment of point standard deviation. The characterisation method generates a set of performance maps for a sensor per material which are a function of distance and surface orientation. A sensor simulation incorporates these performance maps to estimate the statistical properties of a point cloud on objects with arbitrary shape and surface finish, providing the sensor has been characterised on the material in question.A framework for scoring measurement specific candidate viewpoints is presented in the context of the geometric inspection of four artefacts with different surface finish but identical geometry. Views are scored on their ability to perform each measurement based on a novel view score metric, which incorporates the expected point density, noise and occlusion of measurement dependent model features. The simulation is able to score the views reliably on all four surface finishes tested, which range from ideal matt white to highly polished aluminium. In 93% of measurements, a set of optimal or nearly optimal views is correctly selected.</div

Content based image pose manipulation

This thesis proposes the application of space-frequency transformations to the domain of pose estimation in images. This idea is explored using the Wavelet Transform with illustrative applications in pose estimation for face images, and images of planar scenes. The approach is based on examining the spatial frequency components in an image, to allow the inherent scene symmetry balance to be recovered. For face images with restricted pose variation (looking left or right), an algorithm is proposed to maximise this symmetry in order to transform the image into a fronto-parallel pose. This scheme is further employed to identify the optimal frontal facial pose from a video sequence to automate facial capture processes. These features are an important pre-requisite in facial recognition and expression classification systems. The under lying principles of this spatial-frequency approach are examined with respect to images with planar scenes. Using the Continuous Wavelet Transform, full perspective planar transformations are estimated within a featureless framework. Restoring central symmetry to the wavelet transformed images in an iterative optimisation scheme removes this perspective pose. This advances upon existing spatial approaches that require segmentation and feature matching, and frequency only techniques that are limited to affine transformation recovery. To evaluate the proposed techniques, the pose of a database of subjects portraying varying yaw orientations is estimated and the accuracy is measured against the captured ground truth information. Additionally, full perspective homographies for synthesised and imaged textured planes are estimated. Experimental results are presented for both situations that compare favourably with existing techniques in the literature

Map-Based Localization for Unmanned Aerial Vehicle Navigation

Unmanned Aerial Vehicles (UAVs) require precise pose estimation when navigating in indoor and GNSS-denied / GNSS-degraded outdoor environments. The possibility of crashing in these environments is high, as spaces are confined, with many moving obstacles. There are many solutions for localization in GNSS-denied environments, and many different technologies are used. Common solutions involve setting up or using existing infrastructure, such as beacons, Wi-Fi, or surveyed targets. These solutions were avoided because the cost should be proportional to the number of users, not the coverage area. Heavy and expensive sensors, for example a high-end IMU, were also avoided. Given these requirements, a camera-based localization solution was selected for the sensor pose estimation. Several camera-based localization approaches were investigated. Map-based localization methods were shown to be the most efficient because they close loops using a pre-existing map, thus the amount of data and the amount of time spent collecting data are reduced as there is no need to re-observe the same areas multiple times. This dissertation proposes a solution to address the task of fully localizing a monocular camera onboard a UAV with respect to a known environment (i.e., it is assumed that a 3D model of the environment is available) for the purpose of navigation for UAVs in structured environments. Incremental map-based localization involves tracking a map through an image sequence. When the map is a 3D model, this task is referred to as model-based tracking. A by-product of the tracker is the relative 3D pose (position and orientation) between the camera and the object being tracked. State-of-the-art solutions advocate that tracking geometry is more robust than tracking image texture because edges are more invariant to changes in object appearance and lighting. However, model-based trackers have been limited to tracking small simple objects in small environments. An assessment was performed in tracking larger, more complex building models, in larger environments. A state-of-the art model-based tracker called ViSP (Visual Servoing Platform) was applied in tracking outdoor and indoor buildings using a UAVs low-cost camera. The assessment revealed weaknesses at large scales. Specifically, ViSP failed when tracking was lost, and needed to be manually re-initialized. Failure occurred when there was a lack of model features in the cameras field of view, and because of rapid camera motion. Experiments revealed that ViSP achieved positional accuracies similar to single point positioning solutions obtained from single-frequency (L1) GPS observations standard deviations around 10 metres. These errors were considered to be large, considering the geometric accuracy of the 3D model used in the experiments was 10 to 40 cm. The first contribution of this dissertation proposes to increase the performance of the localization system by combining ViSP with map-building incremental localization, also referred to as simultaneous localization and mapping (SLAM). Experimental results in both indoor and outdoor environments show sub-metre positional accuracies were achieved, while reducing the number of tracking losses throughout the image sequence. It is shown that by integrating model-based tracking with SLAM, not only does SLAM improve model tracking performance, but the model-based tracker alleviates the computational expense of SLAMs loop closing procedure to improve runtime performance. Experiments also revealed that ViSP was unable to handle occlusions when a complete 3D building model was used, resulting in large errors in its pose estimates. The second contribution of this dissertation is a novel map-based incremental localization algorithm that improves tracking performance, and increases pose estimation accuracies from ViSP. The novelty of this algorithm is the implementation of an efficient matching process that identifies corresponding linear features from the UAVs RGB image data and a large, complex, and untextured 3D model. The proposed model-based tracker improved positional accuracies from 10 m (obtained with ViSP) to 46 cm in outdoor environments, and improved from an unattainable result using VISP to 2 cm positional accuracies in large indoor environments. The main disadvantage of any incremental algorithm is that it requires the camera pose of the first frame. Initialization is often a manual process. The third contribution of this dissertation is a map-based absolute localization algorithm that automatically estimates the camera pose when no prior pose information is available. The method benefits from vertical line matching to accomplish a registration procedure of the reference model views with a set of initial input images via geometric hashing. Results demonstrate that sub-metre positional accuracies were achieved and a proposed enhancement of conventional geometric hashing produced more correct matches - 75% of the correct matches were identified, compared to 11%. Further the number of incorrect matches was reduced by 80%

Automatic aircraft recognition and identification

Aircraft recognition remains a challenging problem despite a great deal of effort to automate the recognition process. The majority of the aircraft recognition methods assume the successful isolation of the aircraft silhouette from the background, and only a few have actually addressed real world concerns, such as occlusion, clutter and shadows. This thesis presents an automatic aircraft recognition system, which shows improved performance with complex images. This system assumes from the start that the image could possibly be degraded, contain occlusions, clutter, camouflage, shadows and blurring. It is designed to tolerate and overcome the degradations at various analysis stages. The first part of the thesis focuses on the generic aircraft recognition problem using a generic description of aircraft parts and the geometric relationships that exist among them. The system implements line groupings in a hierarchical fashion, progressively leading towards a generic aircraft structure. A voting scheme is used to consolidate line groupings belonging to an aircraft while discouraging the formation of spurious line groupings. The aircraft identification process is carried out in the second part of the thesis, where the generically recognised aircraft is matched to model candidates. Model matching is carried out via pixellevel silhouette boundary matching. The system is tested on numerous real aircraft, scaled-down model aircraft and non-aircraft images with adverse image conditions. The developed system achieves a recognition rate of 84% at a false alarm rate of 7% on real aircraft images, and an correct matching rate of about 90% and a false matching rate of 7% on the generically recognised aircraft from model aircraft images

Automated shape analysis and visualization of the human back.

Spinal and back deformities can lead to pain and discomfort, disrupting productivity, and may require prolonged treatment. The conventional method of assessing and monitoring tile de-formity using radiographs has known radiation hazards. An alternative approach for monitoring the deformity is to base the assessment on the shape of back surface. Though three-dimensional data acquisition methods exist, techniques to extract relevant information for clinical use have not been widely developed. Thi's thesis presentsthe content and progression of research into automated analysis and visu-alization of three-dimensional laser scans of the human back. Using mathematical shape analysis, methods have been developed to compute stable curvature of the back surface and to detect the anatomic landmarks from the curvature maps. Compared with manual palpation, the landmarks have been detected to within accuracy of 1.15mm and precision of 0.8111m.Based on the detected spinous process landmarks, the back midline which is the closest surface approximation of the spine, has been derived using constrained polynomial fitting and statistical techniques. Three-dimensional geometric measurementsbasedon the midline were then corn-puted to quantify the deformity. Visualization plays a crucial role in back shape analysis since it enables the exploration of back deformities without the need for physical manipulation of the subject. In the third phase,various visualization techniques have been developed, namely, continuous and discrete colour maps, contour maps and three-dimensional views. In the last phase of the research,a software system has been developed for automating the tasks involved in analysing, visualizing and quantifying of the back shape. The novel aspectsof this research lie in the development of effective noise smoothing methods for stable curvature computation; improved shape analysis and landmark detection algorithm; effective techniques for visualizing the shape of the back; derivation of the back midline using constrained polynomials and computation of three dimensional surface measurements.

Progress in industrial photogrammetry by means of markerless solutions

174 p.La siguiente tesis está enfocada al desarrollo y uso avanzado de metodologías fotogramétrica sin dianas en aplicaciones industriales. La fotogrametría es una técnica de medición óptica 3D que engloba múltiples configuraciones y aproximaciones. En este estudio se han desarrollado procedimientos de medición, modelos y estrategias de procesamiento de imagen que van más allá que la fotogrametría convencional y buscan el emplear soluciones de otros campos de la visión artificial en aplicaciones industriales. Mientras que la fotogrametría industrial requiere emplear dianas artificiales para definir los puntos o elementos de interés, esta tesis contempla la reducción e incluso la eliminación de las dianas tanto pasivas como activas como alternativas prácticas. La mayoría de los sistemas de medida utilizan las dianas tanto para definir los puntos de control, relacionar las distintas perspectivas, obtener precisión, así como para automatizar las medidas. Aunque en muchas situaciones el empleo de dianas no sea restrictivo existen aplicaciones industriales donde su empleo condiciona y restringe considerablemente los procedimientos de medida empleados en la inspección. Un claro ejemplo es la verificación y control de calidad de piezas seriadas, o la medición y seguimiento de elementos prismáticos relacionados con un sistema de referencia determinado. Es en este punto donde la fotogrametría sin dianas puede combinarse o complementarse con soluciones tradicionales para tratar de mejorar las prestaciones actuales

Large-Scale Textured 3D Scene Reconstruction

Die Erstellung dreidimensionaler Umgebungsmodelle ist eine fundamentale Aufgabe im Bereich des maschinellen Sehens. Rekonstruktionen sind für eine Reihe von Anwendungen von Nutzen, wie bei der Vermessung, dem Erhalt von Kulturgütern oder der Erstellung virtueller Welten in der Unterhaltungsindustrie. Im Bereich des automatischen Fahrens helfen sie bei der Bewältigung einer Vielzahl an Herausforderungen. Dazu gehören Lokalisierung, das Annotieren großer Datensätze oder die vollautomatische Erstellung von Simulationsszenarien. Die Herausforderung bei der 3D Rekonstruktion ist die gemeinsame Schätzung von Sensorposen und einem Umgebunsmodell. Redundante und potenziell fehlerbehaftete Messungen verschiedener Sensoren müssen in eine gemeinsame Repräsentation der Welt integriert werden, um ein metrisch und photometrisch korrektes Modell zu erhalten. Gleichzeitig muss die Methode effizient Ressourcen nutzen, um Laufzeiten zu erreichen, welche die praktische Nutzung ermöglichen. In dieser Arbeit stellen wir ein Verfahren zur Rekonstruktion vor, das fähig ist, photorealistische 3D Rekonstruktionen großer Areale zu erstellen, die sich über mehrere Kilometer erstrecken. Entfernungsmessungen aus Laserscannern und Stereokamerasystemen werden zusammen mit Hilfe eines volumetrischen Rekonstruktionsverfahrens fusioniert. Ringschlüsse werden erkannt und als zusätzliche Bedingungen eingebracht, um eine global konsistente Karte zu erhalten. Das resultierende Gitternetz wird aus Kamerabildern texturiert, wobei die einzelnen Beobachtungen mit ihrer Güte gewichtet werden. Für eine nahtlose Erscheinung werden die unbekannten Belichtungszeiten und Parameter des optischen Systems mitgeschätzt und die Bilder entsprechend korrigiert. Wir evaluieren unsere Methode auf synthetischen Daten, realen Sensordaten unseres Versuchsfahrzeugs und öffentlich verfügbaren Datensätzen. Wir zeigen qualitative Ergebnisse großer innerstädtischer Bereiche, sowie quantitative Auswertungen der Fahrzeugtrajektorie und der Rekonstruktionsqualität. Zuletzt präsentieren wir mehrere Anwendungen und zeigen somit den Nutzen unserer Methode für Anwendungen im Bereich des automatischen Fahrens