Lens distortion correction by analysing the shape of patterns in Hough transform space : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Electronics and Computer Engineering at Massey University, Manawatu, New Zealand

Many low cost, wide angle lenses suffer from lens distortion, resulting from a radial variation in the lens magnification. As a result, straight lines, particularly those in the periphery, appear curved. The Hough transform is a commonly used linear feature detection technique within an image. In Hough transform space, straight lines and curved lines have different shapes of peaks. This thesis proposes a lens distortion correction method named SLDC based on analysing the shape of patterns in the Hough transform space. It works by reconstructing the distorted line from significant points on the smile-shaped Hough pattern. It then optimises the distortion parameter by mapping the reconstructed curved line into a straight line and minimising the RMSE. From both simulation and correcting real world images, the SLDC provides encouraging results

Camera calibration in sport event scenarios

The main goal of this paper is the design of a novel and robust methodology for calibrating cameras from a single image in sport scenarios, such as a soccer field, or a basketball or tennis court. In these sport scenarios, the only references we use to calibrate the camera are the lines and circles delimiting the different regions. The first problem we address is the extraction of image primitives, including the challenging problems of shaded regions and lens distortion. From these primitives, we automatically recognise the location of the sport court in the scene by estimating the homography which matches the actual court with its projection onto the image. This is achieved even when only a few primitives are available. Finally, from this homography, we recover the camera calibration parameters. In particular, we estimate the focal length as well as the position and orientation in the 3D space. We present some experiments on models and real courts which illustrate the accuracy of the proposed methodology

Optical Coherence Tomography guided Laser-Cochleostomy

Despite the high precision of laser, it remains challenging to control the laser-bone ablation without injuring the underlying critical structures. Providing an axial resolution on micrometre scale, OCT is a promising candidate for imaging microstructures beneath the bone surface and monitoring the ablation process. In this work, a bridge connecting these two technologies is established. A closed-loop control of laser-bone ablation under the monitoring with OCT has been successfully realised

Image analysis of sports events

Tese de mestrado integrado. Engenharia Electrotécnica e de Computadores (Major de Automação). Faculdade de Engenharia. Universidade do Porto. 200

Optical measurement of shape and deformation fields on challenging surfaces

A multiple-sensor optical shape measurement system (SMS) based on the principle of white-light fringe projection has been developed and commercialised by Loughborough University and Phase Vision Ltd for over 10 years. The use of the temporal phase unwrapping technique allows precise and dense shape measurements of complex surfaces; and the photogrammetry-based calibration technique offers the ability to calibrate multiple sensors simultaneously in order to achieve 360° measurement coverage. Nevertheless, to enhance the applicability of the SMS in industrial environments, further developments are needed (i) to improve the calibration speed for quicker deployment, (ii) to broaden the application range from shape measurement to deformation field measurement, and (iii) to tackle practically-challenging surfaces of which specular components may disrupt the acquired data and result in spurious measurements. The calibration process typically requires manual positioning of an artefact (i.e., reference object) at many locations within the view of the sensors. This is not only timeconsuming but also complicated for an operator with average knowledge of metrology. This thesis introduces an automated artefact positioning system which enables automatic and optimised distribution of the artefacts, automatic prediction of their whereabouts to increase the artefact detection speed and robustness, and thereby greater overall calibration performance. This thesis also describes a novel technique that integrates the digital image correlation (DIC) technique into the present fringe projection SMS for the purpose of simultaneous shape and deformation field measurement. This combined technique offers three key advantages: (a) the ability to deal with geometrical discontinuities which are commonly present on mechanical surfaces and currently challenging to most deformation measurement methods, (b) the ability to measure 3D displacement fields with a basic single-camera single-projector SMS with no additional hardware components, and (c) the simple implementation on a multiple-sensor hardware platform to achieve complete coverage of large-scale and complex samples, with the resulting displacement fields automatically lying in a single global coordinate system. A displacement measurement iii accuracy of ≅1/12,000 of the measurement volume, which is comparable to that of an industry-standard DIC system, has been achieved. The applications of this novel technique to several structural tests of aircraft wing panels on-site at the research centre of Airbus UK in Filton are also presented. Mechanical components with shiny surface finish and complex geometry may introduce another challenge to present fringe projection techniques. In certain circumstances, multiple reflections of the projected fringes on an object surface may cause ambiguity in the phase estimation process and result in incorrect coordinate measurements. This thesis presents a new technique which adopts a Fourier domain ranging (FDR) method to correctly identifying multiple phase signals and enables unambiguous triangulation for a measured coordinate. Experiments of the new FDR technique on various types of surfaces have shown promising results as compared to the traditional phase unwrapping techniques

Real Time Stereo Cameras System Calibration Tool and Attitude and Pose Computation with Low Cost Cameras

The Engineering in autonomous systems has many strands. The area in which this work falls, the artificial vision, has become one of great interest in multiple contexts and focuses on robotics. This work seeks to address and overcome some real difficulties encountered when developing technologies with artificial vision systems which are, the calibration process and pose computation of robots in real-time. Initially, it aims to perform real-time camera intrinsic (3.2.1) and extrinsic (3.3) stereo camera systems calibration needed to the main goal of this work, the real-time pose (position and orientation) computation of an active coloured target with stereo vision systems. Designed to be intuitive, easy-to-use and able to run under real-time applications, this work was developed for use either with low-cost and easy-to-acquire or more complex and high resolution stereo vision systems in order to compute all the parameters inherent to this same system such as the intrinsic values of each one of the cameras and the extrinsic matrices computation between both cameras. More oriented towards the underwater environments, which are very dynamic and computationally more complex due to its particularities such as light reflections. The available calibration information, whether generated by this tool or loaded configurations from other tools allows, in a simplistic way, to proceed to the calibration of an environment colorspace and the detection parameters of a specific target with active visual markers (4.1.1), useful within unstructured environments. With a calibrated system and environment, it is possible to detect and compute, in real time, the pose of a target of interest. The combination of position and orientation or attitude is referred as the pose of an object. For performance analysis and quality of the information obtained, this tools are compared with others already existent.A engenharia de sistemas autónomos actua em diversas vertentes. Uma delas, a visão artificial, em que este trabalho assenta, tornou-se uma das de maior interesse em múltiplos contextos e focos na robótica. Assim, este trabalho procura abordar e superar algumas dificuldades encontradas aquando do desenvolvimento de tecnologias baseadas na visão artificial. Inicialmente, propõe-se a fornecer ferramentas para realizar as calibrações necessárias de intrínsecos (3.2.1) e extrínsecos (3.3) de sistemas de visão stereo em tempo real para atingir o objectivo principal, uma ferramenta de cálculo da posição e orientação de um alvo activo e colorido através de sistemas de visão stereo. Desenhadas para serem intuitivas, fáceis de utilizar e capazes de operar em tempo real, estas ferramentas foram desenvolvidas tendo em vista a sua integração quer com camaras de baixo custo e aquisição fácil como com camaras mais complexas e de maior resolução. Propõem-se a realizar a calibração dos parâmetros inerentes ao sistema de visão stereo como os intrínsecos de cada uma das camaras e as matrizes de extrínsecos que relacionam ambas as camaras. Este trabalho foi orientado para utilização em meio subaquático onde se presenciam ambientes com elevada dinâmica visual e maior complexidade computacional devido `a suas particularidades como reflexões de luz e má visibilidade. Com a informação de calibração disponível, quer gerada pelas ferramentas fornecidas, quer obtida a partir de outras, pode ser carregada para proceder a uma calibração simplista do espaço de cor e dos parâmetros de deteção de um alvo específico com marcadores ativos coloridos (4.1.1). Estes marcadores são ´uteis em ambientes não estruturados. Para análise da performance e qualidade da informação obtida, as ferramentas de calibração e cálculo de pose (posição e orientação), serão comparadas com outras já existentes

A cost-effective, mobile platform-based, photogrammetric approach for continuous structural deformation monitoring

PhD ThesisWith the evolution of construction techniques and materials technology, the design of modern civil engineering infrastructure has become increasingly advanced and complex. In parallel to this, the development and application of appropriate and efficient monitoring technologies has become essential. Improvement in the performance of structural monitoring systems, reduction of labour and total implementation costs have therefore become important issues that scientists and engineers are committed to solving. In this research, a non-intrusive structural monitoring system was developed based on close-range photogrammetric principles. This research aimed to combine the merits of photogrammetry and latest mobile phone technology to propose a cost-effective, compact (portable) and precise solution for structural monitoring applications. By combining the use of low-cost imaging devices (two or more mobile phone handsets) with in-house control software, a monitoring project can be undertaken within a relatively low budget when compared to conventional methods. The system uses programmable smart phones (Google Android v.2.2 OS) to replace conventional in-situ photogrammetric imaging stations. The developed software suite is able to control multiple handsets to continuously capture high-quality, synchronized image sequences for short or long-term structural monitoring purposes. The operations are fully automatic and the system can be remotely controlled, exempting the operator from having to attend the site, and thus saving considerable labour expense in long-term monitoring tasks. In order to prevent the system from crashing during a long-term monitoring scheme, an automatic system state monitoring program and a system recovery module were developed to enhance the stability. In considering that the image resolution for current mobile phone cameras is relatively low (in comparison to contemporary digital SLR cameras), a target detection algorithm was developed for the mobile platform that, when combined with dedicated target patterns, was found to improve the quality of photogrammetric target measurement. Comparing the photogrammetric results with physical measurements, which were measured using a Zeiss P3 analytical plotter, the returned accuracy achieved was 1/67,000. The feasibility of the system has been proven through the implementation of an indoor simulation test and an outdoor experiment. In terms of using this system for actual structural monitoring applications, the optimal relative accuracy of distance measurement was determined to be approximately 1/28,000 under laboratory conditions, and the outdoor experiment returned a relative accuracy of approximately 1/16,400

Advancement and applications of the template matching approach to indexing electron backscatter patterns

Electron backscatter diffraction is a well-established characterisation technique used to determine the orientation and crystal phase of a crystalline material. A pattern is formed by dynamical interaction of elections with the crystal lattice, which can be understood and simulated by using Bloch wave theory. The conventional method of indexing a diffraction pattern is to use a Hough transform to convert the lines of the pattern to points that are easily accessible to a computer. As the bands of the pattern are direct projections of the crystal planes, the interplanar angles can then be computed and compared to a look up table to determine phase and orientation. This method works well for most examples, however, is not well suited to more complex unit cells, due to the fact it ignores more subtle features of the patterns. This thesis proposes a refined template matching approach which uses efficient pattern matching algorithms, such as those used in the field of computer vision, for phase determination and orientation analysis. This thesis introduces the method and demonstrates its efficacy, as well as introducing advanced methods for pseudosymmetry analysis and phase mapping. A new metric for phase confidence is also proposed and the refined method is shown to be able to correctly determine phases and pseudosymmetric orientations. Finally, preliminary work on a direct electron detector stage is presented. Work on the development, testing the pattern centre reliability, modulation transfer and an example map is shown.Open Acces

A Novel Multi-View Table Tennis Umpiring Framework

This research investigates the development of a low-cost multi-view umpiring framework, as an alternative to the current expensive systems that are almost exclusively restricted to elite professional sports. Table tennis has been selected as the testbed because, while automating the process is challenging, it has many different complex match elements including the service, return and rallies, which are governed by a strict set of regulations. The focus is mainly on the rally element rather than the whole match. Ball detection and tracking in video frames are undertaken to determine reliably the ball position relative to key reference objects like the table surface and net, and the ball’s flight path is used to determine the rally’s status. While a low-cost option has benefits, it is technically challenging due to the limited number of cameras and generally low video resolution used. This thesis presents a portable multi-view umpiring framework that identifies each state change in a rally. It makes three significant contributions to knowledge: i) a reliable ball detection strategy that accurately detects the location of the ball in low-resolution sequences; ii) a novel framework for ball tracking using a multi-view system, and iii) a new state-machine based evaluation system for analysing table tennis rallies. In a series of ten different test scenarios, the system achieved an average of 94% system detection rate and 100% accurate decisions. A test sequence of duration 1 s can be processed in 8 s, leading to a delay of only 7 s, which is considered acceptable for practical purposes. This solution has the potential to reform the way matches are umpired, providing objectivity in resolving disputed decisions. It affords an economic technology for amateur players, while the multi-view facility is extendible to other relevant ball-based sports. Finally, the ball flight path analysis mechanism can be a valuable training tool for skills development