An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region

© 2013 Elsevier B.V. All rights reserved. Although photometric stereo offers an attractive technique for acquiring 3D data using low-cost equipment, inherent limitations in the methodology have served to limit its practical application, particularly in measurement or metrology tasks. Here we address this issue. Traditional photometric stereo assumes that lighting directions at every pixel are the same, which is not usually the case in real applications, and especially where the size of object being observed is comparable to the working distance. Such imperfections of the illumination may make the subsequent reconstruction procedures used to obtain the 3D shape of the scene prone to low frequency geometric distortion and systematic error (bias). Also, the 3D reconstruction of the object results in a geometric shape with an unknown scale. To overcome these problems a novel method of estimating the distance of the object from the camera is developed, which employs photometric stereo images without using other additional imaging modality. The method firstly identifies Lambertian diffused maxima region to calculate the object distance from the camera, from which the corrected per-pixel light vector is able to be derived and the absolute dimensions of the object can be subsequently estimated. We also propose a new calibration process to allow a dynamic(as an object moves in the field of view) calculation of light vectors for each pixel with little additional computation cost. Experiments performed on synthetic as well as real data demonstrates that the proposed approach offers improved performance, achieving a reduction in the estimated surface normal error of up to 45% as well as mean height error of reconstructed surface of up to 6 mm. In addition, when compared to traditional photometric stereo, the proposed method reduces the mean angular and height error so that it is low, constant and independent of the position of the object placement within a normal working range

Innovative 3D and 2D machine vision methods for analysis of plants and crops in the field

© 2018 Elsevier B.V. Machine vision systems offer great potential for automating crop control, harvesting, fruit picking, and a range of other agricultural tasks. However, most of the reported research on machine vision in agriculture involves a 2D approach, where the utility of the resulting data is often limited by effects such as parallax, perspective, occlusion and changes in background light – particularly when operating in the field. The 3D approach to plant and crop analysis described in this paper offers potential to obviate many of these difficulties by utilising the richer information that 3D data can generate. The methodologies presented, such as four-light photometric stereo, also provide advanced functionalities, such as an ability to robustly recover 3D surface texture from plants at very high resolution. This offers potential for enabling, for example, reliable detection of the meristem (the part of the plant where growth can take place), to within a few mm, for directed weeding (with all the associated cost and ecological benefits) as well as offering new capabilities for plant phenotyping. The considerable challenges associated with robust and reliable utilisation of machine vision in the field are also considered and practical solutions are described. Two projects are used to illustrate the proposed approaches: a four-light photometric stereo apparatus able to recover plant textures at high-resolution (even in direct sunlight), and a 3D system able to measure potato sizes in-the-field to an accuracy of within 10%, for extended periods and in a range of environmental conditions. The potential benefits of the proposed 3D methods are discussed, both in terms of the advanced capabilities attainable and the widespread potential uptake facilitated by their low cost

Camera positioning for 3D panoramic image rendering

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University London.Virtual camera realisation and the proposition of trapezoidal camera architecture are the two broad contributions of this thesis. Firstly, multiple camera and their arrangement constitute a critical component which affect the integrity of visual content acquisition for multi-view video. Currently, linear, convergence, and divergence arrays are the prominent camera topologies adopted. However, the large number of cameras required and their synchronisation are two of prominent challenges usually encountered. The use of virtual cameras can significantly reduce the number of physical cameras used with respect to any of the known camera structures, hence adequately reducing some of the other implementation issues. This thesis explores to use image-based rendering with and without geometry in the implementations leading to the realisation of virtual cameras. The virtual camera implementation was carried out from the perspective of depth map (geometry) and use of multiple image samples (no geometry). Prior to the virtual camera realisation, the generation of depth map was investigated using region match measures widely known for solving image point correspondence problem. The constructed depth maps have been compare with the ones generated using the dynamic programming approach. In both the geometry and no geometry approaches, the virtual cameras lead to the rendering of views from a textured depth map, construction of 3D panoramic image of a scene by stitching multiple image samples and performing superposition on them, and computation of virtual scene from a stereo pair of panoramic images. The quality of these rendered images were assessed through the use of either objective or subjective analysis in Imatest software. Further more, metric reconstruction of a scene was performed by re-projection of the pixel points from multiple image samples with a single centre of projection. This was done using sparse bundle adjustment algorithm. The statistical summary obtained after the application of this algorithm provides a gauge for the efficiency of the optimisation step. The optimised data was then visualised in Meshlab software environment, hence providing the reconstructed scene. Secondly, with any of the well-established camera arrangements, all cameras are usually constrained to the same horizontal plane. Therefore, occlusion becomes an extremely challenging problem, and a robust camera set-up is required in order to resolve strongly the hidden part of any scene objects. To adequately meet the visibility condition for scene objects and given that occlusion of the same scene objects can occur, a multi-plane camera structure is highly desirable. Therefore, this thesis also explore trapezoidal camera structure for image acquisition. The approach here is to assess the feasibility and potential of several physical cameras of the same model being sparsely arranged on the edge of an efficient trapezoid graph. This is implemented both Matlab and Maya. The quality of the depth maps rendered in Matlab are better in Quality

Single-imager occupant detection based on surface reconstruction

This thesis introduces a novel framework for a real-time occupant detection system capable of extracting both two- and three-dimensional information using a single imager with active illumination. The primary objective of this thesis is to demonstrate the feasibility of such a low-cost classification system with comparable performance to multi-camera based stereo vision systems. Severe illumination conditions characterised by a frequent and wide illumination fluctuation are also challenging problems addressed in this work. The proposed system is designed to solve a problem of classifying three occupant classes being an adult, a forward-facing child seat, and a rear-facing child seat. DoubleFlash is employed to eliminate the influence of ambient illumination and to compress the optical dynamic range of target scenes. The idea underlying this technique is to subtract images flashed by different illumination power levels. The extension of this active illumination technique leads to the development of a novel shadow removal technique, called ShadowFlash. By simulating an artificial infinite illuminating plane over the field of view, the technique produces a shadowless scene without losing image details by composing multiple images illuminated from different directions. The ShadowFlash technique is then extended to the temporal domain by employing the sliding n-tuple strategy, which is introduced to avoid the reduction of the original frame rate. A modified active contour model, facilitated by morphological operations, extracts the boundary of the target object from the shadow-free scenes produced by the ShadowFlash. Based on the brightness information of the image triplet generated by the DoubleFlash, the orientations of the object surface at pixel points are estimated by the photometric stereo method and integrated into the 3D surface by means of global minimisation. The boundary information is used to specify the region of interest to reconstruct. Investigating both the two- and three-dimensional properties of vehicle occupants, 29 features are defined for the training of a neural network. The system is tested on a database of over 84,000 frames collected from a wide range of objects in various illumination conditions. A classification accuracy of 98.9% was achieved within the decision-time limit of three seconds

Measuring and simulating haemodynamics due to geometric changes in facial expression

The human brain has evolved to be very adept at recognising imperfections in human skin. In particular, observing someone’s facial skin appearance is important in recognising when someone is ill, or when finding a suitable mate. It is therefore a key goal of computer graphics research to produce highly realistic renderings of skin. However, the optical processes that give rise to skin appearance are complex and subtle. To address this, computer graphics research has incorporated more and more sophisticated models of skin reflectance. These models are generally based on static concentrations of skin chromophores; melanin and haemoglobin. However, haemoglobin concentrations are far from static, as blood flow is directly caused by both changes in facial expression and emotional state. In this thesis, we explore how blood flow changes as a consequence of changing facial expression with the aim of producing more accurate models of skin appearance. To build an accurate model of blood flow, we base it on real-world measurements of blood concentrations over time. We describe, in detail, the steps required to obtain blood concentrations from photographs of a subject. These steps are then used to measure blood concentration maps for a series of expressions that define a wide gamut of human expression. From this, we define a blending algorithm that allows us to interpolate these maps to generate concentrations for other expressions. This technique, however, requires specialist equipment to capture the maps in the first place. We try to rectify this problem by investigating a direct link between changes in facial geometry and haemoglobin concentrations. This requires building a unique capture device that captures both simultaneously. Our analysis hints a direct linear connection between the two, paving the way for further investigatio

Detailed and Practical 3D Reconstruction with Advanced Photometric Stereo Modelling

Object 3D reconstruction has always been one of the main objectives of computer vision. After many decades of research, most techniques are still unsuccessful at recovering high resolution surfaces, especially for objects with limited surface texture. Moreover, most shiny materials are particularly hard to reconstruct. Photometric Stereo (PS), which operates by capturing multiple images under changing illumination has traditionally been one of the most successful techniques at recovering a large amount of surface details, by exploiting the relationship between shading and local shape. However, using PS has been highly impractical because most approaches are only applicable in a very controlled lab setting and limited to objects experiencing diffuse reflection. Nevertheless, recent advances in differential modelling have made complicated Photometric Stereo models possible and variational optimisations for these kinds of models show remarkable resilience to real world imperfections such as non-Gaussian noise and other outliers. Thus, a highly accurate, photometric-based reconstruction system is now possible. The contribution of this thesis is threefold. First of all, the Photometric Stereo model is extended in order to be able to deal with arbitrary ambient lighting. This is a step towards acquisition in a non-fully controlled lab setting. Secondly, the need for a priori knowledge of the light source brightness and attenuation characteristics is relaxed as an alternating optimisation procedure is proposed which is able to estimate these parameters. This extension allows for quick acquisition with inexpensive LEDs that exhibit unpredictable illumination characteristics (flickering etc). Finally, a volumetric parameterisation is proposed which allows one to tackle the multi-view Photometric Stereo problem in a similar manner, in a simple unified differential model. This final extension allows for complete object reconstruction merging information from multiple images taken from multiple viewpoints and variable illumination. The theoretical work in this thesis is experimentally evaluated in a number of challenging real world experiments, with data captured by custom-made hardware. In addition, the applicability of the generality of the proposed models is demonstrated by presenting a differential model for the shape of polarisation problem, which leads to a unified optimisation problem, fusing information from both methods. This allows for the acquisition of geometrical information about objects such as semi-transparent glass, hitherto hard to deal with

Vision Sensors and Edge Detection

Vision Sensors and Edge Detection book reflects a selection of recent developments within the area of vision sensors and edge detection. There are two sections in this book. The first section presents vision sensors with applications to panoramic vision sensors, wireless vision sensors, and automated vision sensor inspection, and the second one shows image processing techniques, such as, image measurements, image transformations, filtering, and parallel computing

Colour coded

This 300 word publication to be published by the Society of Dyers and Colourists (SDC) is a collection of the best papers from a 4-year European project that has considered colour from the perspective of both the arts and sciences.The notion of art and science and the crossovers between the two resulted in application and funding for cross disciplinary research to host a series of training events between 2006 and 2010 Marie Curie Conferences & Training Courses (SCF) Call Identifier: FP6-Mobility-4, Euros 532,363.80 CREATE – Colour Research for European Advanced Technology Employment. The research crossovers between the fields of art, science and technology was also a subject that was initiated through Bristol’s Festival if Ideas events in May 2009. The author coordinated and chaired an event during which the C.P Snow lecture “On Two Cultures’ (1959) was re-presented by Actor Simon Cook and then a lecture made by Raymond Tallis on the notion of the Polymath. The CREATE project has a worldwide impact for researchers, academics and scientists. Between January and October 2009, the site has received 221, 414 visits. The most popular route into the site is via the welcome page. The main groups of visitors originate in the UK (including Northern Ireland), Italy, France, Finland, Norway, Hungary, USA, Finland and Spain. A basic percentage breakdown of the traffic over ten months indicates: USA -15%; UK - 16%; Italy - 13%; France -12%; Hungary - 10%; Spain - 6%; Finland - 9%; Norway - 5%. The remaining approximate 14% of visitors are from other countries including Belgium, The Netherlands and Germany (approx 3%). A discussion group has been initiated by the author as part of the CREATE project to facilitate an ongoing dialogue between artists and scientists. http://createcolour.ning.com/group/artandscience www.create.uwe.ac.uk.Related papers to this research: A report on the CREATE Italian event: Colour in cultural heritage.C. Parraman, A. Rizzi, ‘Developing the CREATE network in Europe’, in Colour in Art, Design and Nature, Edinburgh, 24 October 2008.C. Parraman, “Mixing and describing colour”. CREATE (Training event 1), France, 2008