Machine learning for the automation and optimisation of optical coordinate measurement

Camera based methods for optical coordinate metrology are growing in popularity due to their non-contact probing technique, fast data acquisition time, high point density and high surface coverage. However, these optical approaches are often highly user dependent, have high dependence on accurate system characterisation, and can be slow in processing the raw data acquired during measurement. Machine learning approaches have the potential to remedy the shortcomings of such optical coordinate measurement systems. The aim of this thesis is to remove dependence on the user entirely by enabling full automation and optimisation of optical coordinate measurements for the first time. A novel software pipeline is proposed, built, and evaluated which will enable automated and optimised measurements to be conducted. No such automated and optimised system for performing optical coordinate measurements currently exists. The pipeline can be roughly summarised as follows: intelligent characterisation -> view planning -> object pose estimation -> automated data acquisition -> optimised reconstruction. Several novel methods were developed in order to enable the embodiment of this pipeline. Chapter 4 presents an intelligent camera characterisation (the process of determining a mathematical model of the optical system) is performed using a hybrid approach wherein an EfficientNet convolutional neural network provides sub-pixel corrections to feature locations provided by the popular OpenCV library. The proposed characterisation scheme is shown to robustly refine the characterisation result as quantified by a 50 % reduction in the mean residual magnitude. The camera characterisation is performed before measurements are performed and the results are fed as an input to the pipeline. Chapter 5 presents a novel genetic optimisation approach is presented to create an imaging strategy, ie. the positions from which data should be captured relative to part’s specific geometry. This approach exploits the computer aided design (CAD) data of a given part, ensuring any measurement is optimal given a specific target geometry. This view planning approach is shown to give reconstructions with closer agreement to tactile coordinate measurement machine (CMM) results from 18 images compared to unoptimised measurements using 60 images. This view planning algorithm assumes the part is perfectly placed in the centre of the measurement volume so is first adjusted for an arbitrary placement of the part before being used for data acquistion. Chapter 6 presents a generative model for the creation of surface texture data is presented, allowing the generation of synthetic butt realistic datasets for the training of statistical models. The surface texture generated by the proposed model is shown to be quantitatively representative of real focus variation microscope measurements. The model developed in this chapter is used to produce large synthetic but realistic datasets for the training of further statistical models. Chapter 7 presents an autonomous background removal approach is proposed which removes superfluous data from images captured during a measurement. Using images processed by this algorithm to reconstruct a 3D measurement of an object is shown to be effective in reducing data processing times and improving measurement results. Use the proposed background removal on images before reconstruction are shown to benefit from up to a 41 % reduction in data processing times, a reduction in superfluous background points of up to 98 %, an increase in point density on the object surface of up to 10 %, and an improved agreement with CMM as measured by both a reduction in outliers and reduction in the standard deviation of point to mesh distances of up to 51 microns. The background removal algorithm is used to both improve the final reconstruction and within stereo pose estimation. Finally, in Chapter 8, two methods (one monocular and one stereo) for establishing the initial pose of the part to be measured relative to the measurement volume are presented. This is an important step to enabling automation as it allows the user to place the object at an arbitrary location in the measurement volume and for the pipeline to adjust the imaging strategy to account for this placement, enabling the optimised view plan to be carried out without the need for special part fixturing. It is shown that the monocular method can locate a part to within an average of 13 mm and the stereo method can locate apart to within an average of 0.44 mm as evaluated on 240 test images. Pose estimation is used to provide a correction to the view plan for an arbitrary part placement without the need for specialised fixturing or fiducial marking. This pipeline enables an inexperienced user to place a part anywhere in the measurement volume of a system and, from the part’s associated CAD data, the system will perform an optimal measurement without the need for any user input. Each new method which was developed as part of this pipeline has been validated against real experimental data from current measurement systems and shown to be effective. In future work given in Section 9.1, a possible hardware integration of the methods developed in this thesis is presented. Although the creation of this hardware is beyond the scope of this thesis

Machine learning for the automation and optimisation of optical coordinate measurement

Camera based methods for optical coordinate metrology are growing in popularity due to their non-contact probing technique, fast data acquisition time, high point density and high surface coverage. However, these optical approaches are often highly user dependent, have high dependence on accurate system characterisation, and can be slow in processing the raw data acquired during measurement. Machine learning approaches have the potential to remedy the shortcomings of such optical coordinate measurement systems. The aim of this thesis is to remove dependence on the user entirely by enabling full automation and optimisation of optical coordinate measurements for the first time. A novel software pipeline is proposed, built, and evaluated which will enable automated and optimised measurements to be conducted. No such automated and optimised system for performing optical coordinate measurements currently exists. The pipeline can be roughly summarised as follows: intelligent characterisation -> view planning -> object pose estimation -> automated data acquisition -> optimised reconstruction. Several novel methods were developed in order to enable the embodiment of this pipeline. Chapter 4 presents an intelligent camera characterisation (the process of determining a mathematical model of the optical system) is performed using a hybrid approach wherein an EfficientNet convolutional neural network provides sub-pixel corrections to feature locations provided by the popular OpenCV library. The proposed characterisation scheme is shown to robustly refine the characterisation result as quantified by a 50 % reduction in the mean residual magnitude. The camera characterisation is performed before measurements are performed and the results are fed as an input to the pipeline. Chapter 5 presents a novel genetic optimisation approach is presented to create an imaging strategy, ie. the positions from which data should be captured relative to part’s specific geometry. This approach exploits the computer aided design (CAD) data of a given part, ensuring any measurement is optimal given a specific target geometry. This view planning approach is shown to give reconstructions with closer agreement to tactile coordinate measurement machine (CMM) results from 18 images compared to unoptimised measurements using 60 images. This view planning algorithm assumes the part is perfectly placed in the centre of the measurement volume so is first adjusted for an arbitrary placement of the part before being used for data acquistion. Chapter 6 presents a generative model for the creation of surface texture data is presented, allowing the generation of synthetic butt realistic datasets for the training of statistical models. The surface texture generated by the proposed model is shown to be quantitatively representative of real focus variation microscope measurements. The model developed in this chapter is used to produce large synthetic but realistic datasets for the training of further statistical models. Chapter 7 presents an autonomous background removal approach is proposed which removes superfluous data from images captured during a measurement. Using images processed by this algorithm to reconstruct a 3D measurement of an object is shown to be effective in reducing data processing times and improving measurement results. Use the proposed background removal on images before reconstruction are shown to benefit from up to a 41 % reduction in data processing times, a reduction in superfluous background points of up to 98 %, an increase in point density on the object surface of up to 10 %, and an improved agreement with CMM as measured by both a reduction in outliers and reduction in the standard deviation of point to mesh distances of up to 51 microns. The background removal algorithm is used to both improve the final reconstruction and within stereo pose estimation. Finally, in Chapter 8, two methods (one monocular and one stereo) for establishing the initial pose of the part to be measured relative to the measurement volume are presented. This is an important step to enabling automation as it allows the user to place the object at an arbitrary location in the measurement volume and for the pipeline to adjust the imaging strategy to account for this placement, enabling the optimised view plan to be carried out without the need for special part fixturing. It is shown that the monocular method can locate a part to within an average of 13 mm and the stereo method can locate apart to within an average of 0.44 mm as evaluated on 240 test images. Pose estimation is used to provide a correction to the view plan for an arbitrary part placement without the need for specialised fixturing or fiducial marking. This pipeline enables an inexperienced user to place a part anywhere in the measurement volume of a system and, from the part’s associated CAD data, the system will perform an optimal measurement without the need for any user input. Each new method which was developed as part of this pipeline has been validated against real experimental data from current measurement systems and shown to be effective. In future work given in Section 9.1, a possible hardware integration of the methods developed in this thesis is presented. Although the creation of this hardware is beyond the scope of this thesis

Autonomous image background removal for accurate and efficient close-range photogrammetry

Close-range photogrammetry can be used to reconstruct dense point clouds of an object with very high surface coverage, making it useful for manufacturing metrology tasks such as part inspection and validation. However, compared to competing techniques, data processing times can be slow. In this paper we present a method to autonomously remove thebackground from the images within a photogrammetric dataset. We show that using masked images directly in the reconstruction results in much lower data processing times, with lower memory utilisation. Furthermore, we show that the point density on the object surface isincreased while the number of superfluous background points is reduced. Finally, a set of reconstruction results are compared to a set of tactile coordinate measurements.Reconstructions with the background removed are shown to have a standard deviation in the point to mesh distance of up to 30 µm lower than if the background is not removed. This improvement in standard deviation is likely due to the static background, relative to the objecton the rotation stage, causing triangulation errors when points are detected and matched on this background data. The proposed approach is shown to be robust over several example artefacts and can, therefore, be implemented to improve the measurement efficiency and measurement results of photogrammetry coordinate measurement systems

Generation and categorisation of surface texture data using a modified progressively growing adversarial network

As machine learning becomes more popular in the precision engineering sector, the need for large datasets of measurement data increases. Due to the often manual, user dependent and labour-intensive measurement processes, collecting a large amount of data is often infeasible. It would, therefore, be desirable to collect a small amount of data on which to train a model to generate synthetic data that is representative of the real measurement data. To this end, we present an approach to numerical surface texture generation based on a progressively growing generative adversarial network. We show that by encoding height data into grayscale values within an image, the network can create realistic synthetic surface data both qualitatively and quantitatively. The proposed approach is general to any encoded surface; we demonstrate the model trained on two example datasets consisting of surfaces from different manufacturing processes and measured with different techniques. We finally present an extension to the generator model which automatically categorises the produced surfaces, allowing a surface of a desired category to be generated. Finally, we calculate the distributions of areal surface texture parameters for each type of surface and show that there is good agreement between the synthetic and real data

Optimisation of camera positions for optical coordinate measurement based on visible point analysis

In optical coordinate measurement using cameras, the number of images, and positions and orientations of the cameras, are critical to object accessibility and the accuracy of a measurement. In this paper, we propose a technique to optimise the number of cameras and the positions of these cameras for the measurement of a given object using visible point analysis of the object's computer aided design data. The visible point analysis technique is based on a hidden point removal approach; this technique is used to detect which surface points on the object are visible from a given camera position. A genetic algorithm is used to find the set of positions that provide optimum surface point density and overlap between views, while minimising the total number of camera images required. The genetic algorithm is used to minimise the measurement data processing time while maintaining optimum surface point density. We test this optimisation procedure on four artefacts and the measurements are shown to be comparable to that from a traceable contact co-ordinate measurement machine. We show that using our procedure improves the measurement quality compared to the more conventional approach of using equally spaced images. This work is part of a larger effort to fully automate and optimise optical coordinate measurement techniques

Smart optical coordinate and surface metrology

Manufacturing has recently experienced increased adoption of optimised and fast solutions for checking product quality during fabrication, allowing for manufacturing times and costs to be significantly reduced. Due to the integration of machine learning algorithms, advanced sensors and faster processing systems, smart instruments can autonomously plan measurement pipelines, perform decisional tasks and trigger correctional actions as required. In this paper, we summarise the state of the art in smart optical metrology, covering the latest advances in integrated intelligent solutions in optical coordinate and surface metrology, respectively for the measurement of part geometry and surface texture. Within this field, we include the use of a priori knowledge and implementation of machine learning algorithms for measurement planning optimisation. We also cover the development of multi-sensor and multi-view instrument configurations to speed up the measurement process, as well as the design of novel feedback tools for measurement quality evaluation

20,000 years of societal vulnerability and adaptation to climate change in southwest Asia.

The Fertile Crescent, its hilly flanks and surrounding drylands has been a critical region for studying how climate has influenced societal change, and this review focuses on the region over the last 20,000 years. The complex social, economic, and environmental landscapes in the region today are not new phenomena and understanding their interactions requires a nuanced, multidisciplinary understanding of the past. This review builds on a history of collaboration between the social and natural palaeoscience disciplines. We provide a multidisciplinary, multiscalar perspective on the relevance of past climate, environmental, and archaeological research in assessing present day vulnerabilities and risks for the populations of southwest Asia. We discuss the complexity of palaeoclimatic data interpretation, particularly in relation to hydrology, and provide an overview of key time periods of palaeoclimatic interest. We discuss the critical role that vegetation plays in the human-climate-environment nexus and discuss the implications of the available palaeoclimate and archaeological data, and their interpretation, for palaeonarratives of the region, both climatically and socially. We also provide an overview of how modelling can improve our understanding of past climate impacts and associated change in risk to societies. We conclude by looking to future work, and identify themes of "scale" and "seasonality" as still requiring further focus. We suggest that by appreciating a given locale's place in the regional hydroscape, be it an archaeological site or palaeoenvironmental archive, more robust links to climate can be made where appropriate and interpretations drawn will demand the resolution of factors acting across multiple scales. This article is categorized under:Human Water > Water as Imagined and RepresentedScience of Water > Water and Environmental ChangeWater and Life > Nature of Freshwater Ecosystems

20,000 years of societal vulnerability and adaptation to climate change in southwest Asia.

The Fertile Crescent, its hilly flanks and surrounding drylands has been a critical region for studying how climate has influenced societal change, and this review focuses on the region over the last 20,000 years. The complex social, economic, and environmental landscapes in the region today are not new phenomena and understanding their interactions requires a nuanced, multidisciplinary understanding of the past. This review builds on a history of collaboration between the social and natural palaeoscience disciplines. We provide a multidisciplinary, multiscalar perspective on the relevance of past climate, environmental, and archaeological research in assessing present day vulnerabilities and risks for the populations of southwest Asia. We discuss the complexity of palaeoclimatic data interpretation, particularly in relation to hydrology, and provide an overview of key time periods of palaeoclimatic interest. We discuss the critical role that vegetation plays in the human-climate-environment nexus and discuss the implications of the available palaeoclimate and archaeological data, and their interpretation, for palaeonarratives of the region, both climatically and socially. We also provide an overview of how modelling can improve our understanding of past climate impacts and associated change in risk to societies. We conclude by looking to future work, and identify themes of "scale" and "seasonality" as still requiring further focus. We suggest that by appreciating a given locale's place in the regional hydroscape, be it an archaeological site or palaeoenvironmental archive, more robust links to climate can be made where appropriate and interpretations drawn will demand the resolution of factors acting across multiple scales. This article is categorized under:Human Water > Water as Imagined and RepresentedScience of Water > Water and Environmental ChangeWater and Life > Nature of Freshwater Ecosystems