Online synchronous inspection and system optimization of flexible food packaging bags by using machine vision and sensing technique

Flexible food packaging in the market is increasingly favored, and its quality is essential and indispensable for safety and convenience.  However, quality inspection still stays in the manual stage, or partially manual inspection remains, in production, leading low efficiency, lack and even false inspection, hardly meeting the requirements of the modern output.  This paper proposes and optimizes the design of an automatic detection system with intelligence for flexible food packaging bag, which can effectively be adopted to check the quality of packaging trademark patterns, fillers, and sealing quality.  The inspection system runs with two-stage structure, machine vision, pressure sensing and synchronization to improve efficiency and ensure the normal production beat. Simplex Method is adopted to determine the best synchronous speeds online to achieve the best expectation. Comparison has been made between the manual inspection and our automatic operation, the sample of 10000 was statistically analyzed and results have shown that two workers were saved and the correctness rate of inspection raised up to 999.8‰

Non-contact monitoring of railway infrastructure with terrestrial laser scanning and photogrammetry at Network Rail

Current monitoring practices in the railway industry primarily rely on total station and prism based methods. This approach requires the installation and maintenance of prisms directly onto the structure being monitored which can be invasive and expensive. This thesis presents the outcomes of an industrial based doctorate, motivated by the Network Rail Thameslink Programme, to investigate the potential of terrestrial laser scanning and photogrammetry as an alternative non-contact and “target-less” solution to monitoring railway infrastructure. The contributions made by this thesis in the context of Network Rail requirements include: a laboratory based exploration of the state of the art in target and surface-based measurements; a validation of conventional, terrestrial laser scan and photogrammetric surveys of a deforming set of brick arches; and a novel prism-less method of track measurement using terrestrial laser scanner data. The complete project has been carried out as part of the highly complex and dynamic £900m London Bridge Redevelopment Project. The thesis comprises of a review of existing monitoring system performance and highlights challenges in the adoption of this technology through interviews of leading professionals in the monitoring industry. Laboratory tests utilise network adjustment prediction and analysis to compare state of the art total station, terrestrial laser scanning and close-range photogrammetry instrumentation to both target and target-less deformation monitoring scenarios. The developed tests allow the performance of each technique to be assessed within the context of state of the art and Network Rail operational practice and are extensible to developments in each of these technologies. Results demonstrate performances to sub-millimetre level and are validated through the use of a Leica AT401 laser tracker. Each technique is then explored within the London Bridge Redevelopment Project through a series of live monitoring sites where their ability to either augment or replace existing survey techniques is evaluated. Results from the on-site monitoring of historic brick arch structures demonstrate surface measurements compatibility at the millimetre level, highlighting close agreement between instrument performance established in the laboratory. A key use of prism-based techniques is in the determination of engineering track parameters where costly prism systems, both in terms of installation and subsequent maintenance, attached to the track are a key concern. Here laboratory validated track surface measurement, with terrestrial laser scanning, has been deployed on a 15 metre long dual track site and shown to be highly capable of replacing prism systems for the determination of accurate track geometry. This work has included a novel optical non-contact measurement process utilising individual rail cross section designs to automatically extract relevant track geometry parameters within 1mm of prism-based methods. The method offers excellent potential for incorporation into an automated track monitoring system. Outcomes from the thesis have been published in peer-reviewed journals and conferences

Civil integrated management and the implementation of CIM-related technologies in the transportation industry

As advanced technologies are adopted in the transportation industry, it is important to investigate how they can be integrated and better utilized to facilitate the success of public transportation agencies. It is also important to devise efficient ways to address the considerable amount of digital data created as these technologies are used. A newly introduced concept – Civil Integrated Management (CIM) – has potential for addressing these issues, because it involves collection, organization, and managed accessibility of accurate data and information throughout the transportation asset lifecycle. CIM is also expected to facilitate the use of various advanced technologies, so the purpose of this dissertation is to further explore the concept of CIM and investigate how it can be implemented to assist with transportation projects and programs. After initial preparation (i.e., conducting literature reviews, consulting with experts, developing questionnaires, and identifying target agencies), two weeks of on-site visits were conducted with seven state transportation agencies to document their insights and practices associated with the CIM concept. Coding strategies were used to analyze the field notes collected from the presentations provided by host agencies and discussions throughout the visits. To further investigate one of the CIM enabling technologies – light detection and ranging (LiDAR), a web-based survey was disseminated to 28 LiDAR professionals; it produced 15 responses. Five phone interviews were also conducted using the Delphi method to develop a LiDAR data utilization workflow for 3D modeling. To investigate another CIM enabling technology – an electronic document management (EDM) system, important data related to EDM practices were extracted from field notes obtained from the CIM on-site visits. Meanwhile, follow-up interviews were conducted with the four transportation agencies identified as leading agencies with respect to EDM, and a video interview was conducted with one additional construction company involved with enterprise-level EDM implementation. As pioneering research on CIM, the results of this dissertation provide transportation agencies and other researchers with an essential roadmap for implementing and refining the CIM concept. The findings and recommendations listed in this dissertation are also expected to assist transportation agencies in better utilizing and integrating various CIM-related technologies into their transportation projects and programs

Intelligence based error detection and classification for 3D measurement systems

For many years 2D machine vision has been used to perform automated inspection and measuring in the manufacturing environment. A strong drive to automate manufacturing has meant improvements in robotics and sensor technologies. So has machine vision seen a steady movement away from 2D and towards 3D. It is necessary to research and develop software that can use these new 3D sensing equipment in novel and useful ways. One task that is particularly useful, for a variety of situations is object recognition. It was hypothesised that it should be possible to train artificial neural networks to recognise 3D objects. For this purpose a 3D laser scanner was developed. This scanner and its software was developed and tested first in a virtual environment and what was learned there was then used to implemented an actual scanner. This scanner served the purpose of verifying what was done in the virtual environment. Neural networks of different sized were trained to establish whether they are a feasible classifier for the task of object recognition. Testing showed that, with the correct preprocessing, it is possible to perform 3D object recognition on simple geometric shapes by means of artificial neural networks

Advances in Waveform and Photon Counting Lidar Processing for Forest Vegetation Applications

Full waveform (FW) and photon counting LiDAR (PCL) data have garnered greater attention due to increasing data availability, a wealth of information they contain and promising prospects for large scale vegetation mapping. However, many factors such as complex processing steps and scarce non-proprietary tools preclude extensive and practical uses of these data for vegetation characterization. Therefore, the overall goal of this study is to develop algorithms to process FW and PCL data and to explore their potential in real-world applications. Study I explored classical waveform decomposition methods such as the Gaussian decomposition, Richardson–Lucy (RL) deconvolution and a newly introduced optimized Gold deconvolution to process FW LiDAR data. Results demonstrated the advantages of the deconvolution and decomposition method, and the three approaches generated satisfactory results, while the best performances varied when different criteria were used. Built upon Study I, Study II applied the Bayesian non-linear modeling concepts for waveform decomposition and quantified the propagation of error and uncertainty along the processing steps. The performance evaluation and uncertainty analysis at the parameter, derived point cloud and surface model levels showed that the Bayesian decomposition could enhance the credibility of decomposition results in a probabilistic sense to capture the true error of estimates and trace the uncertainty propagation along the processing steps. In study III, we exploited FW LiDAR data to classify tree species through integrating machine learning methods (the Random forests (RF) and Conditional inference forests (CF)) and Bayesian inference method. Results of classification accuracy highlighted that the Bayesian method was a superior alternative to machine learning methods, and rendered users with more confidence for interpreting and applying classification results to real-world tasks such as forest inventory. Study IV focused on developing a framework to derive terrain elevation and vegetation canopy height from test-bed sensor data and to pre-validate the capacity of the upcoming Ice, Cloud and Land Elevation Satellite-2 (ICESat-2) mission. The methodology developed in this study illustrates plausible ways of processing the data that are structurally similar to expected ICESat-2 data and holds the potential to be a benchmark for further method adjustment once genuine ICESat-2 are available

Generation of Horizontally Curved Driving Lines for Autonomous Vehicles Using Mobile Laser Scanning Data

The development of autonomous vehicle desiderates tremendous advances in three-dimensional (3D) high-definition roadmaps. These roadmaps are capable of providing 3D positioning information with 10-to-20 cm accuracy. With the assistance of 3D high-definition roadmaps, the intractable autonomous driving problem is transformed into a solvable localization issue. The Mobile Laser Scanning (MLS) systems can collect accurate, high-density 3D point clouds in road environments for generating 3D high-definition roadmaps. However, few studies have been concentrated on the driving line generation from 3D MLS point clouds for highly autonomous driving, particularly for accident-prone horizontal curves with the problems of ambiguous traffic situations and unclear visual clues. This thesis attempts to develop an effective method for semi-automated generation of horizontally curved driving lines using MLS data. The framework of research methodology proposed in this thesis consists of three steps, including road surface extraction, road marking extraction, and driving line generation. Firstly, the points covering road surface are extracted using curb-based road surface extraction algorithms depending on both the elevation and slope differences. Then, road markings are identified and extracted according to a sequence of algorithms consisting of geo-referenced intensity image generation, multi-threshold road marking extraction, and statistical outlier removal. Finally, the conditional Euclidean clustering algorithm is employed followed by the nonlinear least-squares curve-fitting algorithm for generating horizontally curved driving lines. A total of six test datasets obtained in Xiamen, China by a RIEGL VMX-450 system were used to evaluate the performance and efficiency of the proposed methodology. The experimental results demonstrate that the proposed road marking extraction algorithms can achieve 90.89% in recall, 93.04% in precision and 91.95% in F1-score, respectively. Moreover, the unmanned aerial vehicle (UAV) imagery with 4 cm was used for validation of the proposed driving line generation algorithms. The validation results demonstrate that the horizontally curved driving lines can be effectively generated within 15 cm-level localization accuracy using MLS point clouds. Finally, a comparative study was conducted both visually and quantitatively to indicate the accuracy and reliability of the generated driving lines

Adaptive Methods for Point Cloud and Mesh Processing

Point clouds and 3D meshes are widely used in numerous applications ranging from games to virtual reality to autonomous vehicles. This dissertation proposes several approaches for noise removal and calibration of noisy point cloud data and 3D mesh sharpening methods. Order statistic filters have been proven to be very successful in image processing and other domains as well. Different variations of order statistics filters originally proposed for image processing are extended to point cloud filtering in this dissertation. A brand-new adaptive vector median is proposed in this dissertation for removing noise and outliers from noisy point cloud data. The major contributions of this research lie in four aspects: 1) Four order statistic algorithms are extended, and one adaptive filtering method is proposed for the noisy point cloud with improved results such as preserving significant features. These methods are applied to standard models as well as synthetic models, and real scenes, 2) A hardware acceleration of the proposed method using Microsoft parallel pattern library for filtering point clouds is implemented using multicore processors, 3) A new method for aerial LIDAR data filtering is proposed. The objective is to develop a method to enable automatic extraction of ground points from aerial LIDAR data with minimal human intervention, and 4) A novel method for mesh color sharpening using the discrete Laplace-Beltrami operator is proposed. Median and order statistics-based filters are widely used in signal processing and image processing because they can easily remove outlier noise and preserve important features. This dissertation demonstrates a wide range of results with median filter, vector median filter, fuzzy vector median filter, adaptive mean, adaptive median, and adaptive vector median filter on point cloud data. The experiments show that large-scale noise is removed while preserving important features of the point cloud with reasonable computation time. Quantitative criteria (e.g., complexity, Hausdorff distance, and the root mean squared error (RMSE)), as well as qualitative criteria (e.g., the perceived visual quality of the processed point cloud), are employed to assess the performance of the filters in various cases corrupted by different noisy models. The adaptive vector median is further optimized for denoising or ground filtering aerial LIDAR data point cloud. The adaptive vector median is also accelerated on multi-core CPUs using Microsoft Parallel Patterns Library. In addition, this dissertation presents a new method for mesh color sharpening using the discrete Laplace-Beltrami operator, which is an approximation of second order derivatives on irregular 3D meshes. The one-ring neighborhood is utilized to compute the Laplace-Beltrami operator. The color for each vertex is updated by adding the Laplace-Beltrami operator of the vertex color weighted by a factor to its original value. Different discretizations of the Laplace-Beltrami operator have been proposed for geometrical processing of 3D meshes. This work utilizes several discretizations of the Laplace-Beltrami operator for sharpening 3D mesh colors and compares their performance. Experimental results demonstrated the effectiveness of the proposed algorithms

Edge detection in unorganized 3D point cloud

The application of 3D laser scanning in the mining industry is increasing progressively over the years. This presents an opportunity to visualize and analyze the underground world and potentially save countless man- hours and exposure to safety incidents. This thesis envisions to detect the “Edges of the Rocks” in the 3D point cloud collected via scanner, although edge detection in point cloud is considered as a difficult but meaningful problem. As a solution to noisy and unorganized 3D point cloud, a new method, EdgeScan method, has been proposed and implemented to detect fast and accurate edges from the 3D point cloud for real time systems. EdgeScan method is aimed to make use of 2D edge processing techniques to represent the edge characteristics in 3D point cloud with better accuracy. A comparisons of EdgeScan method with other common edge detection methods for 3D point cloud is administered, eventually, results suggest that the stated EdgeScan method furnishes a better speed and accuracy especially for large dataset in real time systems.Master of Science (MSc) in Computational Science

Recognizing Features in Mobile Laser Scanning Point Clouds Towards 3D High-definition Road Maps for Autonomous Vehicles

The sensors mounted on a driverless vehicle are not always reliable for precise localization and navigation. By comparing the real-time sensory data with a priori map, the autonomous navigation system can transform the complicated sensor perception mission into a simple map-based localization task. However, the lack of robust solutions and standards for creating such lane-level high-definition road maps is a major challenge in this emerging field. This thesis presents a semi-automated method for extracting meaningful road features from mobile laser scanning (MLS) point clouds and creating 3D high-definition road maps for autonomous vehicles. After pre-processing steps including coordinate system transformation and non-ground point removal, a road edge detection algorithm is performed to distinguish road curbs and extract road surfaces followed by extraction of two categories of road markings. On the one hand, textual and directional road markings including arrows, symbols, and words are detected by intensity thresholding and conditional Euclidean clustering. On the other hand, lane markings (lines) are extracted by local intensity analysis and distance thresholding according to road design standards. Afterwards, centerline points in every single lane are estimated based on the position of the extracted lane markings. Ultimately, 3D road maps with precise road boundaries, road markings, and the estimated lane centerlines are created. The experimental results demonstrate the feasibility of the proposed method, which can accurately extract most road features from the MLS point clouds. The average recall, precision, and F1-score obtained from four datasets for road marking extraction are 93.87%, 93.76%, and 93.73%, respectively. All of the estimated lane centerlines are validated using the “ground truthing” data manually digitized from the 4 cm resolution UAV orthoimages. The results of a comparison study show the better performance of the proposed method than that of some other existing methods