A Co-optimal Coverage Path Planning Method for Aerial Scanning of Complex Structures

The utilization of unmanned aerial vehicles (UAVs) in survey and inspection of civil infrastructure has been growing rapidly. However, computationally efficient solvers that find optimal flight paths while ensuring high-quality data acquisition of the complete 3D structure remains a difficult problem. Existing solvers typically prioritize efficient flight paths, or coverage, or reducing computational complexity of the algorithm – but these objectives are not co-optimized holistically. In this work we introduce a co-optimal coverage path planning (CCPP) method that simultaneously co-optimizes the UAV path, the quality of the captured images, and reducing computational complexity of the solver all while adhering to safety and inspection requirements. The result is a highly parallelizable algorithm that produces more efficient paths where quality of the useful image data is improved. The path optimization algorithm utilizes a particle swarm optimization (PSO) framework which iteratively optimizes the coverage paths without needing to discretize the motion space or simplify the sensing models as is done in similar methods. The core of the method consists of a cost function that measures both the quality and efficiency of a coverage inspection path, and a greedy heuristic for the optimization enhancement by aggressively exploring the viewpoints search spaces. To assess the proposed method, a coverage path quality evaluation method is also presented in this research, which can be utilized as the benchmark for assessing other CPP methods for structural inspection purpose. The effectiveness of the proposed method is demonstrated by comparing the quality and efficiency of the proposed approach with the state-of-art through both synthetic and real-world scenes. The experiments show that our method enables significant performance improvement in coverage inspection quality while preserving the path efficiency on different test geometries

Simulation-based Planning of Machine Vision Inspection Systems with an Application to Laser Triangulation

Nowadays, vision systems play a central role in industrial inspection. The experts typically choose the configuration of measurements in such systems empirically. For complex inspections, however, automatic inspection planning is essential. This book proposes a simulation-based approach towards inspection planning by contributing to all components of this problem: simulation, evaluation, and optimization. As an application, inspection of a complex cylinder head by laser triangulation is studied

Asymptotically optimized multi-surface coverage path planning for loco-manipulation in inspection and monitoring

Regular inspection and monitoring of aging assets are crucial to safe operation in industrial facilities, with remote robotic monitoring being a particularly promising approach for asset inspection. However, vessels, pipework, and surfaces to be monitored can follow complex 3D surfaces, and frequently no 3D as-built models exist. In this paper, we present an end-to-end solution that uses an optimization method for coverage path planning of multiple complex surfaces for mobile robot manipulators. The system includes a two-layer hierarchical structure of optimization: mission planning and motion planning. The surface sequence is optimized with a mixed-integer linear programming formulation while motion planning solves a whole-body optimal control problem considering the robot as a floating-base system. The loco-manipulation system automatically plans a full-coverage trajectory over multiple surfaces for contact-based non-destructive monitoring after unrolling the 3D-mesh region-of-interest selected from the user interface and projects it back to the surface. Our pipeline aims at offshore asset inspection and remote monitoring in industrial applications, and is also applicable in manufacturing and maintenance where area coverage is critical. We demonstrate the generality and scalability of our solution in a variety of robotic coverage path planning applications, including for multi-surface asset inspection using a quadrupedal manipulator

Automation of Process Planning for Automated Fiber Placement

Process planning represents an essential stage of the Automated Fiber Placement (AFP) workflow. It develops useful and efficient machine processes based upon the working material, composite design, and manufacturing resources. The current state of process planning requires a high degree of interaction from the process planner and could greatly benefit from increased automation. Therefore, a list of key steps and functions are created to identify the more difficult and time-consuming phases of process planning. Additionally, a set of metrics must exist by which to evaluate the effectiveness of the manufactured laminate from the machine code created during the Process Planning stage. This work begins with a ranking process which was performed through a survey of the Advanced Composites Consortium (ACC) Collaborative Research Team (CRT). Members were interviewed who possessed practical process planning experience in the composites industry. The Process Planning survey collected general input on the overall importance and time requirements for each function and which functions would benefit most greatly from semi-automation or full automation. Layup strategies, in addition to dog ears, stagger shifts, steering constraints, and starting points, represented the group of functions labeled as process optimization and ranked the highest in terms of priority for automation. The laminates resulting from the selected parameters are evaluated through the occurrences of principal defect metrics such as fiber gaps, overlaps, angle deviation and steering violations. This document presents an automated software solution to the layup strategy and starting point selection phase of Process Planning. A series of ply scenarios are generated with variations of these ply parameters and evaluated according to a set of metrics entered by the Process Planner. These metrics are generated through use of the Analytical Hierarchy Process (AHP), where relative importance between each of the fiber features are defined. The ply scenarios are selected which reduce the overall fiber feature scores based on the defects the Process Planner wishes to minimize

Optimisation of surface coverage paths used by a non-contact robot painting system

This thesis proposes an efficient path planning technique for a non-contact optical “painting” system that produces surface images by moving a robot mounted laser across objects covered in photographic emulsion. In comparison to traditional 3D planning approaches (e.g. laminar slicing) the proposed algorithm dramatically reduces the overall path length by optimizing (i.e. minimizing) the amounts of movement between robot configurations required to position and orientate the laser. To do this the pixels of the image (i.e. points on the surface of the object) are sequenced using configuration space rather than Cartesian space. This technique extracts data from a CAD model and then calculates the configuration that the five degrees of freedom system needs to assume to expose individual pixels on the surface. The system then uses a closest point analysis on all the major joints to sequence the points and create an efficient path plan for the component. The implementation and testing of the algorithm demonstrates that sequencing points using a configuration based method tends to produce significantly shorter paths than other approaches to the sequencing problem. The path planner was tested with components ranging from simple to complex and the paths generated demonstrated both the versatility and feasibility of the approach

Prioritized Multi-View Stereo Depth Map Generation Using Confidence Prediction

In this work, we propose a novel approach to prioritize the depth map computation of multi-view stereo (MVS) to obtain compact 3D point clouds of high quality and completeness at low computational cost. Our prioritization approach operates before the MVS algorithm is executed and consists of two steps. In the first step, we aim to find a good set of matching partners for each view. In the second step, we rank the resulting view clusters (i.e. key views with matching partners) according to their impact on the fulfillment of desired quality parameters such as completeness, ground resolution and accuracy. Additional to geometric analysis, we use a novel machine learning technique for training a confidence predictor. The purpose of this confidence predictor is to estimate the chances of a successful depth reconstruction for each pixel in each image for one specific MVS algorithm based on the RGB images and the image constellation. The underlying machine learning technique does not require any ground truth or manually labeled data for training, but instead adapts ideas from depth map fusion for providing a supervision signal. The trained confidence predictor allows us to evaluate the quality of image constellations and their potential impact to the resulting 3D reconstruction and thus builds a solid foundation for our prioritization approach. In our experiments, we are thus able to reach more than 70% of the maximal reachable quality fulfillment using only 5% of the available images as key views. For evaluating our approach within and across different domains, we use two completely different scenarios, i.e. cultural heritage preservation and reconstruction of single family houses.Comment: This paper was accepted to ISPRS Journal of Photogrammetry and Remote Sensing (https://www.journals.elsevier.com/isprs-journal-of-photogrammetry-and-remote-sensing) on March 21, 2018. The official version will be made available on ScienceDirect (https://www.sciencedirect.com

Leveraging Automated Fiber Placement Computer Aided Process Planning Framework for Defect Validation and Dynamic Layup Strategies

Process planning represents an essential stage of the Automated Fiber Placement (AFP) workflow. It develops useful and efficient machine processes based upon the working material, composite design, and manufacturing resources. The current state of process planning requires a high degree of interaction from the process planner and could greatly benefit from increased automation. Therefore, a list of key steps and functions are created to identify the more difficult and time-consuming phases of process planning. Additionally, a set of metrics must exist by which to evaluate the effectiveness of the manufactured laminate from the machine code created during the Process Planning stage. Layup strategies, in addition to dog ears, stagger shifts, steering constraints, and starting points, represented the group of functions labeled as process optimization and ranked the highest in terms of priority for automation. The laminates resulting from the selected parameters are evaluated through the occurrences of principal defect metrics such as fiber gaps, overlaps, angle deviation and steering violations. This document presents an automated software solution to the layup strategy and starting point selection phase of process planning. A series of ply scenarios are generated with variations of these ply parameters and evaluated according to a set of metrics entered by the Process Planner. These metrics are generated through use of the Analytical Hierarchy Process (AHP), where relative importance between each of the fiber features are defined. The ply scenarios are selected which reduce the overall fiber feature scores based on the defects the Process Planner wishes to minimize. An extensive evaluation of the ply scoring algorithms was performed with various tool surfaces. The evaluation provided insight into the interaction of the different layup strategies and the underlying geometry of the tool surface. Additionally, the relationship of the various fiber defects with the tool surface curvature were also investigated. Furthermore, the document covers the incorporation of inspection data for the validation of fiber defect predictions, in addition to a new laminate evaluation tech-nique which begins to consider how undesirable features within the composite may interact through the thickness of the laminate. Utilizing the in-depth understanding of fiber path generation with the evaluation methods within CAPP, a new dynamic layup strategy is devised which considers the assigned relative importance of various defect types to take some of the guess work out of layup strategy selection, which can be a challenging task when complex tool geometry is involved

Comprehensive Process Planning Optimization Framework for Automated Fiber Placement

Advanced composite materials came about in 1966 and have since been widely used due to the possibility of superior structural performance while also achieving weight reductions. Such opportunities have led to composite materials being used to fabricate complex components, often in the aerospace sector. Most components, especially in aviation, are on a large scale and are outside the capabilities of traditional composite manufacturing techniques. Traditional manufacturing methods are also labor intensive, time consuming, have a high level of material scrap, and are prone to human error. This has led to the need for innovative manufacturing solutions to withstand the ever-increasing throughput requirement. One driving technique is Automated Fiber Placement (AFP) which is a relatively new manufacturing technique that has rapidly evolved since its commercial inception in the 1980s. AFP gives industries the ability to manufacture large parts with high speed, repeatability, and process quality. However, even with the state-of-the-art machines and process controls, the AFP process is still far from perfected. With the advent of Industry 4.0, many manufacturing sectors have begun the exploration into the use of smart and digital manufacturing with implementation of machine learning. However, AFP manufacturing, and the remaining composite manufacturing sector, has yet to explore the philosophies of the future of manufacturing. Rather, siloed efforts have been enacted to advance each of the technical challenges associated with AFP resulting in an open loop system that is difficult to optimize on a global level. Such efforts also restrict the possibility of increased manufacturing throughput due to systems operating in suboptimal configurations. To overcome this hurdle, an integrated data flow is enacted that combines the design, process planning, manufacturing, and inspection pillars of AFP into a holistic workflow. This is enabled by employing industry level tools combined with efficient and versatile modeling techniques. The models then allow for an informed analysis that considers the combination of multiple AFP lifecycle trades. With this streamlined workflow, multiple optimization algorithms are developed to determine the globally optimal manufacturing plan to generate a structure with AFP. The combination of these methodologies into a common tool creates a comprehensive AFP process planning optimization. The modeling and optimization framework is applied to a doubly curved tool surface case study. Results demonstrate the effectiveness of the presented framework in terms of manufacturing defect reduction and process efficiency

Kinematic Modeling Of An Automated Laser Line Scanning System

This research work describes the geometric coordinate transformation in an automated laser line scanning system caused by movements required for scanning a component surface. The elements of an automated laser scanning system (robot, laser line scanner, and the component coordinate system) function as a mechanical linkage to obtain a trajectory on a component surface. This methodology solves the forward kinematics, derives the component surface, and uses inverse kinematic equations to characterize the movement of the entire automated scanning system on point trajectory. To reach a point on the component, joint angles of robot have been calculated. As a result, trajectory path is obtained. This obtained robot poses on point trajectory of the component surface can be used as an input for future work that aims to develop optimal scan paths to collect “best” point cloud data sets. This work contributes in laser scanning inspection of component surfaces in manufacturing, remanufacturing, and reverse engineering applications