A Simplified Phase Display System for 3D Surface Measurement and Abnormal Surface Pattern Detection

Today’s engineering products demand increasingly strict tolerances. The shape of a machined surface plays a critical role to the desired functionality of a product. Even a small error can be the difference between a successful product launch and a major delay. It is important to develop tools that confirm the quality and accuracy of manufactured products. The key to assessing the quality is robust measurement and inspection tools combined with advanced analysis. This research is motivated by the goals of 1) developing an advanced optical metrology system that provides accurate 3D profiles of target objects with curvature and irregular texture and 2) developing algorithms that can recognize and extract meaningful surface features with the consideration of machining process information. A new low cost measurement system with a simple coherent interferometric fringe projection system is developed. Comparing with existing optical measurement systems, the developed system generates fringe patterns on object surface through a pair of fiber optics that have a relatively simple and flexible configuration. Three-dimensional measurements of a variety of surfaces with curvatures demonstrate the applicability and flexibility of the developed system. An improved phase unwrapping algorithm based on a flood fill method is developed to enhance the performance of image processing. The developed algorithm performs phase unwrapping under the guidance of a hybrid quality map that is generated by considering the quality of both acquired original intensity images and the calculated wrapped phase map. Advances in metrology systems enable engineers to obtain a large amount of surface information. A systematic framework for surface shape characterization and abnormal pattern detection is proposed to take the advantage of the availability of high definition surface measurements through advanced metrology systems. The proposed framework evaluates a measured surface in two stages. The first step focuses on the extraction of general shape (e.g., surface form) from measurement for surface functionality evaluation and process monitoring. The second step focuses on the extraction of application specific surface details with the consideration of process information (e.g., surface waviness). Applications of automatic abnormal surface pattern detection have been demonstrated. In summary, this research focuses on two core areas: 1) developing metrology system that is capable of measuring engineered surfaces accurately; 2) proposing a methodology that can extract meaningful information from high definition measurements with consideration of process information and product functionality.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/136999/1/xinweng_1.pd

A Multi Camera and Multi Laser Calibration Method for 3D Reconstruction of Revolution Parts

This paper describes a method for calibrating multi camera and multi laser 3D triangulation systems, particularly for those using Scheimpflug adapters. Under this configuration, the focus plane of the camera is located at the laser plane, making it difficult to use traditional calibration methods, such as chessboard pattern-based strategies. Our method uses a conical calibration object whose intersections with the laser planes generate stepped line patterns that can be used to calculate the camera-laser homographies. The calibration object has been designed to calibrate scanners for revolving surfaces, but it can be easily extended to linear setups. The experiments carried out show that the proposed system has a precision of 0.1 mm

Robotic Automation of Turning Machines in Fenceless Production: A Planning Toolset for Economic-based Selection Optimization between Collaborative and Classical Industrial Robots

Ursprünglich wurden Industrieroboter hauptsächlich hinter Schutzzäunen betrieben, um den Sicherheitsanforderungen gerecht zu werden. Mit der Flexibilisierung der Produktion wurden diese scharfen Trennbereiche zunehmend aufgeweicht und externe Sicherheitstechnik, wie Abstandssensoren, genutzt, um Industrieroboter schutzzaunlos zu betreiben. Ausgehend vom Gedanken dieser Koexistenz bzw. Kooperation wurde die Sicherheitssensorik in den Roboter integriert, um eine wirkliche Kollaboration zu ermöglichen. Diese sogenannten kollaborierenden Roboter, oder Cobots, eröffnen neue Applikationsfelder und füllen somit die bestehenden Automatisierungslücken. Doch welche Automatisierungsvariante ist aus wirtschaftlichen Gesichtspunkten die geeignetste? Bisherige Forschung untersucht zum Großteil isoliert eine der beiden Technologien, ohne dabei einen Systemvergleich hinsichtlich technologischer Spezifika und Wirtschaftlichkeit anzustellen. Daher widmet sich diese Dissertation einer Methodik zum wirtschaftlichen Vergleich von kollaborierenden Robotern und Industrierobotern in schutzzaunlosen Maschinenbeladungssystemen. Besonderer Fokus liegt dabei auf dem Herausarbeiten der technischen Faktoren, die die Wirtschaftlichkeit maßgeblich beeinflussen, um ein Systemverständnis der wirtschaftlichen Struktur beider Robotertechnologievarianten zu erhalten. Zur Untersuchung werden die Inhalte eines solchen Planungsvorhabens beschrieben, kategorisiert, systematisiert und modularisiert. Auf wirtschaftlicher Seite wird ein geeignetes Optimierungsmodell vorgestellt, während auf technischer Seite vor allem die Machbarkeit hinsichtlich Greifbarkeit, Layoutplanung, Robotergeschwindigkeiten und Zykluszeitbestimmung untersucht wird. Mit deduktiven, simulativen, empirischen und statistischen Methoden wird das Systemverhalten für die einzelnen Planungsinhalte analysiert, um die Gesamtwirtschaftlichkeit mit einem Minimum an Investment,- Produktions,- und Zykluszeitinformationen a priori vorhersagen zu können. Es wird gezeigt, dass durch einen Reverse Engineering Ansatz die notwendigen Planungsdaten, im Sinne von Layoutkomposition, Robotergeschwindigkeiten und Taktzeiten, mithilfe von Frontloading zu Planungsbeginn zur Verfügung gestellt werden können. Dabei dient der Kapitalwert als wirtschaftliche Bewertungsgrundlage, dessen Abhängigkeit vom Mensch-Roboter-Interaktionsgrad in einem Vorteilhaftigkeitsdiagramm für die einzelnen Technologiealternativen dargestellt werden kann. Wirtschaftlich fundierte Entscheidungen können somit auf quantitiativer Basis getroffen werden.:1. Introduction 25 1.1 Research Domain 25 1.2 Research Niche 26 1.3 Research Structure 28 2. State of the Art and Research 31 2.1 Turning Machines and Machine Tending 31 2.1.1 Tooling Machine Market Trends and Machine Tending Systems 31 2.1.2 Workpiece System 34 2.1.3 Machine System 36 2.1.4 Logistics System 39 2.1.5 Handling System 41 2.2 Robotics 43 2.2.1 Robot Installation Development and Application Fields 43 2.2.2 Fenceless Industrial and Collaborative Robots 48 2.2.3 Robot Grippers 55 2.3 Planning and Evaluation Methods 56 2.3.1 Planning of General and Manual Workstations 56 2.3.2 Cell Planning for Fully Automated and Hybrid Robot Systems 59 2.3.3 Robot Safety Planning 61 2.3.4 Economic Evaluation Methods 70 2.4 Synthesis - State of the Art and Research 71 3. Solution Approach 77 3.1 Need for Research and General Solution Approach 77 3.2 Use Case Delineation and Planning Focus 80 3.3 Economic Module – Solution Approach 86 3.4 Gripper Feasibility Module – Solution Approach 89 3.5 Rough Layout Discretization Model – Solution Approach 94 3.6 Cycle Time Estimation Module – Solution Approach 97 3.7 Collaborative Speed Estimation Module – Solution Approach 103 3.7.1 General Approach 103 3.7.2 Case 1: Quasi-static Contact with Hand 107 3.7.3 Case 2: Transient Contact with Hand 109 3.7.4 Case 3: Transient Contact with Shoulder 111 3.8 Synthesis – Solution Approach 114 4. Module Development 117 4.1 Economic Module – Module Development 117 4.1.1 General Approach 117 4.1.2 Calculation Scheme for Manual Operation 117 4.1.3 Calculation Scheme for Collaborative Robots 118 4.1.4 Calculation Scheme for Industrial Robots 120 4.2 Gripper Feasibility Module – Module Development 121 4.3 Rough Layout Discretization Module – Module Development 122 4.3.1 General Approach 122 4.3.2 Two-Dimensional Layout Pattern 123 4.3.3 Three-Dimensional Layout Pattern 125 4.4 Cycle Time Estimation Module – Module Development 126 4.4.1 General Approach 126 4.4.2 Reachability Study 127 4.4.3 Simulation Results 128 4.5 Collaborative Speed Estimation Module – Module Development 135 4.5.1 General Approach 135 4.5.2 Case 1: Quasi-static Contact with Hand 135 4.5.3 Case 2: Transient Contact with Hand 143 4.5.4 Case 3: Transient Contact with Shoulder 145 4.6 Synthesis – Module Development 149 5. Practical Verification 155 5.1 Use Case Overview 155 5.2 Gripper Feasibility 155 5.3 Layout Discretization 156 5.4 Collaborative Speed Estimation 157 5.5 Cycle Time Estimation 158 5.6 Economic Evaluation 160 5.7 Synthesis – Practical Verification 161 6. Results and Conclusions 165 6.1 Scientific Findings and Results 165 6.2 Critical Appraisal and Outlook 173Initially, industrial robots were mainly operated behind safety fences to account for the safety requirements. With production flexibilization, these sharp separation areas have been increasingly softened by utilizing external safety devices, such as distance sensors, to operate industrial robots fenceless. Based on this idea of coexistence or cooperation, safety technology has been integrated into the robot to enable true collaboration. These collaborative robots, or cobots, open up new application fields and fill the existing automation gap. But which automation variant is most suitable from an economic perspective? Present research dealt primarily isolated with one technology without comparing these systems regarding technological and economic specifics. Therefore, this doctoral thesis pursues a methodology to economically compare collaborative and industrial robots in fenceless machine tending systems. A particular focus lies on distilling the technical factors that mainly influence the profitability to receive a system understanding of the economic structure of both robot technology variants. For examination, the contents of such a planning scheme are described, categorized, systematized, and modularized. A suitable optimization model is presented on the economic side, while the feasibility regarding gripping, layout planning, robot velocities, and cycle time determination is assessed on the technical side. With deductive, simulative, empirical, and statistical methods, the system behavior of the single planning entities is analyzed to predict the overall profitability a priori with a minimum of investment,- production,- and cycle time information. It is demonstrated that the necessary planning data, in terms of layout composition, robot velocities, and cycle times, can be frontloaded to the project’s beginning with a reverse engineering approach. The net present value serves as the target figure, whose dependency on the human-robot interaction grade can be illustrated in an advantageousness diagram for the individual technical alternatives. Consequently, sound economic decisions can be made on a quantitative basis.:1. Introduction 25 1.1 Research Domain 25 1.2 Research Niche 26 1.3 Research Structure 28 2. State of the Art and Research 31 2.1 Turning Machines and Machine Tending 31 2.1.1 Tooling Machine Market Trends and Machine Tending Systems 31 2.1.2 Workpiece System 34 2.1.3 Machine System 36 2.1.4 Logistics System 39 2.1.5 Handling System 41 2.2 Robotics 43 2.2.1 Robot Installation Development and Application Fields 43 2.2.2 Fenceless Industrial and Collaborative Robots 48 2.2.3 Robot Grippers 55 2.3 Planning and Evaluation Methods 56 2.3.1 Planning of General and Manual Workstations 56 2.3.2 Cell Planning for Fully Automated and Hybrid Robot Systems 59 2.3.3 Robot Safety Planning 61 2.3.4 Economic Evaluation Methods 70 2.4 Synthesis - State of the Art and Research 71 3. Solution Approach 77 3.1 Need for Research and General Solution Approach 77 3.2 Use Case Delineation and Planning Focus 80 3.3 Economic Module – Solution Approach 86 3.4 Gripper Feasibility Module – Solution Approach 89 3.5 Rough Layout Discretization Model – Solution Approach 94 3.6 Cycle Time Estimation Module – Solution Approach 97 3.7 Collaborative Speed Estimation Module – Solution Approach 103 3.7.1 General Approach 103 3.7.2 Case 1: Quasi-static Contact with Hand 107 3.7.3 Case 2: Transient Contact with Hand 109 3.7.4 Case 3: Transient Contact with Shoulder 111 3.8 Synthesis – Solution Approach 114 4. Module Development 117 4.1 Economic Module – Module Development 117 4.1.1 General Approach 117 4.1.2 Calculation Scheme for Manual Operation 117 4.1.3 Calculation Scheme for Collaborative Robots 118 4.1.4 Calculation Scheme for Industrial Robots 120 4.2 Gripper Feasibility Module – Module Development 121 4.3 Rough Layout Discretization Module – Module Development 122 4.3.1 General Approach 122 4.3.2 Two-Dimensional Layout Pattern 123 4.3.3 Three-Dimensional Layout Pattern 125 4.4 Cycle Time Estimation Module – Module Development 126 4.4.1 General Approach 126 4.4.2 Reachability Study 127 4.4.3 Simulation Results 128 4.5 Collaborative Speed Estimation Module – Module Development 135 4.5.1 General Approach 135 4.5.2 Case 1: Quasi-static Contact with Hand 135 4.5.3 Case 2: Transient Contact with Hand 143 4.5.4 Case 3: Transient Contact with Shoulder 145 4.6 Synthesis – Module Development 149 5. Practical Verification 155 5.1 Use Case Overview 155 5.2 Gripper Feasibility 155 5.3 Layout Discretization 156 5.4 Collaborative Speed Estimation 157 5.5 Cycle Time Estimation 158 5.6 Economic Evaluation 160 5.7 Synthesis – Practical Verification 161 6. Results and Conclusions 165 6.1 Scientific Findings and Results 165 6.2 Critical Appraisal and Outlook 17

Three-Dimensional Shape Measurements of Specular Objects Using Phase-Measuring Deflectometry

The fast development in the fields of integrated circuits, photovoltaics, the automobile industry, advanced manufacturing, and astronomy have led to the importance and necessity of quickly and accurately obtaining three-dimensional (3D) shape data of specular surfaces for quality control and function evaluation. Owing to the advantages of a large dynamic range, non-contact operation, full-field and fast acquisition, high accuracy, and automatic data processing, phase-measuring deflectometry (PMD, also called fringe reflection profilometry) has been widely studied and applied in many fields. Phase information coded in the reflected fringe patterns relates to the local slope and height of the measured specular objects. The 3D shape is obtained by integrating the local gradient data or directly calculating the depth data from the phase information. We present a review of the relevant techniques regarding classical PMD. The improved PMD technique is then used to measure specular objects having discontinuous and/or isolated surfaces. Some influential factors on the measured results are presented. The challenges and future research directions are discussed to further advance PMD techniques. Finally, the application fields of PMD are briefly introduce

Smart optical coordinate and surface metrology

Manufacturing has recently experienced increased adoption of optimised and fast solutions for checking product quality during fabrication, allowing for manufacturing times and costs to be significantly reduced. Due to the integration of machine learning algorithms, advanced sensors and faster processing systems, smart instruments can autonomously plan measurement pipelines, perform decisional tasks and trigger correctional actions as required. In this paper, we summarise the state of the art in smart optical metrology, covering the latest advances in integrated intelligent solutions in optical coordinate and surface metrology, respectively for the measurement of part geometry and surface texture. Within this field, we include the use of a priori knowledge and implementation of machine learning algorithms for measurement planning optimisation. We also cover the development of multi-sensor and multi-view instrument configurations to speed up the measurement process, as well as the design of novel feedback tools for measurement quality evaluation

A Single Camera Unit-Based Three-Dimensional Surface Imaging Technique

The main objective of this study is to develop a single-camera unit-based three-dimensional surface imaging technique that could be used to reduce the disparity error in three-dimensional (3D) image reconstruction and simplify the calibration process of the imaging system. The current advanced stereoscopic 3D imaging system uses a pair of imaging devices (e.g., complementary metal-oxide semiconductor (CMOS) or charge-coupled device (CCD)), imaging lenses, and other accessories (e.g., light sources, polarizing filters) and diffusers.) To reconstruct the 3D scene, the system needs to calibrate the camera and compute a disparity map. However, in most cases in the industry, a pair of imaging devices is not ideally identical, so it is a necessary step to finely adjust and compensate for camera orientation, lens focal length, and intrinsic parameters for each camera. More importantly, conventional stereoscopic systems may respond differently to incident light reflected from the target surface. It is possible for the pixel information in the left and right images to be slightly different. This results in an increase in disparity error, even though the stereovision system is calibrated and compensated for rotation and vertical offsets between two cameras. This thesis aims to solve the aforementioned challenges by proposing a new stereo vision scheme based on only one camera to obtain target 3D data by 3D image reconstruction of two images obtained from two different camera positions

Robot guidance using machine vision techniques in industrial environments: A comparative review

In the factory of the future, most of the operations will be done by autonomous robots that need visual feedback to move around the working space avoiding obstacles, to work collaboratively with humans, to identify and locate the working parts, to complete the information provided by other sensors to improve their positioning accuracy, etc. Different vision techniques, such as photogrammetry, stereo vision, structured light, time of flight and laser triangulation, among others, are widely used for inspection and quality control processes in the industry and now for robot guidance. Choosing which type of vision system to use is highly dependent on the parts that need to be located or measured. Thus, in this paper a comparative review of different machine vision techniques for robot guidance is presented. This work analyzes accuracy, range and weight of the sensors, safety, processing time and environmental influences. Researchers and developers can take it as a background information for their future works