Locally Adaptive Stereo Vision Based 3D Visual Reconstruction

abstract: Using stereo vision for 3D reconstruction and depth estimation has become a popular and promising research area as it has a simple setup with passive cameras and relatively efficient processing procedure. The work in this dissertation focuses on locally adaptive stereo vision methods and applications to different imaging setups and image scenes. Solder ball height and substrate coplanarity inspection is essential to the detection of potential connectivity issues in semi-conductor units. Current ball height and substrate coplanarity inspection tools are expensive and slow, which makes them difficult to use in a real-time manufacturing setting. In this dissertation, an automatic, stereo vision based, in-line ball height and coplanarity inspection method is presented. The proposed method includes an imaging setup together with a computer vision algorithm for reliable, in-line ball height measurement. The imaging setup and calibration, ball height estimation and substrate coplanarity calculation are presented with novel stereo vision methods. The results of the proposed method are evaluated in a measurement capability analysis (MCA) procedure and compared with the ground-truth obtained by an existing laser scanning tool and an existing confocal inspection tool. The proposed system outperforms existing inspection tools in terms of accuracy and stability. In a rectified stereo vision system, stereo matching methods can be categorized into global methods and local methods. Local stereo methods are more suitable for real-time processing purposes with competitive accuracy as compared with global methods. This work proposes a stereo matching method based on sparse locally adaptive cost aggregation. In order to reduce outlier disparity values that correspond to mis-matches, a novel sparse disparity subset selection method is proposed by assigning a significance status to candidate disparity values, and selecting the significant disparity values adaptively. An adaptive guided filtering method using the disparity subset for refined cost aggregation and disparity calculation is demonstrated. The proposed stereo matching algorithm is tested on the Middlebury and the KITTI stereo evaluation benchmark images. A performance analysis of the proposed method in terms of the I0 norm of the disparity subset is presented to demonstrate the achieved efficiency and accuracy.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

Height inspection of wafer bumps without explicit 3D reconstruction.

by Dong, Mei.Thesis (M.Phil.)--Chinese University of Hong Kong, 2007.Includes bibliographical references (leaves 83-90).Abstracts in English and Chinese.INTRODUCTION --- p.1Chapter 1.1 --- Bump Height Inspection --- p.1Chapter 1.2 --- Our Height Inspection System --- p.2Chapter 1.3 --- Thesis Outline --- p.3BACKGROUND --- p.5Chapter 2.1 --- Wafer Bumps --- p.5Chapter 2.2 --- Common Defects of Wafer Bumps --- p.7Chapter 2.3 --- Traditional Methods for Bump Inspection --- p.11BIPLANAR DISPARITY METHOD --- p.22Chapter 3.1 --- Problem Nature --- p.22Chapter 3.2 --- System Overview --- p.25Chapter 3.3 --- Biplanar Disparity Matrix D --- p.30Chapter 3.4 --- Planar Homography --- p.36Chapter 3.4.1 --- Planar Homography --- p.36Chapter 3.4.2 --- Homography Estimation --- p.39Chapter 3.5 --- Harris Corner Detector --- p.45Chapter 3.6 --- Experiments --- p.47Chapter 3.6.1 --- Synthetic Experiments --- p.47Chapter 3.6.2 --- Real image experiment --- p.52Chapter 3.7 --- Conclusion and problems --- p.61PARAPLANAR DISPARITY METHOD --- p.62Chapter 4.1 --- The Parallel Constraint --- p.63Chapter 4.2 --- Homography estimation --- p.66Chapter 4.3. --- Experiment: --- p.69Chapter 4.3.1 --- Synthetic Experiment: --- p.69Chapter 4.3.2 --- Real Image Experiment: --- p.74CONCLUSION AND FUTURE WORK --- p.80Chapter 5.1 --- Summary of the contributions --- p.80Chapter 5.2 --- Future Work --- p.81Publication related to this work: --- p.83BIBLIOGRAPHY --- p.8

3D inspection of wafer bump quality without explicit 3D reconstruction.

Zhao Yang.Thesis (M.Phil.)--Chinese University of Hong Kong, 2004.Includes bibliographical references (leaves 87-95).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Objectives of the Thesis --- p.1Chapter 1.2 --- Wafer bumping inspection by using Biplanar Disparity approach --- p.2Chapter 1.3 --- Thesis Outline --- p.4Chapter 2 --- Background --- p.5Chapter 2.1 --- What is wafer bump? --- p.5Chapter 2.1.1 --- Common defects of wafer bump --- p.6Chapter 2.1.2 --- Literature review on exist wafer bump inspection method --- p.11Chapter 3 --- Model 1: the one camera model-Homography approach --- p.21Chapter 3.1 --- The introduction of the theoretical base of model 1 --- p.21Chapter 3.1.1 --- The objective of model 1 --- p.21Chapter 3.1.2 --- Desires --- p.22Chapter 3.1.3 --- Some background knowledge on Homography --- p.22Chapter 3.2 --- "Model 1- ""Pseudo Homography"" Approach" --- p.24Chapter 3.2.1 --- The description of the configuration of model 1 --- p.24Chapter 3.2.2 --- The condition of pseudo Homography --- p.25Chapter 3.2.3 --- The formation of pseudo Homgraphy H --- p.26Chapter 3.3 --- Methodology of treatment of the answer set --- p.32Chapter 3.3.1 --- Singular Value Decomposition-SVD --- p.32Chapter 3.3.2 --- The Robust Estimation --- p.33Chapter 3.3.3 --- Some experimental results by using manmade Ping Pang balls to test SVD[31] and Robust Estimation [24] --- p.35Chapter 3.3.4 --- the measurement of the Homography matrix answer set --- p.37Chapter 3.4 --- Preliminary experiment about model 1 --- p.43Chapter 3.5 --- Problems unsolved --- p.47Chapter 4 --- Model 2: the two camera model-Biplanar Disparity approach --- p.48Chapter 4.1 --- Theoretical Background --- p.48Chapter 4.1.1 --- the linearization of Homography matrix changes --- p.49Chapter 4.1.2 --- Problem Nature --- p.51Chapter 4.1.3 --- Imaging system setup --- p.52Chapter 4.1.4 --- Camera Calibration[13] --- p.52Chapter 4.2 --- Methodology --- p.54Chapter 4.2.1 --- Invariance measure --- p.54Chapter 4.2.2 --- The Geometric meaning of the Biplanar Disparity matrix --- p.58Chapter 4.3 --- RANSAC-Random Sample Consensus --- p.64Chapter 4.3.1 --- finding Homography matrix by using RANSAC[72] [35] --- p.64Chapter 4.3.2 --- finding Fundamental matrix by using RANSAC[73] [34] --- p.65Chapter 4.4 --- Harris Corner detection --- p.65Chapter 5 --- Simulation and experimental results --- p.67Chapter 5.1 --- Simulation experiments --- p.67Chapter 5.1.1 --- Preliminary experiments --- p.67Chapter 5.1.2 --- Specification for the synthetic data system --- p.71Chapter 5.1.3 --- Allowed error in the experiment --- p.71Chapter 5.2 --- Real images experiments --- p.72Chapter 5.2.1 --- Experiment instrument --- p.72Chapter 5.2.2 --- The Inspection Procedure --- p.74Chapter 5.2.3 --- Images grabbed under above system --- p.75Chapter 5.2.4 --- Experimental Results --- p.81Chapter 6 --- CONCLUSION AND FUTURE WORKS --- p.83Chapter 6.1 --- Summary on the contribution of my work --- p.83Chapter 6.2 --- Some Weakness of The Method --- p.84Chapter 6.3 --- Future Works and Further Development --- p.84Chapter 6.3.1 --- About the synthetic experiment --- p.84Chapter 6.3.2 --- About the real image experiment --- p.85Bibliography --- p.8

CHARACTERIZATION OF ENGINEERED SURFACES

In the recent years there has been an increasing interest in manufacturing products where surface topography plays a functional role. These surfaces are called engineered surfaces and are used in a variety of industries like semi conductor, data storage, micro- optics, MEMS etc. Engineered products are designed, manufactured and inspected to meet a variety of specifications such as size, position, geometry and surface finish to control the physical, chemical, optical and electrical properties of the surface. As the manufacturing industry strive towards shrinking form factor resulting in miniaturization of surface features, measurement of such micro and nanometer scale surfaces is becoming more challenging. Great strides have been made in the area of instrumentation to capture surface data, but the area of algorithms and procedures to determine form, size and orientation information of surface features still lacks the advancement needed to support the characterization requirements of R&D and high volume manufacturing. This dissertation addresses the development of fast and intelligent surface scanning algorithms and methodologies for engineered surfaces to determine form, size and orientation of significant surface features. Object recognition techniques are used to identify the surface features and CMM type fitting algorithms are applied to calculate the dimensions of the features. Recipes can be created to automate the characterization and process multiple features simultaneously. The developed methodologies are integrated into a surface analysis toolbox developed in MATLAB environment. The deployment of the developed application on the web is demonstrated

Digital fringe projection system for measuring warpage of painted and unpainted PBGAs and boards and FEA studies of PBGA warpage

Improvements in chip package technologies have led to smaller package sizes and higher density circuitry that require superior reliability of chip packages. One of the crucial factors affecting the reliability of chip packages is warpage which primarily occurs during the reflow process. Because warpage may cause serious reliability problems such as solder bump failure and die cracking, warpage control has become a crucial task. Advancements in warpage measurement and prediction would provide important steps toward addressing this concern. Among the various warpage measurement techniques, fringe projection techniques (i.e., laser fringe projection (LFP) and digital fringe projection (DFP)) have emerged as recent trends due to their non-contact, full-field, and high-resolution (for small viewing area) capabilities for measuring the warpage of chip packages and boards (i.e., printed wiring boards (PWBs) and PWB assemblies). In this research, the measurement capabilities of a LFP system were improved by reducing its laser speckle noise and post-processing time, and a novel DFP system for measuring the warpage of painted and unpainted chip packages and boards was developed. Also, parametric studies were performed to predict the warpage of plastic ball grid array packages affected by four geometric factors. Finally, a guideline that manufacturing engineers can use for selecting the most suitable warpage measurement technique for their particular application was developed. The results of this study will help to improve the yields and reliability of chip packages and boards, reduce the manufacturing costs and time to market for chip packages and boards and ultimately reduce the prices of end-products.Ph.D

A Review and Analysis of Automatic Optical Inspection and Quality Monitoring Methods in Electronics Industry

Electronics industry is one of the fastest evolving, innovative, and most competitive industries. In order to meet the high consumption demands on electronics components, quality standards of the products must be well-maintained. Automatic optical inspection (AOI) is one of the non-destructive techniques used in quality inspection of various products. This technique is considered robust and can replace human inspectors who are subjected to dull and fatigue in performing inspection tasks. A fully automated optical inspection system consists of hardware and software setups. Hardware setup include image sensor and illumination settings and is responsible to acquire the digital image, while the software part implements an inspection algorithm to extract the features of the acquired images and classify them into defected and non-defected based on the user requirements. A sorting mechanism can be used to separate the defective products from the good ones. This article provides a comprehensive review of the various AOI systems used in electronics, micro-electronics, and opto-electronics industries. In this review the defects of the commonly inspected electronic components, such as semiconductor wafers, flat panel displays, printed circuit boards and light emitting diodes, are first explained. Hardware setups used in acquiring images are then discussed in terms of the camera and lighting source selection and configuration. The inspection algorithms used for detecting the defects in the electronic components are discussed in terms of the preprocessing, feature extraction and classification tools used for this purpose. Recent articles that used deep learning algorithms are also reviewed. The article concludes by highlighting the current trends and possible future research directions.Framework of the IQONIC Project; European Union’s Horizon 2020 Research and Innovation Program

An experimental assessment of computational fluid dynamics predictive accuracy for electronic component operational temperature

Ever-rising Integrated Circuit (IC) power dissipation, combined with reducing product development cycles times, have placed increasing reliance on the use of Computational Fluid Dynamics (CFD) software for the thermal analysis of electronic equipment. In this study, predictive accuracy is assessed for board-mounted electronic component heat transfer using both a CFD code dedicated to the thermal analysis of electronics, Flotherm, and a general-purpose CFD code, Fluent. Using Flotherm, turbulent flow modelling approaches typically employed for the analysis of electronics cooling, namely algebraic mixing length and two-equation high-Reynolds number k-e models, are assessed. As shown, such models are not specific for the analysis of forced airflows over populated electronic boards, which are typically classified as low-Reynolds number flows. The potential for improved predictive accuracy is evaluated using candidate turbulent flow models more suited to such flows, namely a one-equation SpalartAllmaras model, two-layer zonal model and two equation SST k-co model, all implemented in Fluent. Numerical predictions are compared with experimental benchmark data for a range of componentboard topologies generating different airflow phenomena and varying degrees of component thermal interaction. Test case complexity is incremented in controlled steps, from single board-mounted components in free convection, to forced air-cooled, multi-component board configurations. Apart from the prediction of component operational temperature, the application of CFD analysis to the design of electronic component reliability screens and convective solder reflow temperature profiles is also investigated. Benchmark criteria are based on component junction temperature and component-board surface temperature profiles, measured using thermal test chips and infrared thermography respectively. This data is supplemented by experimental visualisations of the forced airflows over the boards, which are used to help assess predictive accuracy. Component numerical modelling is based on nominal package dimensions and material thermal properties. To eliminate potential numerical modelling uncertainties, both the test component geometry and structural integrity are assessed using destructive and non-destructive testing. While detailed component modelling provides the à priori junction temperature predictions, the capability of compact thermal models to predict multi-mode component heat transfer is also assessed. In free convection, component junction temperature predictions for an in-line array of fifteen boardmounted components are within ±5°C or 7% of measurement. Predictive accuracy decays up to ±20°C or 35% in forced airflows using the k-e flow model. Furthermore, neither the laminar or k-e turbulent flow model accurately resolve the complete flow fields over the boards, suggesting the need for a turbulence model capable of modelling transition. Using a k-co model, significant improvements in junction temperature prediction accuracy are obtained, which are associated with improved prediction of both board leading edge heat transfer and component thermal interaction. Whereas with the k-e flow model, prediction accuracy would only be sufficient for the early to intermediate phase of a thermal design process, the use of the k-co model would enable parametric analysis of product thermal performance to be undertaken with greater confidence. Such models would also permit the generation of more accurate temperature boundary conditions for use in Physics-of-Failure (PoF) based component reliability prediction methods. The case is therefore made for vendors of CFD codes dedicated to the thermal analysis of electronics to consider the adoption of eddy viscosity turbulence models more suited to the analysis of component heat transfer. While this study ultimately highlights that electronic component operational temperature needs to be experimentally measured to quality product thermal performance and reliability, the use of such flow models would help reduce the current dependency on experimental prototyping. This would not only enhance the potential of CFD as a design tool, but also its capability to provide detailed insight into complex multi-mode heat transfer, that would otherwise be difficult to characterise experimentally

Methoden zur Qualifizierung von CCD-Zeilenkameras als Messgerät für die Farb- und 3D-Messung

Kamerabasierte Systeme werden in zunehmenden Maße für messtechnische Aufgaben im industriellen Umfeld eingesetzt. Deren Qualifizierung hinsichtlich der Tauglichkeit für die jeweilige Messaufgabe ist ein anspruchsvolles Feld, welches ein ganzheitliches Verständnis der Kamerasysteme voraussetzt. Im Falle kontinuierlich transportierter Objekte eignen sich zur messtechnischen Erfassung insbesondere Zeilenkameras. Der Grund liegt in der hohen optischen Auflösung und der kontinuierlichen Abtastung des Objektes. In dieser Arbeit werden zwei Zeilenkamerasysteme der Chromasens GmbH wissenschaftlich untersucht und Methoden zu deren messtechnischen Qualifizierung entwickelt. Betrachtet werden die multispektrale Zeilenkamera truePIXA, welche als bildgebendes Farbmessgerät eingesetzt wird, sowie die Stereo-Zeilenkamera 3DPIXA, welche mittels Triangulation die Vermessung von 3D-Oberflächentopografien ermöglicht. Beide Systeme werden durch mehrere Teilkameras realisiert, die Auswertemethoden sind jedoch komplementär. Der in den Kamerasystemen eingesetzte CCD-Zeilensensor wird angelehnt an den EMVA 1288 Standard charakterisiert, um ein messdatengestütztes Simulationsmodell der multispektralen Zeilenkamera entwickeln zu können. Im nächsten Schritt wird ein Verfahren dargestellt, welches die präzise Vermessung der spektralen Empfindlichkeiten der zwölf Kanäle der multispektralen Zeilenkamera erlaubt. Das Simulationsmodell der multispektralen Zeilenkamera wird eingesetzt, um Sensitivitätsanalysen durchzuführen. Durch den Einsatz eines stochastischen Musters wird die relative Änderung der MTF der Stereo-Zeilenkamera über das Messvolumen charakterisiert. Insbesondere wird der Einfluss der optischen Defokussierung, der Größe des zur Korrelation verwendeten Suchfensters und des Rauschens der Bilddaten auf das Rauschen der 3D-Messung untersucht. Zur Betrachtung der Abhängigkeit der Korrelationsfenstergröße und der Defokussierung, wird der Begriff der frequenzabhängigen Schärfentiefe eingeführt. Dieses Vorgehen führte zu dem Ergebnis, dass das Messrauschen stark von dem Verhältnis der Größe des Korrelationsfensters zur auftretenden Wellenlänge der Modulation im Bild abhängt

Cumulative index to NASA Tech Briefs, 1970-1975

Tech briefs of technology derived from the research and development activities of the National Aeronautics and Space Administration are presented. Abstracts and indexes of subject, personal author, originating center, and tech brief number for the 1970-1975 tech briefs are presented