Marking parts to aid robot vision

The premarking of parts for subsequent identification by a robot vision system appears to be beneficial as an aid in the automation of certain tasks such as construction in space. A simple, color coded marking system is presented which allows a computer vision system to locate an object, calculate its orientation, and determine its identity. Such a system has the potential to operate accurately, and because the computer shape analysis problem has been simplified, it has the ability to operate in real time

Decision-making and problem-solving methods in automation technology

The state of the art in the automation of decision making and problem solving is reviewed. The information upon which the report is based was derived from literature searches, visits to university and government laboratories performing basic research in the area, and a 1980 Langley Research Center sponsored conferences on the subject. It is the contention of the authors that the technology in this area is being generated by research primarily in the three disciplines of Artificial Intelligence, Control Theory, and Operations Research. Under the assumption that the state of the art in decision making and problem solving is reflected in the problems being solved, specific problems and methods of their solution are often discussed to elucidate particular aspects of the subject. Synopses of the following major topic areas comprise most of the report: (1) detection and recognition; (2) planning; and scheduling; (3) learning; (4) theorem proving; (5) distributed systems; (6) knowledge bases; (7) search; (8) heuristics; and (9) evolutionary programming

Rotate and Hold and Scan (RAHAS): Structured Light Illumination for Use in Remote Areas

As a critical step after the discovery of material culture in the field, archaeologists have a need to document these findings with a slew of different physical measurements and photographs from varying perspectives. 3-D imaging is becoming increasingly popular as the primary documenting method to replace the plethora of tests and measurements, but for remote areas 3-D becomes more cumbersome due to physical and environmental constraints. The difficulty of using a 3-D imaging system in such environments is drastically lessened while using the RAHAS technique, since it acquires scans untethered to a computer. The goal of this thesis is to present the RAHAS Structured Light Illumination technique for 3-D image acquisition, and the performance of the RAHAS technique as a measurement tool for documentation of material culture on a field trip to the Rio Platano Biosphere in Honduras

A System for 3D Shape Estimation and Texture Extraction via Structured Light

Shape estimation is a crucial problem in the fields of computer vision, robotics and engineering. This thesis explores a shape from structured light (SFSL) approach using a pyramidal laser projector, and the application of texture extraction. The specific SFSL system is chosen for its hardware simplicity, and efficient software. The shape estimation system is capable of estimating the 3D shape of both static and dynamic objects by relying on a fixed pattern. In order to eliminate the need for precision hardware alignment and to remove human error, novel calibration schemes were developed. In addition, selecting appropriate system geometry reduces the typical correspondence problem to that of a labeling problem. Simulations and experiments verify the effectiveness of the built system. Finally, we perform texture extraction by interpolating and resampling sparse range estimates, and subsequently flattening the 3D triangulated graph into a 2D triangulated graph via graph and manifold methods

A Quantification of the 3D Modeling Capabilities of the Kinect Fustion Algorithm

In the eld of three-dimensional modeling, we continually struggle to quantify how closely the resulting model matches the physical object being represented. When precision measurements are required, they are often left to high-end, industrial systems. The aim of this thesis is to quantify the level of precision that can be obtained from commodity systems such as the Microsoft Kinect paired with the KinectFusion algorithm. Although the Kinect alone is considered a noisy sensor, the KinectFusion algorithm has shown the ability to build detailed surface models through the aggregation of depth information taken from multiple perspectives. This work represents the first rigorous validation of the three- dimensional modeling capabilities of the KinectFusion algorithm. One experiment is performed to measure the effects of key algorithm parameters such as resolution and range, while another is performed to measure the lower bounds at which objects can be detected and accurately modeled. The first experiment found that the KinectFusion algorithm reduced the uncertainty of the Kinect sensor alone from 10 mm to just 1.8 mm. Furthermore, the results of the second experiment demonstrate that the KinectFusion algorithm can detect surface deviations as little as 1.3 mm, but cannot accurately measure the deviation. Such results form an initial quantification of the KinectFusion algorithm, thus providing confidence about when and when not to utilize the KinectFusion algorithm for precision modeling. The hope is that this work will open the door for the algorithm to be used in real-world applications, such as alleviating the tedious visual surface inspections required for USAF aircraft

Improving Statistical Machine Translation Through N-best List

Statistical machine translation (SMT) is a method of translating from one natural language (NL) to another using statistical models generated from examples of the NLs. The quality of translation generated by SMT systems is competitive with other premiere machine translation (MT) systems and more improvements can be made. This thesis focuses on improving the quality of translation by re-ranking the n-best lists that are generated by modern phrase-based SMT systems. The n-best lists represent the n most likely translations of a sentence. The research establishes upper and lower limits of the translation quality achievable through re-ranking. Three methods of generating an n-gram language model (LM) from the n-best lists are proposed. Applying the LMs to re-ranking the n-best lists results in improvements of up to six percent in the Bi-Lingual Evaluation Understudy (BLEU) score of the translation

A STUDY OF MACHINE VISION IN THE AUTOMOTIVE INDUSTRY

With the growth of industrial automation, it has become increasingly important to validate the quality of every manufactured part during production. Until now, human visual inspection aided with hard tooling or machines have been the primary means to this end, but the speed of today's production lines, the complexity of production equipment and the highest standards of quality to which parts must adhere frequently, make the traditional methods of industrial inspection and control impractical, if not impossible. Subsequently, new solutions have been developed for the monitoring and control of industrial processes, in real­time. One such technology is the area of machine vision. After many years of research and development, computerised vision systems are now leaving the laboratory and are being used successfully in the factory environment. They are both robust and competitively priced as a sensing technique which has now opened up a whole new sector for automation. Machine vision systems are becoming an important integral part of the automotive manufacturing process, with applications ranging from inspection, classification, robot guidance, assembly verification through to process monitoring and control. Although the number of systems in current use is still relatively small, there can be no doubt, given the issues at stake, that the automotive industry will once again lead the way with the implementation of machine vision just as it has done robotic technology. The thesis considered the issue of machine vision and in particular, its deployment within the automotive industry. The thesis has presented work on machine vision for the prospective end-user and not the designer of such systems. It will provide sufficient background about the subject, to separate machine vision promises from reality and permit intelligent decisions regarding machine vision applications to be made. The initial part of the dissertation focussed on the strategic issues affecting the selection of machine vision at the planning stage, such as a listing of the factors to justify investment, the capability of the technology and type of problems that are associated with this relatively new but complex science. Though it is widely accepted that no two industrial machine vision systems are identical, knowledge of the basic fundamentals which underpin the structure of the technology in its application is presented. This work covered a structured description detailing typical hardware components such as camera technology, lighting systems, etc... which form an integral part of an industrial system and discussions regarding the criteria for selection are presented. To complement this work, a further section is specifically devoted to the bewildering array of vision software analysis techniques which are currently available today. A detailed description of the various techniques that are applied to images in order to make use of and understand the data contained within them are discussed and explored. Applications for machine vision fall into two main categories namely robotic guidance and inspection. Obviously within each category there are many further sub­groups. Within this context the latter part of the thesis reviews with a well structured description of several industrial case studies derived from the automotive industry, which illustrate that machine vision is capable of providing real time solutions to manufacturing based problems. In conclusion, despite the limited availability of industrially based machine vision systems, the success of implementation is not always guaranteed, as the technology imposes both technical limitations and introduce new human engineering considerations. By understanding the application and the implications of the technical requirements on both the "staging" and the "image-processing" power required of the machine vision system. The thesis has shown that the most significant elements of a successful application are indeed the lighting, optics, component design, etc... - the "Staging". From the case studies investigated, optimised "staging" has resulted in the need for less computing power in the machine vision system. Inevitably, greater computing power not only requires more time but is generally more expensive. The experience gained from the this project, has demonstrated that machine vision technology is a realistic alternative means of capturing data in real-time. Since the current limitations of the technology are well suited to the delivery process of the quality function within the manufacturing process

Correspondência entre mapeamento 3D e modelos CAD para operações de bin-picking

Mestrado em Engenharia MecânicaEste trabalho propõe uma solução para Bin-Picking, baseada em mapeamentos 3D e referências do modelo CAD 3D do objeto. Em primeiro lugar foi convertido o modelo STL, da peça, em nuvem de pontos por diferentes métodos. De seguida foi detetado o contentor onde os objetos se encontram no espaço de trabalho, isolando o espaço de busca e definindo as restrições á abordagem do manipulador. Deste modo, as duas nuvens foram conseguidas possibilitando a comparação do modelo sintetizado com a nuvem recolhida pelo sensor. A comparação foi executada a custa de descritores locais, calculados em ambas as nuvens, tais como o SHOT e FPFH. Algumas propriedades do modelo foram analisadas , como a parcialidade e inclusão de ruído.This thesis proposes a solution for bin-picking based on 3D mapping and 3D CAD model reference, of the object. First it was converted STL model of the part, in point cloud by different methods. Then the container was detected where objects are in the workspace, isolating the search space and defining the approach restrictions of the robot. Thus the two clouds were obtained allowing comparison between the synthesized model and the cloud gathered by the sensor. The comparison was performed at the expense of local descriptors calculated in both clouds, such as SHOT and FPFH. Some properties of the model were analyzed, such as partiality and inclusion of noise

Scene segmentation using similarity, motion and depth based cues

Segmentation of complex scenes to aid surveillance is still considered an open research problem. In this thesis a computational model (CM) has been developed to classify a scene into foreground, moving-shadow and background regions. It has been demonstrated how the CM, with the optional use of a channel ratio test, can be applied to demarcate foreground shadow regions in indoor scenes illuminated by a fixed incandescent source of light. A combined approach, involving the CM working in tandem with a traditional motion cue based segmentation method, has also been constructed. In the combined approach, the CM is applied to segregate the foreground shaded regions in a current frame based on a binary mask generated using a standard background subtraction process (BSP). Various popular outlier detection strategies have been investigated to assess their suitabilities in generating a threshold automatically, required to develop a binary mask from a difference frame, the outcome of the BSP. To evaluate the full scope of the pixel labeling capabilities of the CM and to estimate the associated time constraints, the model is deployed for foreground scene segmentation in recorded real-life video streams. The observations made validate the satisfactory performance of the model in most cases. In the second part of the thesis depth based cues have been exploited to perform the task of foreground scene segmentation. An active structured light based depthestimating arrangement has been modeled in the thesis; the choice of modeling an active system over a passive stereovision one has been made to alleviate some of the difficulties associated with the classical correspondence problem. The model developed not only facilitates use of the set-up but also makes possible a method to increase the working volume of the system without explicitly encoding the projected structured pattern. Finally, it is explained how scene segmentation can be accomplished based solely on the structured pattern disparity information, without generating explicit depthmaps. To de-noise the difference frames, generated using the developed method, two median filtering schemes have been implemented. The working of one of the schemes is advocated for practical use and is described in terms of discrete morphological operators, thus facilitating hardware realisation of the method to speed-up the de-noising process

Traitement et analyse d'images stéréoscopiques avec les approches du calcul générique sur un processeur graphique

RÉSUMÉ Étant donné une paire d'images stéréoscopiques, il est possible de calculer une carte de disparité dense qui encode les correspondances par pixel entre deux vues d'une même scène. Étant donné les paramètres de calibration d'une paire d'appareils photo, il est possible de transformer une carte de disparités en une carte de profondeur. Il existe de nombreuses applications allant de la robotique et de l'interaction humain-machine à la photographie 3D qui peuvent bénéficier de l'utilisation de cartes de disparité précises. Nous nous intéressons à la production de cartes de disparité de haute qualité à partir d'images stéréo pour des applications à temps réel ou dans un ordre de grandeur de temps réel par rapport au nombre d'images par seconde pour des vidéos typiques si un traitement hors ligne est acceptable. Nous avons donc étudié de possibilités d'accélérer de divers calculs clés nécessaires pour produire des cartes de disparité à partir d'images stéréoscopiques. Tout d'abord, nous explorons le potentiel de détecter les disparités incompatibles avec un calcul rapide sur les images en basse définition et d'une vérification de consistance basée sur la comparaison entre une paire de cartes de disparité de gauche à droite et de droite à gauche. L'idée est que les disparités incompatibles sont susceptibles de contenir des erreurs. Puis nous évaluons le potentiel d'appliquer de calculs sélectifs en employant les images stéréoscopiques de plus haute définition afin de réduire les erreurs tout en évitant de calculs coûteux sur les images stéréoscopiques en plus haute définition tout entières. Nous avons aussi introduit une méthode d'interpolation simple et rapide qui est capable d'améliorer la qualité d'une carte de disparité si la densité de pixels consistants est élevée. Des travaux récents ont montré que la qualité d'une carte de disparité peut être améliorée en combinant différentes mesures de distance. Nous explorons une fonction de combinaison simple pour la somme des différences au carré et les distances de Hamming entre les blocs d'image représentés par la transformation Census. Nous montrons que cette technique de combinaison peut produire d'améliorations significatives quant à la qualité de carte de disparité. Nous explorons aussi des approches fondées sur la combinaison des deux mesures et la combinaison d'utilisation d'imageries en haute et basse résolutions de manière sélective. Nous montrons aussi comment l'essence de méthodes populaires et d'état de l'art d'inférence semi-globale peut être formulée en utilisant des modèles de Markov cachés. Cela nous permet de généraliser les approches semi-globales à des modèles plus sophistiqués tout en ouvrant la porte aux paramètres des modèles d'apprentissage en utilisant des techniques du maximum de vraisemblance. Pour accélérer les calculs, normalement nous avons employé le calcul générique sur un processeur graphique (GPGPU). En particulier, nous avons implémenté en OpenCL une variation de la mise en correspondance par bloc basée sur la somme des différences au carré et présenté une version corrigée de l'implémentation de l'algorithme Viterbi en OpenCL qui était fournie dans un kit de développement logiciel de GPU. ----------ABSTRACT Given a pair of stereo images it is possible to compute a dense disparity map which encodes the per pixel correspondences between views. Given calibrated cameras it is possible to transform a disparity map into a depth map. There are many applications ranging from robotics and human computer interaction to 3D photography that benefit from the use of precise disparity maps. We are interested in producing high quality disparity maps from stereo imagery as quickly as possible for real-time applications or within an order of magnitude of real-time for typical video rates for applications where off-line processing is acceptable. We therefore explore the problem of accelerating various key computations needed to produce disparity maps from stereo imagery. First, we explore the potential of detecting inconsistent disparities with fast but low resolution comparisons and a consistency check based on comparing left to right and right to left disparity maps. The idea is that inconsistent disparities are likely to contain errors. We then evaluate the potential of selectively applying computation using higher resolution imagery in order to reduce errors while avoiding expensive computations over the entire high resolution image. We also introduce a simple and fast interpolation method that is capable of improving the quality of a disparity map if the density of consistent pixels is high. Recent work has shown that disparity map quality can be also be improved by combining different distance metrics. We explore a simple combination function for sum of squared difference and Hamming distances between image blocks represented using the Census transform. We show that this combination technique can produce significant improvements in disparity map quality. We also explore approaches based on both combining metrics and selectively combining high and low resolution imagery. We also show how the essence of popular, state of the art semi-global inference methods can be formulated using hidden Markov models. This allows us to generalize semi-global approaches to more sophisticated models while also opening the door to learning model parameters using maximum likelihood techniques. To accelerate computations generally we use general purpose graphical processing unit (GPGPU) computing. In particular, we have implemented a variation of sum of squared difference block matching in OpenCL and present a corrected version of an OpenCL Viterbi algorithm implementation that was provided in a GPU software development kit