Structured light techniques for 3D surface reconstruction in robotic tasks

Robotic tasks such as navigation and path planning can be greatly enhanced by a vision system capable of providing depth perception from fast and accurate 3D surface reconstruction. Focused on robotic welding tasks we present a comparative analysis of a novel mathematical formulation for 3D surface reconstruction and discuss image processing requirements for reliable detection of patterns in the image. Models are presented for a parallel and angled configurations of light source and image sensor. It is shown that the parallel arrangement requires 35\% fewer arithmetic operations to compute a point cloud in 3D being thus more appropriate for real-time applications. Experiments show that the technique is appropriate to scan a variety of surfaces and, in particular, the intended metallic parts for robotic welding tasks

Fast 3D reconstruction with single shot technology : engineering and computing challenges

Fast 3D reconstruction with single shot technology: the GMPR 3D scanning technologies provide fast wide area scanning from an instantaneous shot. A surface can be reconstructed in 40 milliseconds from a pattern of stripes projected on the target object. It operates on a single image or on a video sequence both in the near-infrared (NIR) and visible spectra. In this talk we are going to describe the engineering and computing principles behind the technologies, highlight the main achievements of our research to date and discuss a number of remaining challenge

Fast 3D Reconstruction using Structured Light Methods

In this presentation we discuss the use of structured light scanners for the general problem of 3D surface reconstruction. We show that projecting patterns of light provide an inexpensive means of consistent 3D scanning at high resolution, in real-time and from single images. The main problem of such techniques is pattern decoding or stripe indexing, which can be substantially non-trivial and difficult to overcome in a reliable way. We discuss existing techniques and show how a minimal light coding in the projected stripes can resolve inherent ambiguities found in stripe patterns across surface discontinuities. We also discuss how our real-time solution using structured near-infrared light can overcome ambient illumination and used in a variety of medical contexts

An intelligent real time 3D vision system for robotic welding tasks

MARWIN is a top-level robot control system that has been designed for automatic robot welding tasks. It extracts welding parameters and calculates robot trajectories directly from CAD models which are then verified by real-time 3D scanning and registration. MARWIN's 3D computer vision provides a user-centred robot environment in which a task is specified by the user by simply confirming and/or adjusting suggested parameters and welding sequences. The focus of this paper is on describing a mathematical formulation for fast 3D reconstruction using structured light together with the mechanical design and testing of the 3D vision system and show how such technologies can be exploited in robot welding tasks

Developing Interaction 3D Models for E-Learning Applications

Some issues concerning the development of interactive 3D models for e-learning applications are considered. Given that 3D data sets are normally large and interactive display demands high performance computation, a natural solution would be placing the computational burden on the client machine rather than on the server. Mozilla and Google opted for a combination of client-side languages, JavaScript and OpenGL, to handle 3D graphics in a web browser (Mozilla 3D and O3D respectively). Based on the O3D model, core web technologies are considered and an example of the full process involving the generation of a 3D model and their interactive visualization in a web browser is described. The challenging issue of creating realistic 3D models of objects in the real world is discussed and a method based on line projection for fast 3D reconstruction is presented. The generated model is then visualized in a web browser. The experiments demonstrate that visualization of 3D data in a web browser can provide quality user experience. Moreover, the development of web applications are facilitated by O3D JavaScript extension allowing web designers to focus on 3D contents generation

Real-time 3D Face Recognition using Line Projection and Mesh Sampling

The main contribution of this paper is to present a novel method for automatic 3D face recognition based on sampling a 3D mesh structure in the presence of noise. A structured light method using line projection is employed where a 3D face is reconstructed from a single 2D shot. The process from image acquisition to recognition is described with focus on its real-time operation. Recognition results are presented and it is demonstrated that it can perform recognition in just over one second per subject in continuous operation mode and thus, suitable for real time operation

Novel methods for real-time 3D facial recognition

In this paper we discuss our approach to real-time 3D face recognition. We argue the need for real time operation in a realistic scenario and highlight the required pre- and post-processing operations for effective 3D facial recognition. We focus attention to some operations including face and eye detection, and fast post-processing operations such as hole filling, mesh smoothing and noise removal. We consider strategies for hole filling such as bilinear and polynomial interpolation and Laplace and conclude that bilinear interpolation is preferred. Gaussian and moving average smoothing strategies are compared and it is shown that moving average can have the edge over Gaussian smoothing. The regions around the eyes normally carry a considerable amount of noise and strategies for replacing the eyeball with a spherical surface and the use of an elliptical mask in conjunction with hole filling are compared. Results show that the elliptical mask with hole filling works well on face models and it is simpler to implement. Finally performance issues are considered and the system has demonstrated to be able to perform real-time 3D face recognition in just over 1s 200ms per face model for a small database

Efficient 3D data compression through parameterization of free-form surface patches

This paper presents a new method for 3D data compression based on parameterization of surface patches. The technique is applied to data that can be defined as single valued functions; this is the case for 3D patches obtained using standard 3D scanners. The method defines a number of mesh cutting planes and the intersection of planes on the mesh defines a set of sampling points. These points contain an explicit structure that allows us to define parametrically both x and y coordinates. The z values are interpolated using high degree polynomials and results show that compressions over 99% are achieved while preserving the quality of the mesh

Surface scanning with uncoded structured light sources.

Structured Light Scanners measure the surface of a target object, producing a set of vertices which can be used to construct a three-dimensional model of the surface. The techniques are particularly appropriate for measuring the smoothly undulating, featureless forms which Stereo Vision methods find difficult, and the structured light pattern explicitly gives a dense graph of connected vertices, thus obviating the need for vertex-triangulation prior to surface reconstruction. In addition, the technique provides the measurements almost instantaneously, and so is suitable for scanning moving and non-rigid objects. Because of these advantages there is an imperative to extend the range of scannable surfaces to those including occlusions, which often reduce or prevent successful measurement.This thesis investigates ways of improving both the accuracy and the range of surface types which can be scanned using structured light techniques, extending current research by examining the role of occlusions and geometric constraints, and introducing novel algorithms to solve the Indexing Problem. The Indexing Problem demands that for every pattern element in the projected image, its counterpart, reflected from the surface of the target object, must be found in the recorded image, and most researchers have declared this problem to be intractable without resorting to coding schemes which uniquely identify each pattern element. The use of uncoded projection patterns, where the pattern elements are projected without any unique identification, has two advantages: firstly it provides the densest possible set of measured vertices within a single video timeframe, and secondly it allows the investigation of the fundamental problems without the distraction of dealing with coding schemes. These advantages educe the general strategy adopted in this thesis, of attempting to solve the Indexing Problem using uncoded patterns, and then adding some coding where difficulties still remain.In order to carry out these investigations it is necessary to precisely measure the system and its outputs, and to achieve this requirement two scanners have been built, a Single Stripe Scanner and a Multiple Stripe Scanner. The Single Stripe Scanner introduces the geometric measurement methods and provides a reference output which matches the industry standard; the Multiple Stripe Scanner then tests the results of the investigations and evaluates the success of the new algorithms and constraints. In addition, some of the investigations are tested theoretically, by using synthetic data and by the solution of geometric diagrams.These evaluations of success show that, if occlusions are not present in the recorded data, the Indexing Problem can often be completely solved if the new indexing algorithms and geometric constraints are included. Furthermore, while there are some cases where the Indexing Problem cannot be solved without recourse to a coding scheme, the addition of occlusion detection in the algorithms greatly improves the indexing accuracy and therefore the successful measurement of the target surface

3D modelling and recognition

3D face recognition is an open field. In this paper we present a method for 3D facial recognition based on Principal Components Analysis. The method uses a relatively large number of facial measurements and ratios and yields reliable recognition. We also highlight our approach to sensor development for fast 3D model acquisition and automatic facial feature extraction