Structured light techniques for 3D surface reconstruction in robotic tasks

Robotic tasks such as navigation and path planning can be greatly enhanced by a vision system capable of providing depth perception from fast and accurate 3D surface reconstruction. Focused on robotic welding tasks we present a comparative analysis of a novel mathematical formulation for 3D surface reconstruction and discuss image processing requirements for reliable detection of patterns in the image. Models are presented for a parallel and angled configurations of light source and image sensor. It is shown that the parallel arrangement requires 35\% fewer arithmetic operations to compute a point cloud in 3D being thus more appropriate for real-time applications. Experiments show that the technique is appropriate to scan a variety of surfaces and, in particular, the intended metallic parts for robotic welding tasks

A structured light solution for detecting scapular dyskinesis

Scapular dyskinesis is a common occurrence in overhead athletes, i.e. athletes who participate in any sport where the upper arm and shoulder is used above the athlete’s head. However, no consensus has been reached on how to evaluate scapular dyskinesis quantitatively. In this thesis, we developed a measuring tool that can be used to evaluate certain key clinical parameters specific to scapular dyskinesis. The tool employs a 3D structured light computer vision approach to create a surface map of the soft-tissue across the scapula. This surface map is then analysed using surface curvature analysis techniques to identify the key clinical parameters associated with scapular dyskinesis. The main advantage of this method is that it provides a measurement tool that may facilitate future quantitative analysis of these key parameters. This may aid with diagnosis and monitoring of the condition by allowing measurement data to be collected both before and after treatment and rehabilitation. We expect that this tool will make the monitoring of treatment effectiveness easier while contributing to diagnostic computer vision

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Amorphous silicon e 3D sensors applied to object detection

Nowadays, existing 3D scanning cameras and microscopes in the market use digital or discrete sensors, such as CCDs or CMOS for object detection applications. However, these combined systems are not fast enough for some application scenarios since they require large data processing resources and can be cumbersome. Thereby, there is a clear interest in exploring the possibilities and performances of analogue sensors such as arrays of position sensitive detectors with the final goal of integrating them in 3D scanning cameras or microscopes for object detection purposes. The work performed in this thesis deals with the implementation of prototype systems in order to explore the application of object detection using amorphous silicon position sensors of 32 and 128 lines which were produced in the clean room at CENIMAT-CEMOP. During the first phase of this work, the fabrication and the study of the static and dynamic specifications of the sensors as well as their conditioning in relation to the existing scientific and technological knowledge became a starting point. Subsequently, relevant data acquisition and suitable signal processing electronics were assembled. Various prototypes were developed for the 32 and 128 array PSD sensors. Appropriate optical solutions were integrated to work together with the constructed prototypes, allowing the required experiments to be carried out and allowing the achievement of the results presented in this thesis. All control, data acquisition and 3D rendering platform software was implemented for the existing systems. All these components were combined together to form several integrated systems for the 32 and 128 line PSD 3D sensors. The performance of the 32 PSD array sensor and system was evaluated for machine vision applications such as for example 3D object rendering as well as for microscopy applications such as for example micro object movement detection. Trials were also performed involving the 128 array PSD sensor systems. Sensor channel non-linearities of approximately 4 to 7% were obtained. Overall results obtained show the possibility of using a linear array of 32/128 1D line sensors based on the amorphous silicon technology to render 3D profiles of objects. The system and setup presented allows 3D rendering at high speeds and at high frame rates. The minimum detail or gap that can be detected by the sensor system is approximately 350 μm when using this current setup. It is also possible to render an object in 3D within a scanning angle range of 15º to 85º and identify its real height as a function of the scanning angle and the image displacement distance on the sensor. Simple and not so simple objects, such as a rubber and a plastic fork, can be rendered in 3D properly and accurately also at high resolution, using this sensor and system platform. The nip structure sensor system can detect primary and even derived colors of objects by a proper adjustment of the integration time of the system and by combining white, red, green and blue (RGB) light sources. A mean colorimetric error of 25.7 was obtained. It is also possible to detect the movement of micrometer objects using the 32 PSD sensor system. This kind of setup offers the possibility to detect if a micro object is moving, what are its dimensions and what is its position in two dimensions, even at high speeds. Results show a non-linearity of about 3% and a spatial resolution of < 2µm

Valukappaleiden parametrinen mallinnus ja laadunvarmistus

Even though 3D modelling and 3D scanning have been industry standards in casting design and measurement for a long time, casting tolerance standards still usually rely on complete definition of castings in 2D drawings and linear dimensional tolerances. ISO 8062-4, published in 2017, is based on a general surface profile tolerance instead of separate dimensional tolerances, thus forming a logical connection between 3D modelling, tolerancing and 3D scanning. As the contemporary 3D methods are prominently used, the outdated ISO 8062-3 should be replaced with the new standard in casting tolerance definition. In this thesis, possibilities to include casting tolerances in casting CAD models are examined. The aim is to create dependent but separate 3D models that depict the allowed minimum and maximum state of the casting as allowed by the casting tolerance. The benefits of these tolerance models are evaluated considering quality verification, strength calculations and collision detection. ISO 8062-4 casting tolerance and a traction sheave from an NMX hoisting machine are used for modelling case study. In addition, different measurement methods are reviewed to determine guidelines for casting measurement based on contemporary methods and standards. The case study revealed that tolerance model construction is difficult and time-consuming compared to the achieved benefits. Therefore, no implementation to every casting by default is recommended. Attention was also paid to modelling techniques and conventions that should be followed regardless of the tolerance models. 3D scanning should be utilised in casting quality verification because of its coverage, speed and ability to compare scanned 3D model directly to the nominal model. Measurement instructions were determined considering the requirements of ISO 8062-4 and the possibilities and restrictions of 3D scanning. An exemplary casting drawing of the traction sheave was created according to ISO 8062-4. Digital 3D product definition and 3D scanning were taken into account when determining the drawing indication.Vaikka 3D-mallinnus sekä 3D-skannaus ovat jo pitkään olleet vallitsevat suunnittelu- ja mittausmenetelmät valuteollisuudessa, valutoleranssistandardit nojaavat pitkälti vielä valujen täydelliseen esittämiseen 2D-kuvissa sekä lineaaristen mittojen tolerointiin. Vuonna 2017 julkaistu ISO 8062-4 perustuu yleiseen pinnanmuototoleranssiin yksittäisten mittojen toleroinnin sijaan, jolloin uusi standardi muodostaa ehyen ja loogisen jatkumon 3D-mallinnuksen ja 3D-skannauksen kanssa. Nykyaikaisia menetelmiä käytettäessä onkin syytä korvata vanhentuneisiin työkaluihin perustuva ISO 8062-3 uudella standardilla valutoleranssien määrityksessä. Tässä työssä selvitetään, miten valutoleranssin saa parametrisesti sisällytettyä valukappaleen CAD-malliin, jolloin erilliset mallit kuvaisivat toleranssin sallimia valun minimi- ja maksimitiloja. Minimi- ja maksimitilojen mallien hyötyjä arvioidaan laadunvarmistuksen, lujuuslaskennan ja törmäystarkasteluiden kannalta. Toleranssimallien luonnissa käytetään standardin ISO 8062-4 mukaisia toleransseja sekä malliesimerkkinä NMX-hissimoottorin vetopyörää. Lisäksi tarkastellaan eri mittausmenetelmiä, ja pyritään määrittämään suuntaviivat nykyaikaisiin menetelmiin ja standardeihin perustuvaa mittausohjetta varten. Tarkasteluissa havaittiin toleranssimallien mallinnuksen olevan työlästä saatavaan hyötyyn nähden, jolloin niiden sisällyttäminen oletuksena jokaiseen valuun ei ole järkevää. Tarkastelun yhteydessä kiinnitettiin huomiota myös mallinnustekniikkaan ja tapoihin, joita tulisi valusuunnittelussa noudattaa riippumatta toleranssimallien käytöstä. Valujen mittaamisessa tulisi ensisijaisesti hyödyntää 3D-skannausta, jossa nimellistä CAD-mallia verrataan skannattuun 3D-malliin, ja joka kattavuutensa ja nopeutensa ansiosta onkin yleisesti käytössä valuteollisuudessa. Mittausprosessin kulku määritettiin samalla huomioiden standardin ISO 8062-4 vaatimukset sekä 3D-skannauksen mahdollisuudet ja rajoitteet. Malliesimerkkinä käytetystä vetopyörästä laadittiin ISO 8062-4:n mukainen valupiirustus, jossa huomioidaan myös nimellisen muodon määritys CAD-mallilla sekä 3Dskannauksen käyttö laadunvarmistuksessa

State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation

3D imaging sensors for the acquisition of three dimensional (3D) shapes have created, in recent years, a considerable degree of interest for a number of applications. The miniaturization and integration of the optical and electronic components used to build them have played a crucial role in the achievement of compactness, robustness and flexibility of the sensors. Today, several 3D sensors are available on the market, even in combination with other sensors in a “sensor fusion” approach. An importance equal to that of physical miniaturization has the portability of the measurements, via suitable interfaces, into software environments designed for their elaboration, e.g., CAD-CAM systems, virtual renders, and rapid prototyping tools. In this paper, following an overview of the state-of-art of 3D imaging sensors, a number of significant examples of their use are presented, with particular reference to industry, heritage, medicine, and criminal investigation applications