Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Material Recognition Meets 3D Reconstruction : Novel Tools for Efficient, Automatic Acquisition Systems

For decades, the accurate acquisition of geometry and reflectance properties has represented one of the major objectives in computer vision and computer graphics with many applications in industry, entertainment and cultural heritage. Reproducing even the finest details of surface geometry and surface reflectance has become a ubiquitous prerequisite in visual prototyping, advertisement or digital preservation of objects. However, today's acquisition methods are typically designed for only a rather small range of material types. Furthermore, there is still a lack of accurate reconstruction methods for objects with a more complex surface reflectance behavior beyond diffuse reflectance. In addition to accurate acquisition techniques, the demand for creating large quantities of digital contents also pushes the focus towards fully automatic and highly efficient solutions that allow for masses of objects to be acquired as fast as possible. This thesis is dedicated to the investigation of basic components that allow an efficient, automatic acquisition process. We argue that such an efficient, automatic acquisition can be realized when material recognition "meets" 3D reconstruction and we will demonstrate that reliably recognizing the materials of the considered object allows a more efficient geometry acquisition. Therefore, the main objectives of this thesis are given by the development of novel, robust geometry acquisition techniques for surface materials beyond diffuse surface reflectance, and the development of novel, robust techniques for material recognition. In the context of 3D geometry acquisition, we introduce an improvement of structured light systems, which are capable of robustly acquiring objects ranging from diffuse surface reflectance to even specular surface reflectance with a sufficient diffuse component. We demonstrate that the resolution of the reconstruction can be increased significantly for multi-camera, multi-projector structured light systems by using overlappings of patterns that have been projected under different projector poses. As the reconstructions obtained by applying such triangulation-based techniques still contain high-frequency noise due to inaccurately localized correspondences established for images acquired under different viewpoints, we furthermore introduce a novel geometry acquisition technique that complements the structured light system with additional photometric normals and results in significantly more accurate reconstructions. In addition, we also present a novel method to acquire the 3D shape of mirroring objects with complex surface geometry. The aforementioned investigations on 3D reconstruction are accompanied by the development of novel tools for reliable material recognition which can be used in an initial step to recognize the present surface materials and, hence, to efficiently select the subsequently applied appropriate acquisition techniques based on these classified materials. In the scope of this thesis, we therefore focus on material recognition for scenarios with controlled illumination as given in lab environments as well as scenarios with natural illumination that are given in photographs of typical daily life scenes. Finally, based on the techniques developed in this thesis, we provide novel concepts towards efficient, automatic acquisition systems

Innovative Techniques for Digitizing and Restoring Deteriorated Historical Documents

Recent large-scale document digitization initiatives have created new modes of access to modern library collections with the development of new hardware and software technologies. Most commonly, these digitization projects focus on accurately scanning bound texts, some reaching an efficiency of more than one million volumes per year. While vast digital collections are changing the way users access texts, current scanning paradigms can not handle many non-standard materials. Documentation forms such as manuscripts, scrolls, codices, deteriorated film, epigraphy, and rock art all hold a wealth of human knowledge in physical forms not accessible by standard book scanning technologies. This great omission motivates the development of new technology, presented by this thesis, that is not-only effective with deteriorated bound works, damaged manuscripts, and disintegrating photonegatives but also easily utilized by non-technical staff. First, a novel point light source calibration technique is presented that can be performed by library staff. Then, a photometric correction technique which uses known illumination and surface properties to remove shading distortions in deteriorated document images can be automatically applied. To complete the restoration process, a geometric correction is applied. Also unique to this work is the development of an image-based uncalibrated document scanner that utilizes the transmissivity of document substrates. This scanner extracts intrinsic document color information from one or both sides of a document. Simultaneously, the document shape is estimated to obtain distortion information. Lastly, this thesis provides a restoration framework for damaged photographic negatives that corrects photometric and geometric distortions. Current restoration techniques for the discussed form of negatives require physical manipulation to the photograph. The novel acquisition and restoration system presented here provides the first known solution to digitize and restore deteriorated photographic negatives without damaging the original negative in any way. This thesis work develops new methods of document scanning and restoration suitable for wide-scale deployment. By creating easy to access technologies, library staff can implement their own scanning initiatives and large-scale scanning projects can expand their current document-sets

3D Acquisition of Mirroring Objects using Striped Patterns

Objects with mirroring optical characteristics are left out of the scope of most 3D scanning methods. We present here a new automatic acquisition approach, shape-from-distortion, that focuses on that category of objects, requires only a still camera and a color monitor, and produces range scans (plus a normal and a reflectance map) of the target. Our technique consists of two steps: first, an improved environment matte is captured for the mirroring object, using the interference of patterns with different frequencies to obtain sub-pixel accuracy. Then, the matte is converted into a normal and a depth map by exploiting the self-coherence of a surface when integrating the normal map along different paths. The results show very high accuracy, capturing even smallest surface details. The acquired depth maps can be further processed using standard techniques to produce a complete 3D mesh of the object

On the detection of defects on specular car body surfaces

[EN] The automatic detection of small defects (of up to 0.2 mm in diameter) on car body surfaces following the painting process is currently one of the greatest issues facing quality control in the automotive industry. Although several systems have been developed during the last decade to provide a solution to this problem, these, to the best of our knowledge, have been focused solely on flat surfaces and have been unable to inspect other parts of the surfaces, namely style lines, edges and corners as well as deep concavities. This paper introduces a novel approach using deflectometry- and vision-based technologies in order to overcome this problem and ensure that the whole area is inspected. Moreover, since our approach, together with the system used, computes defects in less than 15 s, it satisfies cycle time production requirements (usually of around 30 s per car). Hence, a two-step algorithm is presented here: in the first step, a new pre-processing step (image fusion algorithm) is introduced to enhance the contrast between pixels with a low level of intensity (indicating the presence of defects) and those with a high level of intensity (indicating the absence of defects); for the second step, we present a novel post-processing step with an image background extraction approach based on a local directional blurring method and a modified image contrast enhancement, which enables detection of defects in the entire illuminated area. In addition, the post-processing step is processed several times using a multi-level structure, with computed image backgrounds of different resolution. In doing so, it is possible to detect larger defects, given that each level identifies defects of different sizes. Experimental results presented in this paper are obtained from the industrial automatic quality control system QEyeTunnel employed in the production line at the Mercedes-Benz factory in Vitoria, Spain. A complete analysis of the algorithm performance will be shown here, together with several tests proving the robustness and reliability of our proposal.This work is supported by VALi+d (APOSTD/2016/044) and PROMETEO (PROMETEOII/2014/044) Programs, both from Conselleria d'Educacio, Generalitat Valenciana.Molina, J.; Solanes Galbis, JE.; Arnal-Benedicto, L.; Tornero Montserrat, J. (2017). On the detection of defects on specular car body surfaces. Robotics and Computer-Integrated Manufacturing. 48:263-278. https://doi.org/10.1016/j.rcim.2017.04.009S2632784

Sifat fizikal dan keupayaan mekanikal konkrit campuran abu kulit kupang (perna viridis) sebagai bahan tambah

Matlamat SDGs selari dengan inisiatif RMK-11 iaitu pengaplikasian teknologi hijau di dalam pembinaan infrastruktur yang mampan, berkonsepkan lestari dan pemuliharaan alam sekitar. Kajian ini memfokuskan kulit kupang yakni limpahan sisa akuakultur yang tidak diurus dengan tepat untuk dijadikan bahan tambah di dalam campuran konkrit bagi meningkatkan nilai kulit kupang dan keupayaan konkrit campuran bagi memacu infrastruktur yang lestari. Kupang yang telah di proses menjadi abu di tambah di dalam konkrit sebanyak 1%, 2%, 3% dan 4% dan melalui proses pengawetan air biasa selama 7, 28 dan 60 hari dengan sasaran gred 30MPa. Penyelidikan ini memfokuskan beberapa ujian pencirian bahan iaitu sifat-sifat fizikal (graviti tentu, masa pengerasan dan serakan saiz partikel), dan kimia (X-Ray Powder Diffraction ), morfologi partikel (Scanning Electron Microscope), keupayaan konkrit (kebolehkerjaan, kekuatan mampatan, kekuatan tegangan dan serapan kapilari air dan analisis statistik (korelasi dan signifikan). Dapatan sifat-sifat fizikal abu kulit kupang mendapati kulit kupang lebih halus (saiz diameter purata 8.284μm) dan ringan (nilai graviti tentu 2.52) berbanding simen Portland biasa memberi kelebihan di dalam masa pengerasan, kebolehkerjaan dan serapan kapilari. Bagi nilai keupayaan konkrit mampatan , tegangan dan serapan kapilari air konkrit campuran 2% abu kulit kupang adalah lebih baik berbanding konkrit campuran yang lain. Selain itu, tempoh pengawetan 7, 28 dan didapati tiada perbezaan yang signifikan (p > 0.025) bagi konkrit campuran 2%. Berdasarkan dapatan kajian abu kulit kupang mempunyai potensi sebagai bahan tambah yang menghasilkan konkrit berkekuatan awal dan berkeupayaan tinggi. Penyelidikan ini menambah pengetahuan bahan pembinaan , meningkatkan nilai kulit kupang serta memastikan infrastruktur berkonsepkan kehidupan lestari dan pemeliharaan alam sekitar di dalam matlamat SDGs dan dasar RMK-11 dicapai