Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

A Non-Rigid Map Fusion-Based RGB-Depth SLAM Method for Endoscopic Capsule Robots

In the gastrointestinal (GI) tract endoscopy field, ingestible wireless capsule endoscopy is considered as a minimally invasive novel diagnostic technology to inspect the entire GI tract and to diagnose various diseases and pathologies. Since the development of this technology, medical device companies and many groups have made significant progress to turn such passive capsule endoscopes into robotic active capsule endoscopes to achieve almost all functions of current active flexible endoscopes. However, the use of robotic capsule endoscopy still has some challenges. One such challenge is the precise localization of such active devices in 3D world, which is essential for a precise three-dimensional (3D) mapping of the inner organ. A reliable 3D map of the explored inner organ could assist the doctors to make more intuitive and correct diagnosis. In this paper, we propose to our knowledge for the first time in literature a visual simultaneous localization and mapping (SLAM) method specifically developed for endoscopic capsule robots. The proposed RGB-Depth SLAM method is capable of capturing comprehensive dense globally consistent surfel-based maps of the inner organs explored by an endoscopic capsule robot in real time. This is achieved by using dense frame-to-model camera tracking and windowed surfelbased fusion coupled with frequent model refinement through non-rigid surface deformations

Shape from shading with non-parallel light source.

by Siu-Yuk Yeung.Thesis (M.Phil.)--Chinese University of Hong Kong, 1999.Includes bibliographical references (leaves 96-102).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.5Chapter 1.1 --- Shape recovery techniques --- p.5Chapter 1.2 --- Shape from Shading algorithms --- p.8Chapter 1.2.1 --- Some developments on surface reflection --- p.9Chapter 1.2.2 --- Some developments on computing methods --- p.11Chapter 1.2.3 --- Some developments on light source model --- p.12Chapter 1.3 --- Proposed algorithms in this thesis --- p.13Chapter 1.4 --- Thesis outline --- p.14Chapter 2 --- Camera and surface reflectance models for SFS --- p.15Chapter 2.1 --- Camera models for SFS --- p.16Chapter 2.1.1 --- Pinhole camera model and perspective projection --- p.17Chapter 2.1.2 --- Approximations of perspective projection --- p.20Chapter 2.2 --- Surface reflectance models for SFS --- p.22Chapter 2.2.1 --- Lambertian surface model --- p.23Chapter 2.2.2 --- Bidirectional Reflectance Distribuction Function --- p.23Chapter 2.3 --- Summary --- p.25Chapter 3 --- Review of some related SFS algorithms --- p.26Chapter 3.1 --- The SFS algorithm proposed by Bichsel and Pentland --- p.27Chapter 3.1.1 --- Determine surface height with a minimum downhill principle --- p.28Chapter 3.1.2 --- Implementation on a discrete grid --- p.30Chapter 3.2 --- The SFS algorithm proposed by Kimmel and Bruckstein --- p.31Chapter 3.2.1 --- Level set propagation --- p.32Chapter 3.2.2 --- Problem formulation --- p.33Chapter 3.2.3 --- Equal height contour propagation using level set method --- p.35Chapter 3.3 --- Summary --- p.36Chapter 4 --- Multiple extended light source models for SFS --- p.38Chapter 4.1 --- Three extended light source models for SFS --- p.40Chapter 4.1.1 --- Rectangular light source model --- p.40Chapter 4.1.2 --- Spherical light source model --- p.43Chapter 4.1.3 --- Cylindrical light source model --- p.48Chapter 4.2 --- SFS for an extended light source --- p.53Chapter 4.3 --- Multiple extended light source model --- p.53Chapter 4.4 --- Simulation and experiment result --- p.54Chapter 4.5 --- Error Analysis --- p.55Chapter 4.5.1 --- Descriptions of the error --- p.55Chapter 4.5.2 --- Errors for different light models --- p.55Chapter 4.6 --- Summary --- p.57Chapter 5 --- Global SFS for an endoscope image --- p.70Chapter 5.1 --- Introduction --- p.71Chapter 5.2 --- Local SFS algorithm for endoscope image --- p.73Chapter 5.2.1 --- Imaging system and brightness formulation --- p.74Chapter 5.2.2 --- Equal distance contour propagation and shape reconstruc- tion --- p.75Chapter 5.3 --- Global SFS algorithm for endoscope image --- p.76Chapter 5.3.1 --- A global shape from shading algorithm for a parallel light --- p.77Chapter 5.3.2 --- The relationship between depth map and distance map --- p.78Chapter 5.3.3 --- A global shape from shading algorithm for endoscope image --- p.78Chapter 5.4 --- Simulations and experiments results --- p.83Chapter 5.5 --- Summary --- p.86Chapter 6 --- Summary and conclusion --- p.87Chapter 6.1 --- Problems tackled in this thesis --- p.87Chapter 6.2 --- Discussion on future developments --- p.8

Patient-specific bronchoscope simulation with pq-space-based 2D/3D registration

Objective: The use of patient-specific models for surgical simulation requires photorealistic rendering of 3D structure and surface properties. For bronchoscope simulation, this requires augmenting virtual bronchoscope views generated from 3D tomographic data with patient-specific bronchoscope videos. To facilitate matching of video images to the geometry extracted from 3D tomographic data, this paper presents a new pq-space-based 2D/3D registration method for camera pose estimation in bronchoscope tracking. Methods: The proposed technique involves the extraction of surface normals for each pixel of the video images by using a linear local shape-from-shading algorithm derived from the unique camera/lighting constraints of the endoscopes. The resultant pq-vectors are then matched to those of the 3D model by differentiation of the z-buffer. A similarity measure based on angular deviations of the pq-vectors is used to provide a robust 2D/3D registration framework. Localization of tissue deformation is considered by assessing the temporal variation of the pq-vectors between subsequent frames. Results: The accuracy of the proposed method was assessed by using an electromagnetic tracker and a specially constructed airway phantom. Preliminary in vivo validation of the proposed method was performed on a matched patient bronchoscope video sequence and 3D CT data. Comparison to existing intensity-based techniques was also made. Conclusion: The proposed method does not involve explicit feature extraction and is relatively immune to illumination changes. The temporal variation of the pq distribution also permits the identification of localized deformation, which offers an effective way of excluding such areas from the registration process