Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Dense Vision in Image-guided Surgery

Image-guided surgery needs an efficient and effective camera tracking system in order to perform augmented reality for overlaying preoperative models or label cancerous tissues on the 2D video images of the surgical scene. Tracking in endoscopic/laparoscopic scenes however is an extremely difficult task primarily due to tissue deformation, instrument invasion into the surgical scene and the presence of specular highlights. State of the art feature-based SLAM systems such as PTAM fail in tracking such scenes since the number of good features to track is very limited. When the scene is smoky and when there are instrument motions, it will cause feature-based tracking to fail immediately. The work of this thesis provides a systematic approach to this problem using dense vision. We initially attempted to register a 3D preoperative model with multiple 2D endoscopic/laparoscopic images using a dense method but this approach did not perform well. We subsequently proposed stereo reconstruction to directly obtain the 3D structure of the scene. By using the dense reconstructed model together with robust estimation, we demonstrate that dense stereo tracking can be incredibly robust even within extremely challenging endoscopic/laparoscopic scenes. Several validation experiments have been conducted in this thesis. The proposed stereo reconstruction algorithm has turned out to be the state of the art method for several publicly available ground truth datasets. Furthermore, the proposed robust dense stereo tracking algorithm has been proved highly accurate in synthetic environment (< 0.1 mm RMSE) and qualitatively extremely robust when being applied to real scenes in RALP prostatectomy surgery. This is an important step toward achieving accurate image-guided laparoscopic surgery.Open Acces

Enhanced Image View Synthesis Using Multistage Hybrid Median Filter For Stereo Images

Disparity depth map estimation of stereo matching algorithm is one of the most active research topics in computer vision.In the field of image processing,many existing stereo matching algorithms to obtain disparity depth map are developed and designed with low accuracy.To improve the accuracy of disparity depth map is quite challenging and difficult especially with uncontrolled dynamic environment.The accuracy is affected by many unwanted aspects including random noises,horizontal streaks,low texture,depth map non-edge preserving, occlusion,and depth discontinuities.Thus,this research proposed a new robust method of hybrid stereo matching algorithm with significant accuracy of computation.The thesis will present in detail the development,design, and analysis of performance on Multistage Hybrid Median Filter (MHMF).There are two main parts involved in our developed method which combined in two main stages.Stage 1 consists of the Sum of Absolute Differences (SAD) from Basic Block Matching (BBM) algorithm and the part of Scanline Optimization (SO) from Dynamic Programming (DP) algorithm.While,Stage 2 is the main core of our MHMF as a post-processing step which included segmentation,merging, and hybrid median filtering.The significant feature of the post-processing step is on its ability to handle efficiently the unwanted aspects obtained from the raw disparity depth map on the step of optimization.In order to remove and overcome the challenges unwanted aspects, the proposed MHMF has three stages of filtering process along with the developed approaches in Stage 2 of MHMF algorithm.There are two categories of evaluation performed on the obtained disparity depth map: subjective evaluation and objective evaluation.The objective evaluation involves the evaluation on Middlebury Stereo Vision system and evaluation using traditional methods such as Mean Square Errors (MSE),Peak to Signal Noise Ratio (PSNR) and Structural Similarity Index Metric (SSIM).Based on the results of the standard benchmarking datasets from Middlebury,the proposed algorithm is able to reduce errors of non-occluded and all errors respectively.While,the subjective evaluation is done for datasets captured from MV BLUE FOX camera using human's eyes perception.Based on the results,the proposed MHMF is able to obtain accurate results, specifically 69% and 71% of non-occluded and all errors for disparity depth map, and it outperformed some of the existing methods in the literature such as BBM and DP algorithms