Towards a clinical stereoscopic augmented reality system for laparoscopic surgery

We describe our development of a complete real-time stereoscopic augmented reality system that overlays laparoscopic ultrasound (LUS) images on stereoscopic laparoscopic video for conventional laparoscopic surgery. The system was designed and developed to achieve near-term clinical evaluation as a primary goal. Special consideration was paid to system interactivity, accuracy and easy integration within the existing clinical workflow. Custom-designed fixtures for the two imaging devices were created to avoid their recalibration in the operating room and thus to minimize setup time. The system was assembled on a rolling cart to make it portable for the use in the operating room. Utilizing our optimized design and hardware-accelerated implementation, the system achieved a low system latency of approximately 150 ms. The LUS image-to-video registration accuracy, measured in terms of target registration accuracy and recorded separately for the left and right eye channels of the stereoscopic camera, was 3.34±0.59 mm and 2.76±0.68 mm. © 2014 Springer International Publishing

A mixed reality guidance system for robot assisted laparoscopic radical prostatectomy

© Springer Nature Switzerland AG 2018. Robotic surgery with preoperative imaging data for planning have become increasingly common for surgical treatment of patients. For surgeons using robotic surgical platforms, maintaining spatial awareness of the anatomical structures in the surgical area is key for good outcomes. We propose a Mixed Reality system which allows surgeons to visualize and interact with aligned anatomical models extracted from preoperative imagery as well as the in vivo imagery from the stereo laparoscope. To develop this system, we have employed techniques to 3D reconstruct stereo laparoscope images, model 3D shape of the anatomical structures from preoperative MRI stack and align the two 3D surfaces. The application we have developed allows surgeons to visualize occluded and obscured organ boundaries as well as other important anatomy that is not visible through the laparoscope alone, facilitating better spatial awareness during surgery. The system was deployed in 9 robot assisted laparoscopic prostatectomy procedures as part of a feasibility study