Improving Surgical Training Phantoms by Hyperrealism: Deep Unpaired Image-to-Image Translation from Real Surgeries

Current `dry lab' surgical phantom simulators are a valuable tool for surgeons which allows them to improve their dexterity and skill with surgical instruments. These phantoms mimic the haptic and shape of organs of interest, but lack a realistic visual appearance. In this work, we present an innovative application in which representations learned from real intraoperative endoscopic sequences are transferred to a surgical phantom scenario. The term hyperrealism is introduced in this field, which we regard as a novel subform of surgical augmented reality for approaches that involve real-time object transfigurations. For related tasks in the computer vision community, unpaired cycle-consistent Generative Adversarial Networks (GANs) have shown excellent results on still RGB images. Though, application of this approach to continuous video frames can result in flickering, which turned out to be especially prominent for this application. Therefore, we propose an extension of cycle-consistent GANs, named tempCycleGAN, to improve temporal consistency.The novel method is evaluated on captures of a silicone phantom for training endoscopic reconstructive mitral valve procedures. Synthesized videos show highly realistic results with regard to 1) replacement of the silicone appearance of the phantom valve by intraoperative tissue texture, while 2) explicitly keeping crucial features in the scene, such as instruments, sutures and prostheses. Compared to the original CycleGAN approach, tempCycleGAN efficiently removes flickering between frames. The overall approach is expected to change the future design of surgical training simulators since the generated sequences clearly demonstrate the feasibility to enable a considerably more realistic training experience for minimally-invasive procedures.Comment: 8 pages, accepted at MICCAI 2018, supplemental material at https://youtu.be/qugAYpK-Z4

Emerging Optical Methods for Endoscopic Barrett’s Surveillance

Barrett’s oesophagus is an acquired metaplastic condition that predisposes patients to the development of oesophageal adenocarcinoma, prompting the use of surveillance regimes to detect early malignancy for endoscopic therapy with curative intent. The currently accepted surveillance regime uses white light endoscopy together with random biopsies, but suffers poor sensitivity and discards information from numerous light-tissue interactions that could be exploited to probe structural, functional and molecular changes in the tissue. Advanced optical methods are now emerging that are exquisitely sensitive to these changes and hold significant potential to improve surveillance of Barrett’s oesophagus if they can be applied endoscopically. The next decade will see some of these exciting new methods applied to Barrett’s surveillance in new device architectures for the first time, potentially leading to a longawaited improvement of the standard of care

Online Super-Resolution For Fibre-Bundle-Based Confocal Laser Endomicroscopy

Probe-based Confocal Laser Endomicroscopy (pCLE) produces microscopic images enabling real-time in vivo optical biopsy. However, the miniaturisation of the optical hardware, specifically the reliance on an optical fibre bundle as an imaging guide, fundamentally limits image quality by producing artefacts, noise, and relatively low contrast and resolution. The reconstruction approaches in clinical pCLE products do not fully alleviate these problems. Consequently, image quality remains a barrier that curbs the full potential of pCLE. Enhancing the image quality of pCLE in real-time remains a challenge. The research in this thesis is a response to this need. I have developed dedicated online super-resolution methods that account for the physics of the image acquisition process. These methods have the potential to replace existing reconstruction algorithms without interfering with the fibre design or the hardware of the device. In this thesis, novel processing pipelines are proposed for enhancing the image quality of pCLE. First, I explored a learning-based super-resolution method that relies on mapping from the low to the high-resolution space. Due to the lack of high-resolution pCLE, I proposed to simulate high-resolution data and use it as a ground truth model that is based on the pCLE acquisition physics. However, pCLE images are reconstructed from irregularly distributed fibre signals, and grid-based Convolutional Neural Networks are not designed to take irregular data as input. To alleviate this problem, I designed a new trainable layer that embeds Nadaraya- Watson regression. Finally, I proposed a novel blind super-resolution approach by deploying unsupervised zero-shot learning accompanied by a down-sampling kernel crafted for pCLE. I evaluated these new methods in two ways: a robust image quality assessment and a perceptual quality test assessed by clinical experts. The results demonstrate that the proposed super-resolution pipelines are superior to the current reconstruction algorithm in terms of image quality and clinician preference

Ergonomics of the Operative Field in Paediatric Minimal Access Surgery

Imperial Users onl

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

High-definition colonoscopy versus Endocuff versus EndoRings versus Full-Spectrum Endoscopy for adenoma detection at colonoscopy: a multicenter randomized trial

Background Devices used to improve polyp detection during colonoscopy have seldom been compared with each other. Methods We performed a 3-center prospective randomized trial comparing high-definition (HD) forward-viewing colonoscopy alone to HD with Endocuff to HD with EndoRings to the Full Spectrum Endoscopy (FUSE) system. Patients were age ≥50 years and had routine indications and intact colons. The study colonoscopists were all proven high-level detectors. The primary endpoint was adenomas per colonoscopy (APC) Results Among 1,188 patients who completed the study, APC with Endocuff (APC Mean ± SD 1.82 ± 2.58), EndoRings (1.55 ± 2.42), and standard HD colonoscopy (1.53 ± 2.33) were all higher than FUSE (1.30 ± 1.96,) (p<0.001 for APC). Endocuff was higher than standard HD colonoscopy for APC (p=0.014) . Mean cecal insertion times with FUSE (468 ± 311 seconds) and EndoRings (403 ± 263 seconds) were both longer than with Endocuff (354 ± 216 seconds) (p=0.006 and 0.018, respectively). Conclusions For high-level detectors at colonoscopy, forward-viewing HD instruments dominate the FUSE system, indicating that for these examiners image resolution trumps angle of view. Further, Endocuff is a dominant strategy over EndoRings and no mucosal exposure device on a forward-viewing HD colonoscope