Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Intraoperative bone registration: An implementation in orthopaedic surgery using polaris vicra system

The introduction of the surgical navigation system in the operating room has brought a great development to the surgeries. Nowadays, the navigated surgeries are present in several fields of medicine. This is a technology that requires an optical tracking system to perform the registration procedure during an intraoperative situation. The preoperative plan consists in performing a 3D reconstruction of the treated region and planning all procedures during the surgery. The registration is an important procedure once it calculates the transformation between the coordinate system of the preoperative plan on the software and the coordinate system of the surgical room in order to be able the visualization of the position and orientation of the surgical instruments in relation to the 3D model obtained in the preoperative plan. A workflow in 3DSlicer software to perform the registration procedure is presented in this paper with all required steps to achieve the navigated surgery. Also, it is presented the results obtained from the registration in order to evaluate the procedure

Advanced cranial navigation

Neurosurgery is performed with extremely low margins of error. Surgical inaccuracy may have disastrous consequences. The overall aim of this thesis was to improve accuracy in cranial neurosurgical procedures by the application of new technical aids. Two technical methods were evaluated: augmented reality (AR) for surgical navigation (Papers I-II) and the optical technique of diffuse reflectance spectroscopy (DRS) for real-time tissue identification (Papers III-V). Minimally invasive skull-base endoscopy has several potential benefits compared to traditional craniotomy, but approaching the skull base through this route implies that at-risk organs and surgical targets are covered by bone and out of the surgeon’s direct line of sight. In Paper I, a new application for AR-navigated endoscopic skull-base surgery, based on an augmented-reality surgical navigation (ARSN) system, was developed. The accuracy of the system, defined by mean target registration error (TRE), was evaluated and found to be 0.55±0.24 mm, the lowest value reported error in the literature. As a first step toward the development of a cranial application for AR navigation, in Paper II this ARSN system was used to enable insertions of biopsy needles and external ventricular drainages (EVDs). The technical accuracy (i.e., deviation from the target or intended path) and efficacy (i.e., insertion time) were assessed on a 3D-printed realistic, anthropomorphic skull and brain phantom; Thirty cranial biopsies and 10 EVD insertions were performed. Accuracy for biopsy was 0.8±0.43 mm with a median insertion time of 149 (87-233) seconds, and for EVD accuracy was 2.9±0.8 mm at the tip with a median angular deviation of 0.7±0.5° and a median insertion time of 188 (135-400) seconds. Glial tumors grow diffusely in the brain, and patient survival is correlated with the extent of tumor removal. Tumor borders are often invisible. Resection beyond borders as defined by conventional methods may further improve a patient’s prognosis. In Paper III, DRS was evaluated for discrimination between glioma and normal brain tissue ex vivo. DRS spectra and histology were acquired from 22 tumor samples and 9 brain tissue samples retrieved from 30 patients. Sensitivity and specificity for the detection of low-grade gliomas were 82.0% and 82.7%, respectively, with an AUC of 0.91. Acute ischemic stroke caused by large vessel occlusion is treated with endovascular thrombectomy, but treatment failure can occur when clot composition and thrombectomy technique are mismatched. Intra-procedural knowledge of clot composition could guide the choice of treatment modality. In Paper IV, DRS, in vivo, was evaluated for intravascular clot characterization. Three types of clot analogs, red blood cell (RBC)-rich, fibrin-rich and mixed clots, were injected into the external carotids of a domestic pig. An intravascular DRS probe was used for in-situ measurements of clots, blood, and vessel walls, and the spectral data were analyzed. DRS could differentiate clot types, vessel walls, and blood in vivo (p<0,001). The sensitivity and specificity for detection were 73.8% and 98.8% for RBC clots, 100% and 100% for mixed clots, and 80.6% and 97.8% for fibrin clots, respectively. Paper V evaluated DRS for characterization of human clot composition ex vivo: 45 clot units were retrieved from 29 stroke patients and examined with DRS and histopathological evaluation. DRS parameters correlated with clot RBC fraction (R=81, p<0.001) and could be used for the classification of clot type with sensitivity and specificity rates for the detection of RBC-rich clots of 0.722 and 0.846, respectively. Applied in an intravascular probe, DRS may provide intra-procedural information on clot composition to improve endovascular thrombectomy efficiency

Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery.

Sequential multispectral imaging is an acquisition technique that involves collecting images of a target at different wavelengths, to compile a spectrum for each pixel. In surgical applications it suffers from low illumination levels and motion artefacts. A three-channel rigid endoscope system has been developed that allows simultaneous recording of stereoscopic and multispectral images. Salient features on the tissue surface may be tracked during the acquisition in the stereo cameras and, using multiple camera triangulation techniques, this information used to align the multispectral images automatically even though the tissue or camera is moving. This paper describes a detailed validation of the set-up in a controlled experiment before presenting the first in vivo use of the device in a porcine minimally invasive surgical procedure. Multispectral images of the large bowel were acquired and used to extract the relative concentration of haemoglobin in the tissue despite motion due to breathing during the acquisition. Using the stereoscopic information it was also possible to overlay the multispectral information on the reconstructed 3D surface. This experiment demonstrates the ability of this system for measuring blood perfusion changes in the tissue during surgery and its potential use as a platform for other sequential imaging modalities

Autofluorescence lifetime augmented reality as a means for real-time robotic surgery guidance in human patients.

Due to loss of tactile feedback the assessment of tumor margins during robotic surgery is based only on visual inspection, which is neither significantly sensitive nor specific. Here we demonstrate time-resolved fluorescence spectroscopy (TRFS) as a novel technique to complement the visual inspection of oral cancers during transoral robotic surgery (TORS) in real-time and without the need for exogenous contrast agents. TRFS enables identification of cancerous tissue by its distinct autofluorescence signature that is associated with the alteration of tissue structure and biochemical profile. A prototype TRFS instrument was integrated synergistically with the da Vinci Surgical robot and the combined system was validated in swine and human patients. Label-free and real-time assessment and visualization of tissue biochemical features during robotic surgery procedure, as demonstrated here, not only has the potential to improve the intraoperative decision making during TORS but also other robotic procedures without modification of conventional clinical protocols

Multimodal optical systems for clinical oncology

This thesis presents three multimodal optical (light-based) systems designed to improve the capabilities of existing optical modalities for cancer diagnostics and theranostics. Optical diagnostic and therapeutic modalities have seen tremendous success in improving the detection, monitoring, and treatment of cancer. For example, optical spectroscopies can accurately distinguish between healthy and diseased tissues, fluorescence imaging can light up tumours for surgical guidance, and laser systems can treat many epithelial cancers. However, despite these advances, prognoses for many cancers remain poor, positive margin rates following resection remain high, and visual inspection and palpation remain crucial for tumour detection. The synergistic combination of multiple optical modalities, as presented here, offers a promising solution. The first multimodal optical system (Chapter 3) combines Raman spectroscopic diagnostics with photodynamic therapy using a custom-built multimodal optical probe. Crucially, this system demonstrates the feasibility of nanoparticle-free theranostics, which could simplify the clinical translation of cancer theranostic systems without sacrificing diagnostic or therapeutic benefit. The second system (Chapter 4) applies computer vision to Raman spectroscopic diagnostics to achieve spatial spectroscopic diagnostics. It provides an augmented reality display of the surgical field-of-view, overlaying spatially co-registered spectroscopic diagnoses onto imaging data. This enables the translation of Raman spectroscopy from a 1D technique to a 2D diagnostic modality and overcomes the trade-off between diagnostic accuracy and field-of-view that has limited optical systems to date. The final system (Chapter 5) integrates fluorescence imaging and Raman spectroscopy for fluorescence-guided spatial spectroscopic diagnostics. This facilitates macroscopic tumour identification to guide accurate spectroscopic margin delineation, enabling the spectroscopic examination of suspicious lesions across large tissue areas. Together, these multimodal optical systems demonstrate that the integration of multiple optical modalities has potential to improve patient outcomes through enhanced tumour detection and precision-targeted therapies.Open Acces

Goggle Augmented Imaging and Navigation System for Fluorescence-Guided Surgery

Surgery remains the only curative option for most solid tumors. The standard-of-care usually involves tumor resection and sentinel lymph node biopsy for cancer staging. Surgeons rely on their vision and touch to distinguish healthy from cancer tissue during surgery, often leading to incomplete tumor resection that necessitates repeat surgery. Sentinel lymph node biopsy by conventional radioactive tracking exposes patients and caregivers to ionizing radiation, while blue dye tracking stains the tissue highlighting only superficial lymph nodes. Improper identification of sentinel lymph nodes may misdiagnose the stage of the cancer. Therefore there is a clinical need for accurate intraoperative tumor and sentinel lymph node visualization. Conventional imaging modalities such as x-ray computed tomography, positron emission tomography, magnetic resonance imaging, and ultrasound are excellent for preoperative cancer diagnosis and surgical planning. However, they are not suitable for intraoperative use, due to bulky complicated hardware, high cost, non-real-time imaging, severe restrictions to the surgical workflow and lack of sufficient resolution for tumor boundary assessment. This has propelled interest in fluorescence-guided surgery, due to availability of simple hardware that can achieve real-time, high resolution and sensitive imaging. Near-infrared fluorescence imaging is of particular interest due to low background absorbance by photoactive biomolecules, enabling thick tissue assessment. As a result several near-infrared fluorescence-guided surgery systems have been developed. However, they are limited by bulky hardware, disruptive information display and non-matched field of view to the user. To address these limitations we have developed a compact, light-weight and wearable goggle augmented imaging and navigation system (GAINS). It detects the near-infrared fluorescence from a tumor accumulated contrast agent, along with the normal color view and displays accurately aligned, color-fluorescence images via a head-mounted display worn by the surgeon, in real-time. GAINS is a platform technology and capable of very sensitive fluorescence detection. Image display options include both video see-through and optical see-through head-mounted displays for high-contrast image guidance as well as direct visual access to the surgical bed. Image capture options from large field of view camera as well high magnification handheld microscope, ensures macroscopic as well as microscopic assessment of the tumor bed. Aided by tumor targeted near-infrared contrast agents, GAINS guided complete tumor resection in subcutaneous, metastatic and spontaneous mouse models of cancer with high sensitivity and specificity, in real-time. Using a clinically-approved near-infrared contrast agent, GAINS provided real-time image guidance for accurate visualization of lymph nodes in a porcine model and sentinel lymph nodes in human breast cancer and melanoma patients with high sensitivity. This work has addressed issues that have limited clinical adoption of fluorescence-guided surgery and paved the way for research into developing this approach towards standard-of-care practice that can potentially improve surgical outcomes in cancer

Performance of image guided navigation in laparoscopic liver surgery – A systematic review

Background: Compared to open surgery, minimally invasive liver resection has improved short term outcomes. It is however technically more challenging. Navigated image guidance systems (IGS) are being developed to overcome these challenges. The aim of this systematic review is to provide an overview of their current capabilities and limitations. Methods: Medline, Embase and Cochrane databases were searched using free text terms and corresponding controlled vocabulary. Titles and abstracts of retrieved articles were screened for inclusion criteria. Due to the heterogeneity of the retrieved data it was not possible to conduct a meta-analysis. Therefore results are presented in tabulated and narrative format. Results: Out of 2015 articles, 17 pre-clinical and 33 clinical papers met inclusion criteria. Data from 24 articles that reported on accuracy indicates that in recent years navigation accuracy has been in the range of 8–15 mm. Due to discrepancies in evaluation methods it is difficult to compare accuracy metrics between different systems. Surgeon feedback suggests that current state of the art IGS may be useful as a supplementary navigation tool, especially in small liver lesions that are difficult to locate. They are however not able to reliably localise all relevant anatomical structures. Only one article investigated IGS impact on clinical outcomes. Conclusions: Further improvements in navigation accuracy are needed to enable reliable visualisation of tumour margins with the precision required for oncological resections. To enhance comparability between different IGS it is crucial to find a consensus on the assessment of navigation accuracy as a minimum reporting standard

Systems and technologies for objective evaluation of technical skills in laparoscopic surgery

Minimally invasive surgery is a highly demanding surgical approach regarding technical requirements for the surgeon, who must be trained in order to perform a safe surgical intervention. Traditional surgical education in minimally invasive surgery is commonly based on subjective criteria to quantify and evaluate surgical abilities, which could be potentially unsafe for the patient. Authors, surgeons and associations are increasingly demanding the development of more objective assessment tools that can accredit surgeons as technically competent. This paper describes the state of the art in objective assessment methods of surgical skills. It gives an overview on assessment systems based on structured checklists and rating scales, surgical simulators, and instrument motion analysis. As a future work, an objective and automatic assessment method of surgical skills should be standardized as a means towards proficiency-based curricula for training in laparoscopic surgery and its certification

Hand eye coordination in surgery

The coordination of the hand in response to visual target selection has always been regarded as an essential quality in a range of professional activities. This quality has thus far been elusive to objective scientific measurements, and is usually engulfed in the overall performance of the individuals. Parallels can be drawn to surgery, especially Minimally Invasive Surgery (MIS), where the physical constraints imposed by the arrangements of the instruments and visualisation methods require certain coordination skills that are unprecedented. With the current paradigm shift towards early specialisation in surgical training and shortened focused training time, selection process should identify trainees with the highest potentials in certain specific skills. Although significant effort has been made in objective assessment of surgical skills, it is only currently possible to measure surgeons’ abilities at the time of assessment. It has been particularly difficult to quantify specific details of hand-eye coordination and assess innate ability of future skills development. The purpose of this thesis is to examine hand-eye coordination in laboratory-based simulations, with a particular emphasis on details that are important to MIS. In order to understand the challenges of visuomotor coordination, movement trajectory errors have been used to provide an insight into the innate coordinate mapping of the brain. In MIS, novel spatial transformations, due to a combination of distorted endoscopic image projections and the “fulcrum” effect of the instruments, accentuate movement generation errors. Obvious differences in the quality of movement trajectories have been observed between novices and experts in MIS, however, this is difficult to measure quantitatively. A Hidden Markov Model (HMM) is used in this thesis to reveal the underlying characteristic movement details of a particular MIS manoeuvre and how such features are exaggerated by the introduction of rotation in the endoscopic camera. The proposed method has demonstrated the feasibility of measuring movement trajectory quality by machine learning techniques without prior arbitrary classification of expertise. Experimental results have highlighted these changes in novice laparoscopic surgeons, even after a short period of training. The intricate relationship between the hands and the eyes changes when learning a skilled visuomotor task has been previously studied. Reactive eye movement, when visual input is used primarily as a feedback mechanism for error correction, implies difficulties in hand-eye coordination. As the brain learns to adapt to this new coordinate map, eye movements then become predictive of the action generated. The concept of measuring this spatiotemporal relationship is introduced as a measure of hand-eye coordination in MIS, by comparing the Target Distance Function (TDF) between the eye fixation and the instrument tip position on the laparoscopic screen. Further validation of this concept using high fidelity experimental tasks is presented, where higher cognitive influence and multiple target selection increase the complexity of the data analysis. To this end, Granger-causality is presented as a measure of the predictability of the instrument movement with the eye fixation pattern. Partial Directed Coherence (PDC), a frequency-domain variation of Granger-causality, is used for the first time to measure hand-eye coordination. Experimental results are used to establish the strengths and potential pitfalls of the technique. To further enhance the accuracy of this measurement, a modified Jensen-Shannon Divergence (JSD) measure has been developed for enhancing the signal matching algorithm and trajectory segmentations. The proposed framework incorporates high frequency noise filtering, which represents non-purposeful hand and eye movements. The accuracy of the technique has been demonstrated by quantitative measurement of multiple laparoscopic tasks by expert and novice surgeons. Experimental results supporting visual search behavioural theory are presented, as this underpins the target selection process immediately prior to visual motor action generation. The effects of specialisation and experience on visual search patterns are also examined. Finally, pilot results from functional brain imaging are presented, where the Posterior Parietal Cortical (PPC) activation is measured using optical spectroscopy techniques. PPC has been demonstrated to involve in the calculation of the coordinate transformations between the visual and motor systems, which establishes the possibilities of exciting future studies in hand-eye coordination