Image overlay surgery based on augmented reality : a systematic review

Acknowledgements We thank the staff of the Medical Library of the University of Aberdeen for their advice and Prof. Jennifer Cleland and Dr Jenny Gregory for discussion and support. This work was funded by the Roland Sutton Academic Trust (0053/R/17) and an Elphinstone PhD Scholarship from the University of Aberdeen.Postprin

Intra-operative applications of augmented reality in glioma surgery: a systematic review

BackgroundAugmented reality (AR) is increasingly being explored in neurosurgical practice. By visualizing patient-specific, three-dimensional (3D) models in real time, surgeons can improve their spatial understanding of complex anatomy and pathology, thereby optimizing intra-operative navigation, localization, and resection. Here, we aimed to capture applications of AR in glioma surgery, their current status and future potential.MethodsA systematic review of the literature was conducted. This adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline. PubMed, Embase, and Scopus electronic databases were queried from inception to October 10, 2022. Leveraging the Population, Intervention, Comparison, Outcomes, and Study design (PICOS) framework, study eligibility was evaluated in the qualitative synthesis. Data regarding AR workflow, surgical application, and associated outcomes were then extracted. The quality of evidence was additionally examined, using hierarchical classes of evidence in neurosurgery.ResultsThe search returned 77 articles. Forty were subject to title and abstract screening, while 25 proceeded to full text screening. Of these, 22 articles met eligibility criteria and were included in the final review. During abstraction, studies were classified as “development” or “intervention” based on primary aims. Overall, AR was qualitatively advantageous, due to enhanced visualization of gliomas and critical structures, frequently aiding in maximal safe resection. Non-rigid applications were also useful in disclosing and compensating for intra-operative brain shift. Irrespective, there was high variance in registration methods and measurements, which considerably impacted projection accuracy. Most studies were of low-level evidence, yielding heterogeneous results.ConclusionsAR has increasing potential for glioma surgery, with capacity to positively influence the onco-functional balance. However, technical and design limitations are readily apparent. The field must consider the importance of consistency and replicability, as well as the level of evidence, to effectively converge on standard approaches that maximize patient benefit

Goggle Augmented Imaging and Navigation System for Fluorescence-Guided Surgery

Surgery remains the only curative option for most solid tumors. The standard-of-care usually involves tumor resection and sentinel lymph node biopsy for cancer staging. Surgeons rely on their vision and touch to distinguish healthy from cancer tissue during surgery, often leading to incomplete tumor resection that necessitates repeat surgery. Sentinel lymph node biopsy by conventional radioactive tracking exposes patients and caregivers to ionizing radiation, while blue dye tracking stains the tissue highlighting only superficial lymph nodes. Improper identification of sentinel lymph nodes may misdiagnose the stage of the cancer. Therefore there is a clinical need for accurate intraoperative tumor and sentinel lymph node visualization. Conventional imaging modalities such as x-ray computed tomography, positron emission tomography, magnetic resonance imaging, and ultrasound are excellent for preoperative cancer diagnosis and surgical planning. However, they are not suitable for intraoperative use, due to bulky complicated hardware, high cost, non-real-time imaging, severe restrictions to the surgical workflow and lack of sufficient resolution for tumor boundary assessment. This has propelled interest in fluorescence-guided surgery, due to availability of simple hardware that can achieve real-time, high resolution and sensitive imaging. Near-infrared fluorescence imaging is of particular interest due to low background absorbance by photoactive biomolecules, enabling thick tissue assessment. As a result several near-infrared fluorescence-guided surgery systems have been developed. However, they are limited by bulky hardware, disruptive information display and non-matched field of view to the user. To address these limitations we have developed a compact, light-weight and wearable goggle augmented imaging and navigation system (GAINS). It detects the near-infrared fluorescence from a tumor accumulated contrast agent, along with the normal color view and displays accurately aligned, color-fluorescence images via a head-mounted display worn by the surgeon, in real-time. GAINS is a platform technology and capable of very sensitive fluorescence detection. Image display options include both video see-through and optical see-through head-mounted displays for high-contrast image guidance as well as direct visual access to the surgical bed. Image capture options from large field of view camera as well high magnification handheld microscope, ensures macroscopic as well as microscopic assessment of the tumor bed. Aided by tumor targeted near-infrared contrast agents, GAINS guided complete tumor resection in subcutaneous, metastatic and spontaneous mouse models of cancer with high sensitivity and specificity, in real-time. Using a clinically-approved near-infrared contrast agent, GAINS provided real-time image guidance for accurate visualization of lymph nodes in a porcine model and sentinel lymph nodes in human breast cancer and melanoma patients with high sensitivity. This work has addressed issues that have limited clinical adoption of fluorescence-guided surgery and paved the way for research into developing this approach towards standard-of-care practice that can potentially improve surgical outcomes in cancer

Bio-Inspired Multi-Spectral Image Sensor and Augmented Reality Display for Near-Infrared Fluorescence Image-Guided Surgery

Background: Cancer remains a major public health problem worldwide and poses a huge economic burden. Near-infrared (NIR) fluorescence image-guided surgery (IGS) utilizes molecular markers and imaging instruments to identify and locate tumors during surgical resection. Unfortunately, current state-of-the-art NIR fluorescence imaging systems are bulky, costly, and lack both fluorescence sensitivity under surgical illumination and co-registration accuracy between multimodal images. Additionally, the monitor-based display units are disruptive to the surgical workflow and are suboptimal at indicating the 3-dimensional position of labeled tumors. These major obstacles have prevented the wide acceptance of NIR fluorescence imaging as the standard of care for cancer surgery. The goal of this dissertation is to enhance cancer treatment by developing novel image sensors and presenting the information using holographic augmented reality (AR) display to the physician in intraoperative settings. Method: By mimicking the visual system of the Morpho butterfly, several single-chip, color-NIR fluorescence image sensors and systems were developed with CMOS technologies and pixelated interference filters. Using a holographic AR goggle platform, an NIR fluorescence IGS display system was developed. Optoelectronic evaluation was performed on the prototypes to evaluate the performance of each component, and small animal models and large animal models were used to verify the overall effectiveness of the integrated systems at cancer detection. Result: The single-chip bio-inspired multispectral logarithmic image sensor I developed has better main performance indicators than the state-of-the-art NIR fluorescence imaging instruments. The image sensors achieve up to 140 dB dynamic range. The sensitivity under surgical illumination achieves 6108 V/(mW/cm2), which is up to 25 times higher. The signal-to-noise ratio is up to 56 dB, which is 11 dB greater. These enable high sensitivity fluorescence imaging under surgical illumination. The pixelated interference filters enable temperature-independent co-registration accuracy between multimodal images. Pre-clinical trials with small animal model demonstrate that the sensor can achieve up to 95% sensitivity and 94% specificity with tumor-targeted NIR molecular probes. The holographic AR goggle provides the physician with a non-disruptive 3-dimensional display in the clinical setup. This is the first display system that co-registers a virtual image with human eyes and allows video rate image transmission. The imaging system is tested in the veterinary science operating room on canine patients with naturally occurring cancers. In addition, a time domain pulse-width-modulation address-event-representation multispectral image sensor and a handheld multispectral camera prototype are developed. Conclusion: The major problems of current state-of-the-art NIR fluorescence imaging systems are successfully solved. Due to enhanced performance and user experience, the bio-inspired sensors and augmented reality display system will give medical care providers much needed technology to enable more accurate value-based healthcare

Life Sciences Program Tasks and Bibliography for FY 1997

This document includes information on all peer reviewed projects funded by the Office of Life and Microgravity Sciences and Applications, Life Sciences Division during fiscal year 1997. This document will be published annually and made available to scientists in the space life sciences field both as a hard copy and as an interactive internet web page