3 research outputs found
Deep Reinforcement Learning Based System for Intraoperative Hyperspectral Video Autofocusing
Hyperspectral imaging (HSI) captures a greater level of spectral detail than
traditional optical imaging, making it a potentially valuable intraoperative
tool when precise tissue differentiation is essential. Hardware limitations of
current optical systems used for handheld real-time video HSI result in a
limited focal depth, thereby posing usability issues for integration of the
technology into the operating room. This work integrates a focus-tunable liquid
lens into a video HSI exoscope, and proposes novel video autofocusing methods
based on deep reinforcement learning. A first-of-its-kind robotic focal-time
scan was performed to create a realistic and reproducible testing dataset. We
benchmarked our proposed autofocus algorithm against traditional policies, and
found our novel approach to perform significantly () better than
traditional techniques ( mean absolute focal error compared to
). In addition, we performed a blinded usability trial by having
two neurosurgeons compare the system with different autofocus policies, and
found our novel approach to be the most favourable, making our system a
desirable addition for intraoperative HSI.Comment: To be presented at MICCAI 202
Lightfield hyperspectral imaging in neuro-oncology surgery: an IDEAL 0 and 1 study
IntroductionHyperspectral imaging (HSI) has shown promise in the field of intra-operative imaging and tissue differentiation as it carries the capability to provide real-time information invisible to the naked eye whilst remaining label free. Previous iterations of intra-operative HSI systems have shown limitations, either due to carrying a large footprint limiting ease of use within the confines of a neurosurgical theater environment, having a slow image acquisition time, or by compromising spatial/spectral resolution in favor of improvements to the surgical workflow. Lightfield hyperspectral imaging is a novel technique that has the potential to facilitate video rate image acquisition whilst maintaining a high spectral resolution. Our pre-clinical and first-in-human studies (IDEAL 0 and 1, respectively) demonstrate the necessary steps leading to the first in-vivo use of a real-time lightfield hyperspectral system in neuro-oncology surgery.MethodsA lightfield hyperspectral camera (Cubert Ultris ×50) was integrated in a bespoke imaging system setup so that it could be safely adopted into the open neurosurgical workflow whilst maintaining sterility. Our system allowed the surgeon to capture in-vivo hyperspectral data (155 bands, 350–1,000 nm) at 1.5 Hz. Following successful implementation in a pre-clinical setup (IDEAL 0), our system was evaluated during brain tumor surgery in a single patient to remove a posterior fossa meningioma (IDEAL 1). Feedback from the theater team was analyzed and incorporated in a follow-up design aimed at implementing an IDEAL 2a study.ResultsFocusing on our IDEAL 1 study results, hyperspectral information was acquired from the cerebellum and associated meningioma with minimal disruption to the neurosurgical workflow. To the best of our knowledge, this is the first demonstration of HSI acquisition with 100+ spectral bands at a frame rate over 1Hz in surgery.DiscussionThis work demonstrated that a lightfield hyperspectral imaging system not only meets the design criteria and specifications outlined in an IDEAL-0 (pre-clinical) study, but also that it can translate into clinical practice as illustrated by a successful first in human study (IDEAL 1). This opens doors for further development and optimisation, given the increasing evidence that hyperspectral imaging can provide live, wide-field, and label-free intra-operative imaging and tissue differentiation