17 research outputs found
Semantic segmentation of surgical hyperspectral images under geometric domain shifts
Robust semantic segmentation of intraoperative image data could pave the way
for automatic surgical scene understanding and autonomous robotic surgery.
Geometric domain shifts, however, although common in real-world open surgeries
due to variations in surgical procedures or situs occlusions, remain a topic
largely unaddressed in the field. To address this gap in the literature, we (1)
present the first analysis of state-of-the-art (SOA) semantic segmentation
networks in the presence of geometric out-of-distribution (OOD) data, and (2)
address generalizability with a dedicated augmentation technique termed "Organ
Transplantation" that we adapted from the general computer vision community.
According to a comprehensive validation on six different OOD data sets
comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs
semantically annotated with 19 classes, we demonstrate a large performance drop
of SOA organ segmentation networks applied to geometric OOD data. Surprisingly,
this holds true not only for conventional RGB data (drop of Dice similarity
coefficient (DSC) by 46 %) but also for HSI data (drop by 45 %), despite the
latter's rich information content per pixel. Using our augmentation scheme
improves on the SOA DSC by up to 67 % (RGB) and 90 % (HSI) and renders
performance on par with in-distribution performance on real OOD test data. The
simplicity and effectiveness of our augmentation scheme makes it a valuable
network-independent tool for addressing geometric domain shifts in semantic
scene segmentation of intraoperative data. Our code and pre-trained models are
available at https://github.com/IMSY-DKFZ/htc.Comment: The first two authors (Jan Sellner and Silvia Seidlitz) contributed
equally to this pape
Hyperspectral Tissue Classification
<p>This release features our PyTorch Conference poster <a href="https://github.com/IMSY-DKFZ/htc/blob/main/README.md#-dealing-with-io-bottlenecks-in-high-throughput-model-training">Dealing with I/O bottlenecks in high-throughput model training</a> including the code for the experiments.</p>
<p>Additionally, the code is now compatible with PyTorch 2.1.</p>If you use the htc framework, please consider citing our 2022 publication in the Medical Image Analysis journal
Wie sich Licht für Forschung und Technik nutzbar machen lässt - Die Universität Heidelberg ist Gastgeber der Photonik-Akademie für Studierende.
Video-rate multispectral imaging in laparoscopic surgery: First-in-human application
Multispectral and hyperspectral imaging (MSI/HSI) can provide clinically
relevant information on morphological and functional tissue properties.
Application in the operating room (OR), however, has so far been limited by
complex hardware setups and slow acquisition times. To overcome these
limitations, we propose a novel imaging system for video-rate spectral imaging
in the clinical workflow. The system integrates a small snapshot multispectral
camera with a standard laparoscope and a clinically commonly used light source,
enabling the recording of multispectral images with a spectral dimension of 16
at a frame rate of 25 Hz. An ongoing in patient study shows that multispectral
recordings from this system can help detect perfusion changes in partial
nephrectomy surgery, thus opening the doors to a wide range of clinical
applications
Light source calibration for multispectral imaging in surgery
Purpose!#!Live intra-operative functional imaging has multiple potential clinical applications, such as localization of ischemia, assessment of organ transplantation success and perfusion monitoring. Recent research has shown that live monitoring of functional tissue properties, such as tissue oxygenation and blood volume fraction, is possible using multispectral imaging in laparoscopic surgery. While the illuminant spectrum is typically kept constant in laparoscopic surgery and can thus be estimated from preoperative calibration images, a key challenge in open surgery originates from the dynamic changes of lighting conditions.!##!Methods!#!The present paper addresses this challenge with a novel approach to light source calibration based on specular highlight analysis. It involves the acquisition of low-exposure time images serving as a basis for recovering the illuminant spectrum from pixels that contain a dominant specular reflectance component.!##!Results!#!Comprehensive in silico and in vivo experiments with a range of different light sources demonstrate that our approach enables an accurate and robust recovery of the illuminant spectrum in the field of view of the camera, which results in reduced errors with respect to the estimation of functional tissue properties. Our approach further outperforms state-of-the-art methods proposed in the field of computer vision.!##!Conclusion!#!Our results suggest that low-exposure multispectral images are well suited for light source calibration via specular highlight analysis. This work thus provides an important first step toward live functional imaging in open surgery
Personality is associated with perceived health and functional status in older primary care patients.
HeiPorSPECTRAL - the Heidelberg Porcine HyperSPECTRAL Imaging Dataset of 20 Physiological Organs
Abstract Hyperspectral Imaging (HSI) is a relatively new medical imaging modality that exploits an area of diagnostic potential formerly untouched. Although exploratory translational and clinical studies exist, no surgical HSI datasets are openly accessible to the general scientific community. To address this bottleneck, this publication releases HeiPorSPECTRAL ( https://www.heiporspectral.org ; https://doi.org/10.5281/zenodo.7737674 ), the first annotated high-quality standardized surgical HSI dataset. It comprises 5,758 spectral images acquired with the TIVITA® Tissue and annotated with 20 physiological porcine organs from 8 pigs per organ distributed over a total number of 11 pigs. Each HSI image features a resolution of 480 × 640 pixels acquired over the 500–1000 nm wavelength range. The acquisition protocol has been designed such that the variability of organ spectra as a function of several parameters including the camera angle and the individual can be assessed. A comprehensive technical validation confirmed both the quality of the raw data and the annotations. We envision potential reuse within this dataset, but also its reuse as baseline data for future research questions outside this dataset. Measurement(s) Spectral Reflectance Technology Type(s) Hyperspectral Imaging Sample Characteristic - Organism Sus scrof