90 research outputs found
Physiological parameter estimation from multispectral images unleashed
Multispectral imaging in laparoscopy can provide tissue reflectance measurements for each point in the image at multiple wavelengths of light. These reflectances encode information on important physiological parameters not visible to the naked eye. Fast decoding of the data during surgery, however, remains challenging. While model-based methods suffer from inaccurate base assumptions, a major bottleneck related to competing machine learning-based solutions is the lack of labelled training data. In this paper, we address this issue with the first transfer learning-based method to physiological parameter estimation from multispectral images. It relies on a highly generic tissue model that aims to capture the full range of optical tissue parameters that can potentially be observed in vivo. Adaptation of the model to a specific clinical application based on unlabelled in vivo data is achieved using a new concept of domain adaptation that explicitly addresses the high variance often introduced by conventional covariance-shift correction methods. According to comprehensive in silico and in vivo experiments our approach enables accurate parameter estimation for various tissue types without the need for incorporating specific prior knowledge on optical properties and could thus pave the way for many exciting applications in multispectral laparoscopy
Out of distribution detection for intra-operative functional imaging
Multispectral optical imaging is becoming a key tool in the operating room.
Recent research has shown that machine learning algorithms can be used to
convert pixel-wise reflectance measurements to tissue parameters, such as
oxygenation. However, the accuracy of these algorithms can only be guaranteed
if the spectra acquired during surgery match the ones seen during training. It
is therefore of great interest to detect so-called out of distribution (OoD)
spectra to prevent the algorithm from presenting spurious results. In this
paper we present an information theory based approach to OoD detection based on
the widely applicable information criterion (WAIC). Our work builds upon recent
methodology related to invertible neural networks (INN). Specifically, we make
use of an ensemble of INNs as we need their tractable Jacobians in order to
compute the WAIC. Comprehensive experiments with in silico, and in vivo
multispectral imaging data indicate that our approach is well-suited for OoD
detection. Our method could thus be an important step towards reliable
functional imaging in the operating room.Comment: The final authenticated version is available online at
https://doi.org/10.1007/978-3-030-32689-0_
Multispectral image analysis in laparoscopy â A machine learning approach to live perfusion monitoring
Modern visceral surgery is often performed through small incisions. Compared to open surgery, these minimally invasive interventions result in smaller scars, fewer complications and a quicker recovery. While to the patients benefit, it has the drawback of limiting the physicianâs perception largely to that of visual feedback through a camera mounted on a rod lens: the laparoscope. Conventional laparoscopes are limited by âimitatingâ the human eye. Multispectral cameras remove this arbitrary restriction of recording only red, green and blue colors. Instead, they capture many specific bands of light. Although these could help characterize important indications such as ischemia and early stage adenoma, the lack of powerful digital image processing prevents realizing the techniqueâs full potential.
The primary objective of this thesis was to pioneer fluent functional multispectral imaging (MSI) in laparoscopy. The main technical obstacles were: (1) The lack of image analysis concepts that provide both high accuracy and speed. (2) Multispectral image recording is slow, typically ranging from seconds to minutes. (3) Obtaining a quantitative ground truth for the measurements is hard or even impossible.
To overcome these hurdles and enable functional laparoscopy, for the first time in this field physical models are combined with powerful machine learning techniques. The physical model is employed to create highly accurate simulations, which in turn teach the algorithm to rapidly relate multispectral pixels to underlying functional changes. To reduce the domain shift introduced by learning from simulations, a novel transfer learning approach automatically adapts generic simulations to match almost arbitrary
recordings of visceral tissue. In combination with the only available video-rate capable multispectral sensor, the method pioneers fluent perfusion monitoring with MSI. This system was carefully tested in a multistage process, involving in silico quantitative evaluations, tissue phantoms and a porcine study. Clinical applicability was ensured through in-patient recordings in the context of partial nephrectomy; in these, the novel system characterized ischemia live during the intervention. Verified against a fluorescence reference, the results indicate that fluent, non-invasive ischemia detection and monitoring is now possible.
In conclusion, this thesis presents the first multispectral laparoscope capable of videorate functional analysis. The system was successfully evaluated in in-patient trials, and future work should be directed towards evaluation of the system in a larger study. Due to the broad applicability and the large potential clinical benefit of the presented functional estimation approach, I am confident the descendants of this system are an integral part
of the next generation OR
Unsupervised Domain Transfer with Conditional Invertible Neural Networks
Synthetic medical image generation has evolved as a key technique for neural
network training and validation. A core challenge, however, remains in the
domain gap between simulations and real data. While deep learning-based domain
transfer using Cycle Generative Adversarial Networks and similar architectures
has led to substantial progress in the field, there are use cases in which
state-of-the-art approaches still fail to generate training images that produce
convincing results on relevant downstream tasks. Here, we address this issue
with a domain transfer approach based on conditional invertible neural networks
(cINNs). As a particular advantage, our method inherently guarantees cycle
consistency through its invertible architecture, and network training can
efficiently be conducted with maximum likelihood training. To showcase our
method's generic applicability, we apply it to two spectral imaging modalities
at different scales, namely hyperspectral imaging (pixel-level) and
photoacoustic tomography (image-level). According to comprehensive experiments,
our method enables the generation of realistic spectral data and outperforms
the state of the art on two downstream classification tasks (binary and
multi-class). cINN-based domain transfer could thus evolve as an important
method for realistic synthetic data generation in the field of spectral imaging
and beyond
A Review on Advances in Intra-operative Imaging for Surgery and Therapy: Imagining the Operating Room of the Future
none4openZaffino, Paolo; Moccia, Sara; De Momi, Elena; Spadea, Maria FrancescaZaffino, Paolo; Moccia, Sara; De Momi, Elena; Spadea, Maria Francesc
- âŠ