Automatic Search for Photoacoustic Marker Using Automated Transrectal Ultrasound

Real-time transrectal ultrasound (TRUS) image guidance during robot-assisted laparoscopic radical prostatectomy has the potential to enhance surgery outcomes. Whether conventional or photoacoustic TRUS is used, the robotic system and the TRUS must be registered to each other. Accurate registration can be performed using photoacoustic (PA markers). However, this requires a manual search by an assistant [19]. This paper introduces the first automatic search for PA markers using a transrectal ultrasound robot. This effectively reduces the challenges associated with the da Vinci-TRUS registration. This paper investigated the performance of three search algorithms in simulation and experiment: Weighted Average (WA), Golden Section Search (GSS), and Ternary Search (TS). For validation, a surgical prostate scenario was mimicked and various ex vivo tissues were tested. As a result, the WA algorithm can achieve 0.53 degree average error after 9 data acquisitions, while the TS and GSS algorithm can achieve 0.29 degree and 0.48 degree average errors after 28 data acquisitions.Comment: 13 pages, 9 figure

Uncertainty Quantification in Biophotonic Imaging using Invertible Neural Networks

Owing to high stakes in the field of healthcare, medical machine learning (ML) applications have to adhere to strict safety standards. In particular, their performance needs to be robust toward volatile clinical inputs. The aim of the work presented in this thesis was to develop a framework for uncertainty handling in medical ML applications as a way to increase their robustness and trustworthiness. In particular, it addresses three root causes for lack of robustness that can be deemed central to the successful clinical translation of ML methods: First, many tasks in medical imaging can be phrased in the language of inverse problems. Most common ML methods aimed at solving such inverse problems implicitly assume that they are well-posed, especially that the problem has a unique solution. However, the solution might be ambiguous. In this thesis, we introduce a data-driven method for analyzing the well-posedness of inverse problems. In addition, we propose a framework to validate the suggested method in a problem-aware manner. Second, simulation is an important tool for the development of medical ML systems due to small in vivo data sets and/or a lack of annotated references (e. g. spatially resolved blood oxygenation (sO 2 )). However, simulation introduces a new uncertainty to the ML pipeline as ML performance guarantees generally rely on the testing data being sufficiently similar to the training data. This thesis addresses the uncertainty by quantifying the domain gap between training and testing data via an out-of-distribution (OoD) detection approach. Third, we introduce a new paradigm for medical ML based on personalized models. In a data-scarce regime with high inter-patient variability, classical ML models cannot be assumed to generalize well to new patients. To overcome this problem, we propose to train ML models on a per-patient basis. This approach circumvents the inter-patient variability, but it requires training without a supervision signal. We address this issue via OoD detection, where the current status quo is encoded as in-distribution (ID) using a personalized ML model. Changes to the status quo are then detected as OoD. While these three facets might seem distinct, the suggested framework provides a unified view of them. The enabling technology is the so-called invertible neural network (INN), which can be used as a flexible and expressive (conditional) density estimator. In this way, they can encode solutions to inverse problems as a probability distribution as well as tackle OoD detection tasks via density-based scores, like the widely applicable information criterion (WAIC). The present work validates our framework on the example of biophotonic imaging. Biophotonic imaging promises the estimation of tissue parameters such as sO 2 in a non-invasive way by evaluating the “fingerprint” of the tissue in the light spectrum. We apply our framework to analyze the well-posedness of the tissue parameter estimation problem at varying spectral and spatial resolutions. We find that with sufficient spectral and/or spatial context, the sO 2 estimation problem is well-posed. Furthermore, we examine the realism of simulated biophotonic data using the proposed OoD approach to gauge the generalization capabilities of our ML models to in vivo data. Our analysis shows a considerable remaining domain gap between the in silico and in vivo spectra. Lastly, we validate the personalized ML approach on the example of non-invasive ischemia monitoring in minimally invasive kidney surgery, for which we developed the first-in-human laparoscopic multispectral imaging system. In our study, we find a strong OoD signal between perfused and ischemic kidney spectra. Furthermore, the proposed approach is video-rate capable. In conclusion, we successfully developed a framework for uncertainty handling in medical ML and validated it using a diverse set of medical ML tasks, highlighting the flexibility and potential impact of our approach. The framework opens the door to robust solutions to applications like (recording) device design, quality control for simulation pipelines, and personalized video-rate tissue parameter monitoring. In this way, this thesis facilitates the development of the next generation of trustworthy ML systems in medicine

A review of intrinsic optical imaging serial blockface histology (ICI-SBH) for whole rodent brain imaging

In recent years, multiple serial histology techniques were developed to enable whole rodent brain imaging in 3-D. The main driving forces behind the emergence of these imaging techniques were the genome-wide atlas of gene expression in the mouse brain, the pursuit of the mouse brain connectome, and the BigBrain project. These projects rely on the use of optical imaging to target neuronal structures with histological stains or fluorescent dyes that are either expressed by transgenic mice or injected at specific locations in the brain. Efforts to adapt the serial histology acquisition scheme to use intrinsic contrast imaging (ICI) were also put forward, thus leveraging the natural contrast of neuronal tissue. This review focuses on these efforts. First, the origin of optical contrast in brain tissue is discussed with emphasis on the various imaging modalities exploiting these contrast mechanisms. Serial blockface histology (SBH) systems using ICI modalities are then reported, followed by a review of some of their applications. These include validation studies and the creation of multimodal brain atlases at a micrometer resolution. The paper concludes with a perspective of future developments, calling for a consolidation of the SBH research and development efforts around the world. The goal would be to offer the neuroscience community a single standardized open-source SBH solution, including optical design, acquisition automation, reconstruction algorithms, and analysis pipelines

Multimodal optical spectroscopy for application in the life sciences

Many optical modalities are being investigated, applied, and further developed for non-invasive analysis and sensing in the life sciences. Often, the combination of two or more modalities is required for in depth analysis because of the complexity of the study objects and questions in this field. The work presents multimodal sensing concepts for applications ranging from non-invasive quantification of biomolecules in the living organism to supporting medical diagnosis showing the combined capabilities of Raman spectroscopy, optical coherence tomography, and optoacoustic