(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'

Techniques basées sur des modèles et apprentissage machine pour la reconstruction d’image non-linéaire en tomographie optique diffuse

La tomographie optique diffuse (TOD) est une modalité d’imagerie biomédicale 3D peu dispendieuse et non-invasive qui permet de reconstruire les propriétés optiques d’un tissu biologique. Le processus de reconstruction d’images en TOD est difficile à réaliser puisqu’il nécessite de résoudre un problème non-linéaire et mal posé. Les propriétés optiques sont calculées à partir des mesures de surface du milieu à l’étude. Dans ce projet, deux méthodes de reconstruction non-linéaire pour la TOD ont été développées. La première méthode utilise un modèle itératif, une approche encore en développement qu’on retrouve dans la littérature. L’approximation de la diffusion est le modèle utilisé pour résoudre le problème direct. Par ailleurs, la reconstruction d’image à été réalisée dans différents régimes, continu et temporel, avec des mesures intrinsèques et de fluorescence. Dans un premier temps, un algorithme de reconstruction en régime continu et utilisant des mesures multispectrales est développé pour reconstruire la concentration des chromophores qui se trouve dans différents types de tissus. Dans un second temps, un algorithme de reconstruction est développé pour calculer le temps de vie de différents marqueurs fluorescents à partir de mesures optiques dans le domaine temporel. Une approche innovatrice a été d’utiliser la totalité de l’information du signal temporel dans le but d’améliorer la reconstruction d’image. Par ailleurs, cet algorithme permettrait de distinguer plus de trois temps de vie, ce qui n’a pas encore été démontré en imagerie de fluorescence. La deuxième méthode qui a été développée utilise l’apprentissage machine et plus spécifiquement l’apprentissage profond. Un modèle d’apprentissage profond génératif est mis en place pour reconstruire la distribution de sources d’émissions de fluorescence à partir de mesures en régime continu. Il s’agit de la première utilisation d’un algorithme d’apprentissage profond appliqué à la reconstruction d’images en TOD de fluorescence. La validation de la méthode est réalisée avec une mire aux propriétés optiques connues dans laquelle sont inséres des marqueurs fluorescents. La robustesse de cette méthode est démontrée même dans les situations où le nombre de mesures est limité et en présence de bruit.Abstract : Diffuse optical tomography (DOT) is a low cost and noninvasive 3D biomedical imaging technique to reconstruct the optical properties of biological tissues. Image reconstruction in DOT is inherently a difficult problem, because the inversion process is nonlinear and ill-posed. During DOT image reconstruction, the optical properties of the medium are recovered from the boundary measurements at the surface of the medium. In this work, two approaches are proposed for non-linear DOT image reconstruction. The first approach relies on the use of iterative model-based image reconstruction, which is still under development for DOT and that can be found in the literature. A 3D forward model is developed based on the diffusion equation, which is an approximation of the radiative transfer equation. The forward model developed can simulate light propagation in complex geometries. Additionally, the forward model is developed to deal with different types of optical data such as continuous-wave (CW) and time-domain (TD) data for both intrinsic and fluorescence signals. First, a multispectral image reconstruction algorithm is developed to reconstruct the concentration of different tissue chromophores simultaneously from a set of CW measurements at different wavelengths. A second image reconstruction algorithm is developed to reconstruct the fluorescence lifetime (FLT) of different fluorescent markers from time-domain fluorescence measurements. In this algorithm, all the information contained in full temporal curves is used along with an acceleration technique to render the algorithm of practical use. Moreover, the proposed algorithm has the potential of being able to distinguish more than 3 FLTs, which is a first in fluorescence imaging. The second approach is based on machine learning techniques, in particular deep learning models. A deep generative model is proposed to reconstruct the fluorescence distribution map from CW fluorescence measurements. It is the first time that such a model is applied for fluorescence DOT image reconstruction. The performance of the proposed algorithm is validated with an optical phantom and a fluorescent marker. The proposed algorithm recovers the fluorescence distribution even from very noisy and sparse measurements, which is a big limitation in fluorescence DOT imaging

Bayesian field theoretic reconstruction of bond potential and bond mobility in single molecule force spectroscopy

Quantifying the forces between and within macromolecules is a necessary first step in understanding the mechanics of molecular structure, protein folding, and enzyme function and performance. In such macromolecular settings, dynamic single-molecule force spectroscopy (DFS) has been used to distort bonds. The resulting responses, in the form of rupture forces, work applied, and trajectories of displacements, have been used to reconstruct bond potentials. Such approaches often rely on simple parameterizations of one-dimensional bond potentials, assumptions on equilibrium starting states, and/or large amounts of trajectory data. Parametric approaches typically fail at inferring complex-shaped bond potentials with multiple minima, while piecewise estimation may not guarantee smooth results with the appropriate behavior at large distances. Existing techniques, particularly those based on work theorems, also do not address spatial variations in the diffusivity that may arise from spatially inhomogeneous coupling to other degrees of freedom in the macromolecule, thereby presenting an incomplete picture of the overall bond dynamics. To solve these challenges, we have developed a comprehensive empirical Bayesian approach that incorporates data and regularization terms directly into a path integral. All experiemental and statistical parameters in our method are estimated empirically directly from the data. Upon testing our method on simulated data, our regularized approach requires fewer data and allows simultaneous inference of both complex bond potentials and diffusivity profiles.Comment: In review - Python source code available on github. Abridged abstract on arXi

Adaptive finite element methods for fluorescence enhanced optical tomography

Fluorescence enhanced optical tomography is a promising molecular imaging modality which employs a near infrared fluorescent molecule as an imaging agent and time-dependent measurements of fluorescent light propagation and generation. In this dissertation a novel fluorescence tomography algorithm is proposed to reconstruct images of targets contrasted by fluorescence within the tissues from boundary fluorescence emission measurements. An adaptive finite element based reconstruction algorithm for high resolution, fluorescence tomography was developed and validated with non-contact, planewave frequency-domain fluorescence measurements on a tissue phantom. The image reconstruction problem was posed as an optimization problem in which the fluorescence optical property map which minimized the difference between the experimentally observed boundary fluorescence and that predicted from the diffusion model was sought. A regularized Gauss-Newton algorithm was derived and dual adaptive meshes were employed for solution of coupled photon diffusion equations and for updating the fluorescence optical property map in the tissue phantom. The algorithm was developed in a continuous function space setting in a mesh independent manner. This allowed the meshes to adapt during the tomography process to yield high resolution images of fluorescent targets and to accurately simulate the light propagation in tissue phantoms from area-illumination. Frequency-domain fluorescence data collected at the illumination surface was used for reconstructing the fluorescence yield distribution in a 512 cm3, tissue phantom filled with 1% Liposyn solution. Fluorescent targets containing 1 micro-molar Indocyanine Green solution in 1% Liposyn and were suspended at the depths of up to 2cm from the illumination surface. Fluorescence measurements at the illumination surface were acquired by a gain-modulated image intensified CCD camera system outfitted with holographic band rejection and optical band pass filters. Excitation light at the phantom surface source was quantified by utilizing cross polarizers. Rayleigh resolution studies to determine the minimum detectable sepatation of two embedded fluorescent targets was attempted and in the absence of measurement noise, resolution down to the transport limit of 1mm was attained. The results of this work demonstrate the feasibility of high-resolution, molecular tomography in clinic with rapid non-contact area measurements

Mathematics of biomedical imaging today—a perspective

Biomedical imaging is a fascinating, rich and dynamic research area, which has huge importance in biomedical research and clinical practice alike. The key technology behind the processing, and automated analysis and quantification of imaging data is mathematics. Starting with the optimisation of the image acquisition and the reconstruction of an image from indirect tomographic measurement data, all the way to the automated segmentation of tumours in medical images and the design of optimal treatment plans based on image biomarkers, mathematics appears in all of these in different flavours. Non-smooth optimisation in the context of sparsity-promoting image priors, partial differential equations for image registration and motion estimation, and deep neural networks for image segmentation, to name just a few. In this article, we present and review mathematical topics that arise within the whole biomedical imaging pipeline, from tomographic measurements to clinical support tools, and highlight some modern topics and open problems. The article is addressed to both biomedical researchers who want to get a taste of where mathematics arises in biomedical imaging as well as mathematicians who are interested in what mathematical challenges biomedical imaging research entails

An Efficient Numerical Method for General

Reconstruction algorithms for fluorescence tomography have to address two crucial issues : 1) the ill-posedness of the reconstruction problem, 2) the large scale of numerical problems arising from imaging of 3-D samples. Our contribution is the design and implementation of a reconstruction algorithm that incorporates general Lp regularization (p ≥ 1). The originality of this work lies in the application of general Lp constraints to fluorescence tomography, combined with an efficient matrix-free strategy that enables the algorithm to deal with large reconstruction problems at reduced memory and computational costs. In the experimental part, we specialize the application of the algorithm to the case of sparsity promoting constraints (L1). We validate the adequacy of L1 regularization for the investigation of phenomena that are well described by a sparse model, using data acquired during phantom experiments

Accelerated High-Resolution Photoacoustic Tomography via Compressed Sensing

Current 3D photoacoustic tomography (PAT) systems offer either high image quality or high frame rates but are not able to deliver high spatial and temporal resolution simultaneously, which limits their ability to image dynamic processes in living tissue. A particular example is the planar Fabry-Perot (FP) scanner, which yields high-resolution images but takes several minutes to sequentially map the photoacoustic field on the sensor plane, point-by-point. However, as the spatio-temporal complexity of many absorbing tissue structures is rather low, the data recorded in such a conventional, regularly sampled fashion is often highly redundant. We demonstrate that combining variational image reconstruction methods using spatial sparsity constraints with the development of novel PAT acquisition systems capable of sub-sampling the acoustic wave field can dramatically increase the acquisition speed while maintaining a good spatial resolution: First, we describe and model two general spatial sub-sampling schemes. Then, we discuss how to implement them using the FP scanner and demonstrate the potential of these novel compressed sensing PAT devices through simulated data from a realistic numerical phantom and through measured data from a dynamic experimental phantom as well as from in-vivo experiments. Our results show that images with good spatial resolution and contrast can be obtained from highly sub-sampled PAT data if variational image reconstruction methods that describe the tissues structures with suitable sparsity-constraints are used. In particular, we examine the use of total variation regularization enhanced by Bregman iterations. These novel reconstruction strategies offer new opportunities to dramatically increase the acquisition speed of PAT scanners that employ point-by-point sequential scanning as well as reducing the channel count of parallelized schemes that use detector arrays

4D imaging in tomography and optical nanoscopy

Diese Dissertation gehört zu den Gebieten mathematische Bildverarbeitung und inverse Probleme. Ein inverses Problem ist die Aufgabe, Modellparameter anhand von gemessenen Daten zu berechnen. Solche Probleme treten in zahlreichen Anwendungen in Wissenschaft und Technik auf, z.B. in medizinischer Bildgebung, Biophysik oder Astronomie. Wir betrachten Rekonstruktionsprobleme mit Poisson Rauschen in der Tomographie und optischen Nanoskopie. Bei letzterer gilt es Bilder ausgehend von verzerrten und verrauschten Messungen zu rekonstruieren, wohingegen in der Positronen-Emissions-Tomographie die Aufgabe in der Visualisierung physiologischer Prozesse eines Patienten besteht. Standardmethoden zur 3D Bildrekonstruktion berücksichtigen keine zeitabhängigen Informationen oder Dynamik, z.B. Herzschlag oder Atmung in der Tomographie oder Zellmigration in der Mikroskopie. Diese Dissertation behandelt Modelle, Analyse und effiziente Algorithmen für 3D und 4D zeitabhängige inverse Probleme. This thesis contributes to the field of mathematical image processing and inverse problems. An inverse problem is a task, where the values of some model parameters must be computed from observed data. Such problems arise in a wide variety of applications in sciences and engineering, such as medical imaging, biophysics or astronomy. We mainly consider reconstruction problems with Poisson noise in tomography and optical nanoscopy. In the latter case, the task is to reconstruct images from blurred and noisy measurements, whereas in positron emission tomography the task is to visualize physiological processes of a patient. In 3D static image reconstruction standard methods do not incorporate time-dependent information or dynamics, e.g. heart beat or breathing in tomography or cell motion in microscopy. This thesis is a treatise on models, analysis and efficient algorithms to solve 3D and 4D time-dependent inverse problems

Large Scale Inverse Problems

This book is thesecond volume of a three volume series recording the "Radon Special Semester 2011 on Multiscale Simulation &amp Analysis in Energy and the Environment" that took placein Linz, Austria, October 3-7, 2011. This volume addresses the common ground in the mathematical and computational procedures required for large-scale inverse problems and data assimilation in forefront applications. The solution of inverse problems is fundamental to a wide variety of applications such as weather forecasting, medical tomography, and oil exploration. Regularisation techniques are needed to ensure solutions of sufficient quality to be useful, and soundly theoretically based. This book addresses the common techniques required for all the applications, and is thus truly interdisciplinary. This collection of survey articles focusses on the large inverse problems commonly arising in simulation and forecasting in the earth sciences