Inverse and forward modeling tools for biophotonic data

Biophotonic data require specific treatments due to the difficulty of directly extracting information from them. Therefore, artificial intelligence tools including machine learning and deep learning brought into play. These tools can be grouped into inverse modeling, preprocessing and data modeling categories. In each of these three categories, one research question was investigated. First, the aim was to develop a method that can acquire the Raman-like spectra from coherent anti-Stokes Raman scattering (CARS) spectra without apriori knowledge. In general, CARS spectra suffer from the non-resonant background (NRB) contribution, and existing methods were commonly implemented to remove it. However, these methods were not able to completely remove the NRB and need additional preprocessing afterward. Therefore, deep learning via the long-short-term memory network was applied and outperformed these existing methods. Then, a denoising technique via deep learning was developed for reconstructing high-quality (HQ) multimodal images (MM) from low-quality (LQ) ones. Since the measurement of HQ MM images is time-consuming, which is impractical for clinical applications, we developed a network, namely incSRCNN, to directly predict HQ images using only LQ ones. This network shows better performance when compared with standard methods. Finally, we intended to improve the accuracy of the classification model in particular when LQ Raman data or Raman data with varying quality are obtained. Therefore, a novel method based on functional data analysis was implemented, which converts the Raman data into functions and then applies functional dimension reduction followed by a classification method. The results showed better performance for the functional approach in comparison with the classical method

Adaptive Compressive Sampling for Mid-infrared Spectroscopic Imaging

Fourier transform infrared (FTIR) spectroscopy enables label-free molecular identification and quantification of biological specimens. The resolution of diffraction limited FTIR imaging is poor due to the long optical wavelengths (2.5{\mu}m to 12.5{\mu}m)used and this is particularly limiting in biomedical imaging. Photothermal imaging overcomes this diffraction limit by using a multimodal pump/probe approach. However, these measurements require approximately 1 s per spectrum, making them impractical for large samples. This paper introduces an adaptive compressive sampling technique to dramatically reduce hyperspectral data acquisition time by utilizing both spectral and spatial sparsity. This method identifies the most informative spatial and spectral features and integrates a fast tensor completion algorithm to reconstruct megapixel-scale images and demonstrates speed advantages over FTIR imagin

Integrated Imaging and Spectroscopic Analysis of Painted Fresco Surfaces Using Terahertz Time-Domain Technique

Terahertz time-domain (THz-TD) imaging plays an increasingly significant role in the study of solid-state materials by enabling the simultaneous extraction of spectroscopic composition and surface topography in the far-infrared region (3–300 cm^-1). However, when applied to works of art in reflection configuration, significant challenges arise, including weak signal intensity, multiple signal losses, and surface distortion. This study proposes a practical solution to overcome these limitations and conducts an integrated imaging and spectroscopic analysis on painted fresco surfaces, allowing for the retrieval of surface thicknesses, material distribution, and pigment spectroscopic signals. The study addresses the issue of surface geometrical distortion, which hampers the accurate determination of the THz phase signal. By tackling this challenge, this work successfully determines the absorption coefficient for each point on the surface and retrieves spectroscopic signatures. Additionally, the temporal deconvolution technique is employed to separate different layers of the sample and differentiate between outer and inner surface topography. The objective of this study is to demonstrate the advantages and limitations of THz-TD imaging in determining surface thicknesses, material distribution, and pigment spectroscopic signals. The results obtained highlight the potential of THz-TD imaging in investigating painted works of art, offering new possibilities for routine analysis in the field of cultural heritage preservation

On the Adjoint Operator in Photoacoustic Tomography

Photoacoustic Tomography (PAT) is an emerging biomedical "imaging from coupled physics" technique, in which the image contrast is due to optical absorption, but the information is carried to the surface of the tissue as ultrasound pulses. Many algorithms and formulae for PAT image reconstruction have been proposed for the case when a complete data set is available. In many practical imaging scenarios, however, it is not possible to obtain the full data, or the data may be sub-sampled for faster data acquisition. In such cases, image reconstruction algorithms that can incorporate prior knowledge to ameliorate the loss of data are required. Hence, recently there has been an increased interest in using variational image reconstruction. A crucial ingredient for the application of these techniques is the adjoint of the PAT forward operator, which is described in this article from physical, theoretical and numerical perspectives. First, a simple mathematical derivation of the adjoint of the PAT forward operator in the continuous framework is presented. Then, an efficient numerical implementation of the adjoint using a k-space time domain wave propagation model is described and illustrated in the context of variational PAT image reconstruction, on both 2D and 3D examples including inhomogeneous sound speed. The principal advantage of this analytical adjoint over an algebraic adjoint (obtained by taking the direct adjoint of the particular numerical forward scheme used) is that it can be implemented using currently available fast wave propagation solvers.Comment: submitted to "Inverse Problems

Hyperspectral Image Unmixing Incorporating Adjacency Information

While the spectral information contained in hyperspectral images is rich, the spatial resolution of such images is in many cases very low. Many pixel spectra are mixtures of pure materials’ spectra and therefore need to be decomposed into their constituents. This work investigates new decomposition methods taking into account spectral, spatial and global 3D adjacency information. This allows for faster and more accurate decomposition results

Advanced Image Acquisition, Processing Techniques and Applications

"Advanced Image Acquisition, Processing Techniques and Applications" is the first book of a series that provides image processing principles and practical software implementation on a broad range of applications. The book integrates material from leading researchers on Applied Digital Image Acquisition and Processing. An important feature of the book is its emphasis on software tools and scientific computing in order to enhance results and arrive at problem solution

Deep learning a boon for biophotonics

This review covers original articles using deep learning in the biophotonic field published in the last years. In these years deep learning, which is a subset of machine learning mostly based on artificial neural network geometries, was applied to a number of biophotonic tasks and has achieved state-of-the-art performances. Therefore, deep learning in the biophotonic field is rapidly growing and it will be utilized in the next years to obtain real-time biophotonic decision-making systems and to analyze biophotonic data in general. In this contribution, we discuss the possibilities of deep learning in the biophotonic field including image classification, segmentation, registration, pseudostaining and resolution enhancement. Additionally, we discuss the potential use of deep learning for spectroscopic data including spectral data preprocessing and spectral classification. We conclude this review by addressing the potential applications and challenges of using deep learning for biophotonic data. © 2020 The Authors. Journal of Biophotonics published by WILEY-VCH Verlag GmbH & Co. KGaA, Weinhei

Variational and learning models for image and time series inverse problems

Inverse problems are at the core of many challenging applications. Variational and learning models provide estimated solutions of inverse problems as the outcome of specific reconstruction maps. In the variational approach, the result of the reconstruction map is the solution of a regularized minimization problem encoding information on the acquisition process and prior knowledge on the solution. In the learning approach, the reconstruction map is a parametric function whose parameters are identified by solving a minimization problem depending on a large set of data. In this thesis, we go beyond this apparent dichotomy between variational and learning models and we show they can be harmoniously merged in unified hybrid frameworks preserving their main advantages. We develop several highly efficient methods based on both these model-driven and data-driven strategies, for which we provide a detailed convergence analysis. The arising algorithms are applied to solve inverse problems involving images and time series. For each task, we show the proposed schemes improve the performances of many other existing methods in terms of both computational burden and quality of the solution. In the first part, we focus on gradient-based regularized variational models which are shown to be effective for segmentation purposes and thermal and medical image enhancement. We consider gradient sparsity-promoting regularized models for which we develop different strategies to estimate the regularization strength. Furthermore, we introduce a novel gradient-based Plug-and-Play convergent scheme considering a deep learning based denoiser trained on the gradient domain. In the second part, we address the tasks of natural image deblurring, image and video super resolution microscopy and positioning time series prediction, through deep learning based methods. We boost the performances of supervised, such as trained convolutional and recurrent networks, and unsupervised deep learning strategies, such as Deep Image Prior, by penalizing the losses with handcrafted regularization terms

Fusing Multiple Multiband Images

We consider the problem of fusing an arbitrary number of multiband, i.e., panchromatic, multispectral, or hyperspectral, images belonging to the same scene. We use the well-known forward observation and linear mixture models with Gaussian perturbations to formulate the maximum-likelihood estimator of the endmember abundance matrix of the fused image. We calculate the Fisher information matrix for this estimator and examine the conditions for the uniqueness of the estimator. We use a vector total-variation penalty term together with nonnegativity and sum-to-one constraints on the endmember abundances to regularize the derived maximum-likelihood estimation problem. The regularization facilitates exploiting the prior knowledge that natural images are mostly composed of piecewise smooth regions with limited abrupt changes, i.e., edges, as well as coping with potential ill-posedness of the fusion problem. We solve the resultant convex optimization problem using the alternating direction method of multipliers. We utilize the circular convolution theorem in conjunction with the fast Fourier transform to alleviate the computational complexity of the proposed algorithm. Experiments with multiband images constructed from real hyperspectral datasets reveal the superior performance of the proposed algorithm in comparison with the state-of-the-art algorithms, which need to be used in tandem to fuse more than two multiband images

Assessment and optimisation of 3D optical topography for brain imaging

Optical topography has recently evolved into a widespread research tool for non-invasively mapping blood flow and oxygenation changes in the adult and infant cortex. The work described in this thesis has focused on assessing the potential and limitations of this imaging technique, and developing means of obtaining images which are less artefactual and more quantitatively accurate. Due to the diffusive nature of biological tissue, the image reconstruction is an ill-posed problem, and typically under-determined, due to the limited number of optodes (sources and detectors). The problem must be regularised in order to provide meaningful solutions, and requires a regularisation parameter (\lambda), which has a large influence on the image quality. This work has focused on three-dimensional (3D) linear reconstruction using zero-order Tikhonov regularisation and analysis of different methods to select the regularisation parameter. The methods are summarised and applied to simulated data (deblurring problem) and experimental data obtained with the University College London (UCL) optical topography system. This thesis explores means of optimising the reconstruction algorithm to increase imaging performance by using spatially variant regularisation. The sensitivity and quantitative accuracy of the method is investigated using measurements on tissue-equivalent phantoms. Our optical topography system is based on continuous-wave (CW) measurements, and conventional image reconstruction methods cannot provide unique solutions, i.e., cannot separate tissue absorption and scattering simultaneously. Improved separation between absorption and scattering and between the contributions of different chromophores can be obtained by using multispectral image reconstruction. A method is proposed to select the optimal wavelength for optical topography based on the multispectral method that involves determining which wavelengths have overlapping sensitivities. Finally, we assess and validate the new three-dimensional imaging tools using in vivo measurements of evoked response in the infant brain