Deep spectral learning for label-free optical imaging oximetry with uncertainty quantification

Measurement of blood oxygen saturation (sO2) by optical imaging oximetry provides invaluable insight into local tissue functions and metabolism. Despite different embodiments and modalities, all label-free optical-imaging oximetry techniques utilize the same principle of sO2-dependent spectral contrast from haemoglobin. Traditional approaches for quantifying sO2 often rely on analytical models that are fitted by the spectral measurements. These approaches in practice suffer from uncertainties due to biological variability, tissue geometry, light scattering, systemic spectral bias, and variations in the experimental conditions. Here, we propose a new data-driven approach, termed deep spectral learning (DSL), to achieve oximetry that is highly robust to experimental variations and, more importantly, able to provide uncertainty quantification for each sO2 prediction. To demonstrate the robustness and generalizability of DSL, we analyse data from two visible light optical coherence tomography (vis-OCT) setups across two separate in vivo experiments on rat retinas. Predictions made by DSL are highly adaptive to experimental variabilities as well as the depth-dependent backscattering spectra. Two neural-network-based models are tested and compared with the traditional least-squares fitting (LSF) method. The DSL-predicted sO2 shows significantly lower mean-square errors than those of the LSF. For the first time, we have demonstrated en face maps of retinal oximetry along with a pixel-wise confidence assessment. Our DSL overcomes several limitations of traditional approaches and provides a more flexible, robust, and reliable deep learning approach for in vivo non-invasive label-free optical oximetry.R01 CA224911 - NCI NIH HHS; R01 CA232015 - NCI NIH HHS; R01 NS108464 - NINDS NIH HHS; R21 EY029412 - NEI NIH HHSAccepted manuscrip

Illumination coding meets uncertainty learning: toward reliable AI-augmented phase imaging

We propose a physics-assisted deep learning (DL) framework for large space-bandwidth product (SBP) phase imaging. We design an asymmetric coded illumination scheme to encode high-resolution phase information across a wide field-of-view. We then develop a matching DL algorithm to provide large-SBP phase estimation. We show that this illumination coding scheme is highly scalable in achieving flexible resolution, and robust to experimental variations. We demonstrate this technique on both static and dynamic biological samples, and show that it can reliably achieve 5X resolution enhancement across 4X FOVs using only five multiplexed measurements -- more than 10X data reduction over the state-of-the-art. Typical DL algorithms tend to provide over-confident predictions, whose errors are only discovered in hindsight. We develop an uncertainty learning framework to overcome this limitation and provide predictive assessment to the reliability of the DL prediction. We show that the predicted uncertainty maps can be used as a surrogate to the true error. We validate the robustness of our technique by analyzing the model uncertainty. We quantify the effect of noise, model errors, incomplete training data, and "out-of-distribution" testing data by assessing the data uncertainty. We further demonstrate that the predicted credibility maps allow identifying spatially and temporally rare biological events. Our technique enables scalable AI-augmented large-SBP phase imaging with dependable predictions.Published versio

Scalable and reliable deep learning for computational microscopy in complex media

Emerging deep learning based computational microscopy techniques promise novel imaging capabilities beyond traditional techniques. In this talk, I will discuss two microscopy applications. First, high space-bandwidth product microscopy typically requires a large number of measurements. I will present a novel physics-assisted deep learning (DL) framework for large space-bandwidth product (SBP) phase imaging [1], enabling significant reduction of the required measurements, opening up real-time applications. In this technique, we design asymmetric coded illumination patterns to encode high-resolution phase information across a wide field-of-view. We then develop a matching DL algorithm to provide large-SBP phase estimation. We demonstrate this technique on both static and dynamic biological samples, and show that it can reliably achieve 5× resolution enhancement across 4× FOVs using only five multiplexed measurements. In addition, we develop an uncertainty learning framework to provide predictive assessment to the reliability of the DL prediction. We show that the predicted uncertainty maps can be used as a surrogate to the true error. We validate the robustness of our technique by analyzing the model uncertainty. We quantify the effect of noise, model errors, incomplete training data, and “out-of-distribution” testing data by assessing the data uncertainty. We further demonstrate that the predicted credibility maps allow identifying spatially and temporally rare biological events. Our technique enables scalable DL-augmented large-SBP phase imaging with reliable predictions and uncertainty quantifications. Second, I will turn to the pervasive problem of imaging in scattering media. I will discuss a new deep learning- based technique that is highly generalizable and resilient to statistical variations of the scattering media [2]. We develop a statistical ‘one-to-all’ deep learning technique that encapsulates a wide range of statistical variations for the model to be resilient to speckle decorrelations. Specifically, we develop a convolutional neural network (CNN) that is able to learn the statistical information contained in the speckle intensity patterns captured on a set of diffusers having the same macroscopic parameter. We then show that the trained CNN is able to generalize and make high-quality object predictions through an entirely different set of diffusers of the same class. Our work paves the way to a highly scalable deep learning approach for imaging through scattering media. REFERENCES [1] Xue, Y., Cheng, S., Li, Y., and Tian, L., “Illumination coding meets uncertainty learning: toward reliable ai-augmented phase imaging,” arXiv:1901.02038 (2019). [2] Li, Y., Xue, Y., and Tian, L., “Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media,” Optica 5, 1181 (2018)

Optimized Dimensionality Reduction for Moment-based Distributionally Robust Optimization

Moment-based distributionally robust optimization (DRO) provides an optimization framework to integrate statistical information with traditional optimization approaches. Under this framework, one assumes that the underlying joint distribution of random parameters runs in a distributional ambiguity set constructed by moment information and makes decisions against the worst-case distribution within the set. Although most moment-based DRO problems can be reformulated as semidefinite programming (SDP) problems that can be solved in polynomial time, solving high-dimensional SDPs is still time-consuming. Unlike existing approximation approaches that first reduce the dimensionality of random parameters and then solve the approximated SDPs, we propose an optimized dimensionality reduction (ODR) approach. We first show that the ranks of the matrices in the SDP reformulations are small, by which we are then motivated to integrate the dimensionality reduction of random parameters with the subsequent optimization problems. Such integration enables two outer and one inner approximations of the original problem, all of which are low-dimensional SDPs that can be solved efficiently. More importantly, these approximations can theoretically achieve the optimal value of the original high-dimensional SDPs. As these approximations are nonconvex SDPs, we develop modified Alternating Direction Method of Multipliers (ADMM) algorithms to solve them efficiently. We demonstrate the effectiveness of our proposed ODR approach and algorithm in solving two practical problems. Numerical results show significant advantages of our approach on the computational time and solution quality over the three best possible benchmark approaches. Our approach can obtain an optimal or near-optimal (mostly within 0.1%) solution and reduce the computational time by up to three orders of magnitude

Reliable deep-learning-based phase imaging with uncertainty quantification: supplementary material

Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedical imaging. However, one outstanding challenge is the lack of reliability assessment in the DL predictions, whose errors are commonly revealed only in hindsight. Here, we propose a new Bayesian convolutional neural network (BNN)-based framework that overcomes this issue by quantifying the uncertainty of DL predictions. Foremost, we show that BNN-predicted uncertainty maps provide surrogate estimates of the true error from the network model and measurement itself. The uncertainty maps characterize imperfections often unknown in real-world applications, such as noise, model error, incomplete training data, and out-of-distribution testing data. Quantifying this uncertainty provides a per-pixel estimate of the confidence level of the DL prediction as well as the quality of the model and data set. We demonstrate this framework in the application of large space–bandwidth product phase imaging using a physics-guided coded illumination scheme. From only five multiplexed illumination measurements, our BNN predicts gigapixel phase images in both static and dynamic biological samples with quantitative credibility assessment. Furthermore, we show that low-certainty regions can identify spatially and temporally rare biological phenomena. We believe our uncertainty learning framework is widely applicable to many DL-based biomedical imaging techniques for assessing the reliability of DL predictions.https://www.osapublishing.org/optica/fulltext.cfm?uri=optica-7-4-332&id=429860Published versio

Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network

Coherent imaging through scatter is a challenging task in computational imaging. Both model-based and data-driven approaches have been explored to solve the inverse scattering problem. In our previous work, we have shown that a deep learning approach can make high-quality and highly generalizable predictions through unseen diffusers. Here, we propose a new deep neural network (DNN) model that is agnostic to a broader class of perturbations including scatterer change, displacements, and system defocus up to 10X depth of field. In addition, we develop a new analysis framework for interpreting the mechanism of our DNN model and visualizing its generalizability based on an unsupervised dimension reduction technique. We show that our DNN can unmix the scattering-specific information and extract the object-specific information so as to achieve generalization under different scattering conditions. Our work paves the way to a highly robust and interpretable deep learning approach to imaging through scattering media.Accepted manuscrip

Different immunological mechanisms between AQP4 antibody-positive and MOG antibody-positive optic neuritis based on RNA sequencing analysis of whole blood

PurposeTo compare the different immunological mechanisms between aquaporin 4 antibody-associated optic neuritis (AQP4-ON) and myelin oligodendrocyte glycoprotein antibody-associated optic neuritis (MOG-ON) based on RNA sequencing (RNA-seq) of whole blood.MethodsWhole blood was collected from seven healthy volunteers, 6 patients with AQP4-ON and 8 patients with MOG-ON, and used for RNA-seq analysis. An examination of immune cell infiltration was performed using the CIBERSORTx algorithm to identify infiltrated immune cells.ResultsRNA-seq analysis showed that the inflammatory signaling was mainly activated by TLR2, TLR5, TLR8 and TLR10 in AQP4-ON patients, while which was mainly activated by TLR1, TLR2, TLR4, TLR5 and TLR8 in MOG-ON patients. Biological function identification of differentially expressed genes (DEGs) based on Gene Ontology (GO) term and Kyoto encyclopedia of genes and genomes (KEGG) pathway enrichment analysis, as well as Disease Ontology (DO) analysis, showed that the inflammation in AQP4-ON was likely mediated by damage-associated molecular pattern (DAMP), while which in MOG-ON was likely mediated by pathogen-associated molecular pattern (PAMP). Analysis of immune cell infiltration showed that the proportion of immune cell infiltration was related to patients’ vision. The infiltration ratios of monocytes (rs=0.69, P=0.006) and M0 macrophages (rs=0.66, P=0.01) were positively correlated with the BCVA (LogMAR), and the infiltration ratio of neutrophils was negatively correlated with the BCVA (LogMAR) (rs=0.65, P=0.01).ConclusionThis study reveals different immunological mechanisms between AQP4-ON and MOG-ON based on transcriptomics analysis of patients’ whole blood, which may expand the current knowledge regarding optic neuritis