Image processing for correlative light and electron microscopy

People have never stopped exploring the microscopic world. Studying the microstructure of cells helps people better understand the people themselves and has the potential to overcome specific diseases at a fundamental level. Correlative light and electron microscopy (CLEM) can let people intuitively understand the sample information through imaging. In CLEM measurements, samples are measured in both fluorescence microscopy and electron microscopy. Due to technical differences between LM and EM, images obtained from LM and EM contain different information. With the fluorescent labels, one can easily observe the structures of interest. However, due to the diffraction limit, LM image resolution is limited to a few hundred nanometers. EM images can capture the detailed structure of a sample down to the atomic level. However, grayscale images obtained from EM often contain very complex structures. Identifying structures of interest from these complex structures using only EM images is usually a challenge. The CLEM technology provides an opportunity to specify the structures of interest by comparing the CLEM images. However, due to the resolution difference between LM and EM, these structures are usually still not directly distinguishable by simply overlaying the fluorescence microscopy images on high-resolution grayscale electron microscopy images. This thesis aims to investigate a new deconvolution algorithm, EM-guided deconvolution, to automate fusing the LM information on the correlative EM image. We discuss the algorithm with simulated CLEM images and further apply it to experimental data sets. The algorithm can enhance the image resolution to nanometers from correlative wide-field (or confocal) fluorescence microscopy images. The algorithm can effectively recognise, e.g., membrane structures or identify the structures with a suitable point spread function and precise image registration

Improving Range Estimation of a 3D FLASH LADAR via Blind Deconvolution

The purpose of this research effort is to improve and characterize range estimation in a three-dimensional FLASH LAser Detection And Ranging (3D FLASH LADAR) by investigating spatial dimension blurring effects. The myriad of emerging applications for 3D FLASH LADAR both as primary and supplemental sensor necessitate superior performance including accurate range estimates. Along with range information, this sensor also provides an imaging or laser vision capability. Consequently, accurate range estimates would also greatly aid in image quality of a target or remote scene under interrogation. Unlike previous efforts, this research accounts for pixel coupling by defining the range image mathematical model as a convolution between the system spatial impulse response and the object (target or remote scene) at a particular range slice. Using this model, improved range estimation is possible by object restoration from the data observations. Object estimation is principally performed by deriving a blind deconvolution Generalized Expectation Maximization (GEM) algorithm with the range determined from the estimated object by a normalized correlation method. Theoretical derivations and simulation results are verified with experimental data of a bar target taken from a 3D FLASH LADAR system in a laboratory environment. Additionally, among other factors, range separation estimation variance is a function of two LADAR design parameters (range sampling interval and transmitted pulse-width), which can be optimized using the expected range resolution between two point sources. Using both CRB theory and an unbiased estimator, an investigation is accomplished that finds the optimal pulse-width for several range sampling scenarios using a range resolution metric

Advances in image processing for single-particle analysis by electron cryomicroscopy and challenges ahead

Electron cryomicroscopy (cryo-EM) is essential for the study and functional understanding of non-crystalline macromolecules such as proteins. These molecules cannot be imaged using X-ray crystallography or other popular methods. CryoEM has been successfully used to visualize molecules such as ribosomes, viruses, and ion channels, for example. Obtaining structural models of these at various conformational states leads to insight on how these molecules function. Recent advances in imaging technology have given cryo-EM a scientific rebirth. Because of imaging improvements, image processing and analysis of the resultant images have increased the resolution such that molecular structures can be resolved at the atomic level. Cryo-EM is ripe with stimulating image processing challenges. In this article, we will touch on the most essential in order to build an accurate structural three-dimensional model from noisy projection images. Traditional approaches, such as k-means clustering for class averaging, will be provided as background. With this review, however, we will highlight fresh approaches from new and varied angles for each image processing sub-problem, including a 3D reconstruction method for asymmetric molecules using just two projection images and deep learning algorithms for automated particle picking. Keywords: Cryo-electron microscopy, Single Particle Analysis, Image processing algorithms

Unbiased risk estimate algorithms for image deconvolution.

本論文工作的主題是圖像反卷積問題。在很多實際應用，例如生物醫學成像，地震學，天文學，遙感和光學成像中，觀測數據經常會出現令人不愉快的退化現象，這種退化一般由模糊效應(例如光學衍射限條件)和噪聲汙染(比如光子計數噪聲和讀出噪聲)造成的，這兩者都是物理儀器自身的條件限制造成的。作為一個標准的線性反問題，圖像反卷積經常被用作恢複觀測到的模糊的有噪點的圖像。我們旨在基于無偏差風險估計准則研究新的反卷積算法。本論文工作主要分為以下兩大部分。首先，我們考慮在加性高斯白噪聲條件下的圖像非盲反卷積問題，即准確的點擴散函數已知。我們的研究准則是最小化均方誤差的無偏差估計，即SURE. SURE- LET方法最初被應用于圖像降噪問題。本論文工作擴展該方法至討論圖像反卷積問題.我們提出了一個新的SURE-LET算法，用于快速有效地實現圖像複原功能。具體而言，我們將反卷積過程參數化表示為有限個基本函數的線性組合，稱作LET方法。反卷積問題最終簡化為求解該線性組合的最優線性系數。由于SURE的二次項本質和線性參數化表示，求解線性系數可由求解線性方程組而得。實驗結果顯示該論文提出的方法在信噪比，圖像的視覺質量和運算時間等方面均優于其他迄今最優秀的算法。論文的第二部分討論圖像盲複原中的點擴散函數估計問題。我們提出了blur-SURE -一個均方誤差修正版的無偏差估計 - 作為點擴散函數估計的最新准則，即點擴散函數由最小化這個新的目標函數獲得。然後我們利用這個估計的點擴散函數，用第一部分所提出的SURE-LET算法進行圖像的非盲複原。我們以一些典型的點擴散函數形式(高斯函數最為典型)為例詳細闡述該blur-SURE理論框架。實驗結果顯示最小化blur-SURE能夠更准確的估計點擴散函數，從而獲得更加優越的反卷積佳能。相比于圖像非盲複原，盲複原所得的圖片的視覺質量損失可忽略不計。本論文所提出的基于無偏差估計的算法可擴展至其他噪聲模型。由于本論文以SURE基礎的方法在理論上並不僅限于卷積問題，該方法可用于解決數據的其他線性失真問題。The subject of this thesis is image deconvolution. In many real applications, e.g. biomedical imaging, seismology, astronomy, remote sensing and optical imaging, undesirable degradations by blurring effect (e.g. optical diffraction-limited condition) and noise corruption (e.g. photon-counting noise and readout noise) are inherent to any physical acquisition device. Image deconvolution, as a standard linear inverse problem, is often applied to recover the images from their blurred and noisy observations. Our interest lies in novel deconvolution algorithms based on unbiased risk estimate. This thesis is organized in two main parts as briefly summarized below.We first consider non-blind image deconvolution with the corruption of additive white Gaussian noise (AWGN), where the point spread function (PSF) is exactly known. Our driving principle is the minimization of an unbiased estimate of mean squared error (MSE) between observed and clean data, known as "Stein's unbiased risk estimate" (SURE). The SURE-LET approach, which was originally developed for denoising, is extended to the deconvolution problem: a new SURE-LET deconvolution algorithm for fast and efficient implementation is proposed. More specifically, we parametrize the deconvolution process as a linear combination of a small number of known basic processings, which we call the linear expansion of thresholds (LET), and then minimize the SURE over the unknown linear coefficients. Due to the quadratic nature of SURE and the linear parametrization, the optimal linear weights of the combination is finally achieved by solving a linear system of equations. Experiments show that the proposed approach outperforms other state-of-the-art methods in terms of PSNR, SSIM, visual quality, as well as computation time.The second part of this thesis is concerned with PSF estimation for blind deconvolution. We propose a "blur-SURE" - an unbiased estimate of a filtered version of MSE - as a novel criterion for estimating the PSF, from the observed image only, i.e. the PSF is identified by minimizing this new objective functional, whose validity has been theoretically verified. The blur-SURE framework is exemplified with a number of parametric forms of the PSF, most typically, the Gaussian kernel. Experiments show that the blur-SURE minimization yields highly accurate estimate of PSF parameters. We then perform non-blind deconvolution using the SURE-LET algorithm proposed in Part I, with the estimated PSF. Experiments show that the estimated PSF results in superior deconvolution performance, with a negligible quality loss, compared to the deconvolution with the exact PSF.One may extend the algorithms based on unbiased risk estimate to other noise model. Since the SURE-based approaches does not restrict themselves to convolution operation, it is possible to extend them to other distortion scenarios.Detailed summary in vernacular field only.Detailed summary in vernacular field only.Detailed summary in vernacular field only.Detailed summary in vernacular field only.Xue, Feng.Thesis (Ph.D.)--Chinese University of Hong Kong, 2013.Includes bibliographical references (leaves 119-130).Abstracts also in Chinese.Dedication --- p.iAcknowledgments --- p.iiiAbstract --- p.ixList of Notations --- p.xiContents --- p.xviList of Figures --- p.xxList of Tables --- p.xxiiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Motivations and objectives --- p.1Chapter 1.2 --- Mathematical formulation for problem statement --- p.2Chapter 1.3 --- Survey of non-blind deconvolution approaches --- p.2Chapter 1.3.1 --- Regularization --- p.2Chapter 1.3.2 --- Regularized inversion followed by denoising --- p.4Chapter 1.3.3 --- Bayesian approach --- p.4Chapter 1.3.4 --- Remark --- p.5Chapter 1.4 --- Survey of blind deconvolution approaches --- p.5Chapter 1.4.1 --- Non-parametric blind deconvolution --- p.5Chapter 1.4.2 --- Parametric blind deconvolution --- p.7Chapter 1.5 --- Objective assessment of the deconvolution quality --- p.8Chapter 1.5.1 --- Peak Signal-to-Noise Ratio (PSNR) --- p.8Chapter 1.5.2 --- Structural Similarity Index (SSIM) --- p.8Chapter 1.6 --- Thesis contributions --- p.9Chapter 1.6.1 --- Theoretical contributions --- p.9Chapter 1.6.2 --- Algorithmic contributions --- p.10Chapter 1.7 --- Organization --- p.11Chapter I --- The SURE-LET Approach to Non-blind Deconvolution --- p.13Chapter 2 --- The SURE-LET Framework for Deconvolution --- p.15Chapter 2.1 --- Motivations --- p.15Chapter 2.2 --- Related work --- p.15Chapter 2.3 --- Problem statement --- p.17Chapter 2.4 --- Stein's Unbiased Risk Estimate (SURE) for deconvolution --- p.17Chapter 2.4.1 --- Original SURE --- p.17Chapter 2.4.2 --- Regularized approximation of SURE --- p.18Chapter 2.5 --- The SURE-LET approach --- p.19Chapter 2.6 --- Summary --- p.20Chapter 3 --- Multi-Wiener SURE-LET Approach --- p.23Chapter 3.1 --- Problem statement --- p.23Chapter 3.2 --- Linear deconvolution: multi-Wiener filtering --- p.23Chapter 3.3 --- SURE-LET in orthonormal wavelet representation --- p.24Chapter 3.3.1 --- Mathematical formulation --- p.24Chapter 3.3.2 --- SURE minimization in orthonormal wavelet domain --- p.26Chapter 3.3.3 --- Computational issues --- p.27Chapter 3.4 --- SURE-LET approach for redundant wavelet representation --- p.30Chapter 3.5 --- Computational aspects --- p.32Chapter 3.5.1 --- Periodic boundary extensions --- p.33Chapter 3.5.2 --- Symmetric convolution --- p.36Chapter 3.5.3 --- Half-point symmetric boundary extensions --- p.36Chapter 3.5.4 --- Whole-point symmetric boundary extensions --- p.43Chapter 3.6 --- Results and discussions --- p.46Chapter 3.6.1 --- Experimental setting --- p.46Chapter 3.6.2 --- Influence of the number of Wiener lters --- p.47Chapter 3.6.3 --- Influence of the parameters on the deconvolution performance --- p.48Chapter 3.6.4 --- Influence of the boundary conditions: periodic vs symmetric --- p.52Chapter 3.6.5 --- Comparison with the state-of-the-art --- p.52Chapter 3.6.6 --- Analysis of computational complexity --- p.59Chapter 3.7 --- Conclusion --- p.60Chapter II --- The SURE-based Approach to Blind Deconvolution --- p.63Chapter 4 --- The Blur-SURE Framework to PSF Estimation --- p.65Chapter 4.1 --- Introduction --- p.65Chapter 4.2 --- Problem statement --- p.66Chapter 4.3 --- The blur-SURE framework for general linear model --- p.66Chapter 4.3.1 --- Blur-MSE: a modified version of MSE --- p.66Chapter 4.3.2 --- Blur-MSE minimization --- p.67Chapter 4.3.3 --- Blur-SURE: an unbiased estimate of the blur-MSE --- p.67Chapter 4.4 --- Application of blur-SURE framework for PSF estimation --- p.68Chapter 4.4.1 --- Problem statement in the context of convolution --- p.68Chapter 4.4.2 --- Blur-MSE minimization for PSF estimation --- p.69Chapter 4.4.3 --- Approximation of exact Wiener filtering --- p.70Chapter 4.4.4 --- Blur-SURE minimization for PSF estimation --- p.72Chapter 4.5 --- Concluding remarks --- p.72Chapter 5 --- The Blur-SURE Approach to Parametric PSF Estimation --- p.75Chapter 5.1 --- Introduction --- p.75Chapter 5.1.1 --- Overview of parametric PSF estimation --- p.75Chapter 5.1.2 --- Gaussian PSF as a typical example --- p.75Chapter 5.1.3 --- Outline of this chapter --- p.76Chapter 5.2 --- Parametric estimation: problem formulation --- p.77Chapter 5.3 --- Examples of PSF parameter estimation --- p.77Chapter 5.3.1 --- Gaussian kernel --- p.77Chapter 5.3.2 --- Non-Gaussian PSF with scaling factor s --- p.78Chapter 5.4 --- Minimization via the approximated function λ = λ (s) --- p.79Chapter 5.5 --- Results and discussions --- p.82Chapter 5.5.1 --- Experimental setting --- p.82Chapter 5.5.2 --- Non-Gaussian functions: estimation of scaling factor s --- p.83Chapter 5.5.3 --- Gaussian function: estimation of standard deviation s --- p.84Chapter 5.5.4 --- Comparison of deconvolution performance with the state-of-the-art --- p.84Chapter 5.5.5 --- Application to real images --- p.87Chapter 5.6 --- Conclusion --- p.90Chapter 6 --- The Blur-SURE Approach to Motion Deblurring --- p.93Chapter 6.1 --- Introduction --- p.93Chapter 6.1.1 --- Background of motion deblurring --- p.93Chapter 6.1.2 --- Related work: parametric estimation of motion blur --- p.93Chapter 6.1.3 --- Outline of this chapter --- p.94Chapter 6.2 --- Parametric estimation of motion blur: problem formulation --- p.94Chapter 6.2.1 --- Parametrized form of linear motion blur --- p.94Chapter 6.2.2 --- The blur-SURE framework to motion blur estimation --- p.94Chapter 6.3 --- An example of the blur-SURE approach to motion blur estimation --- p.95Chapter 6.4 --- Implementation issues --- p.96Chapter 6.4.1 --- Estimation of motion direction --- p.97Chapter 6.4.2 --- Estimation of blur length --- p.97Chapter 6.4.3 --- Short summary --- p.98Chapter 6.5 --- Results and discussions --- p.98Chapter 6.5.1 --- Experimental setting --- p.98Chapter 6.5.2 --- Estimations of blur direction and length --- p.99Chapter 6.5.3 --- Motion deblurring: the synthetic experiments --- p.99Chapter 6.5.4 --- Motion deblurring: the real experiment --- p.101Chapter 6.6 --- Conclusion --- p.103Chapter 7 --- Epilogue --- p.107Chapter 7.1 --- Summary --- p.107Chapter 7.2 --- Perspectives --- p.108Chapter A --- Proof --- p.109Chapter A.1 --- Proof of Theorem 2.1 --- p.109Chapter A.2 --- Proof of Eq.(2.6) in Section 2.4.2 --- p.110Chapter A.3 --- Proof of Eq.(3.5) in Section 3.3.1 --- p.110Chapter A.4 --- Proof of Theorem 3.6 --- p.112Chapter A.5 --- Proof of Theorem 3.12 --- p.112Chapter A.6 --- Derivation of noise variance in 2-D case (Section 3.5.4) --- p.114Chapter A.7 --- Proof of Theorem 4.1 --- p.116Chapter A.8 --- Proof of Theorem 4.2 --- p.11

The use of scRNA-seq to characterise the tumour microenvironment of high grade serous ovarian carincoma (HGSOC)

High Grade Serous Ovarian Carcinoma (HGSOC) is the most common type of ovarian cancer. Patients with this disease typically experience relapse in their disease following surgical debulking and initially effective chemotherapy. HGSOC has been intensely studied at the genomic and transcriptomic levels in efforts to advance knowledge of the biological mechanisms that drive the behaviour of this malignancy, and so that new treatment strategies may curb the disease progression relapse. This body of work contributes an optimised protocol for generating robust 10X scRNA-seq libraries from fresh and preserved HGSOC tissue, aiming to dissect the cellular heterogeneity of HGSOC’s Tumour microenvironment (TME). Through unsupervised clustering analysis, it uncovers distinct cellular communities, elucidates transcriptomic signatures across HGSOC tumours, and augments bulk RNA-seq datasets via computational deconvolution, enhancing understanding of HGSOC's cellular complexity across an expanded clinical cohort. The sequencing and analysis of these HGSOC patient tumours revealed 11 distinct cell types, including 2 that are novel in this tumour type; namely ciliated epithelial cells and metallothionein expressing T-cells. These 11 distinct cell types can be broadly categorised into 3 TME components (Tumour, Stroma and Immune) as in other previous tumour scRNA-seq studies. An additional analysis of these components examined the copy number variation (CNV) in the profiled cells and revealed HGSOC tumour cells to be mostly aneuploid while ciliated epithelial cells were diploid. A novel integrative subcluster analysis of HGSOC aneuploid tumour cells identified several apparently tumourigenic gene expression signatures. These include a KRT17+, protease inhibitory signature, an increased cellular metabolism signature, and an immune-reactive signature. Additionally, a ciliated cluster re-emerged within the HGSOC tumour cells, even though the diploid ciliated epithelial cells were not included in the integrative analysis. Finally, the high granularity of HGSOC cellular composition revealed by scRNA-seq is utilised to perform deconvolution analyses to estimate cellular proportions and infer the TME of earlier bulk RNA-seq profiled HGSOC tumour samples. This investigation of earlier sequenced HGSOC samples revealed heterogeneity in the proportions of the TME compartments across the patient cohorts. Survival analysis using these inferred cellular proportions suggest that immune cell presence alone is not associated with survival, but metastatic fibroblast burden in tumour samples is significantly associated with worsen overall survival in HGSOC patients. In conclusion, the laboratory protocol, the scRNA-seq datasets produced, and their analysis and application presented in this work expands the collective knowledge base of HGSOC. Specifically by characterising the cells of the HGSOC tumour microenvironment, and nuances of expression signatures of the malignant cells. The deconvolution approach showcases how scRNA-seq data can expand the clinical utility of earlier RNA-seq HGSOC datasets in a way that is scalable

Tumour-retained activated CCR7⁺ dendritic cells are heterogeneous and regulate local anti-tumour cytolytic activity

Tumour dendritic cells (DCs) internalise antigen and upregulate CCR7, which directs their migration to tumour-draining lymph nodes (dLN). CCR7 expression is coupled to an activation programme enriched in regulatory molecule expression, including PD-L1. However, the spatio-temporal dynamics of CCR7+ DCs in anti-tumour immune responses remain unclear. Here, we use photoconvertible mice to precisely track DC migration. We report that CCR7+ DCs are the dominant DC population that migrate to the dLN, but a subset remains tumour-resident despite CCR7 expression. These tumour-retained CCR7+ DCs are phenotypically and transcriptionally distinct from their dLN counterparts and heterogeneous. Moreover, they progressively downregulate the expression of antigen presentation and pro-inflammatory transcripts with more prolonged tumour dwell-time. Tumour-residing CCR7+ DCs co-localise with PD-1+CD8+ T cells in human and murine solid tumours, and following anti-PD-L1 treatment, upregulate stimulatory molecules including OX40L, thereby augmenting anti-tumour cytolytic activity. Altogether, these data uncover previously unappreciated heterogeneity in CCR7+ DCs that may underpin a variable capacity to support intratumoural cytotoxic T cells.</p

Methods for analyzing the influence of molecular dynamics on neuronal activity

Magdeburg, Univ., Fak. für Informatik, Diss., 2015von Stefan Sokol

Air Force Institute of Technology Research Report 2010

This report summarizes the research activities of the Air Force Institute of Technology’s Graduate School of Engineering and Management. It describes research interests and faculty expertise; lists student theses/dissertations; identifies research sponsors and contributions; and outlines the procedures for contacting the school. Included in the report are: faculty publications, conference presentations, consultations, and funded research projects. Research was conducted in the areas of Aeronautical and Astronautical Engineering, Electrical Engineering and Electro-Optics, Computer Engineering and Computer Science, Systems and Engineering Management, Operational Sciences, Mathematics, Statistics and Engineering Physic

Resolving the fibrotic niche of human liver cirrhosis at single-cell level.

Liver cirrhosis is a major cause of death worldwide and is characterized by extensive fibrosis. There are currently no effective antifibrotic therapies available. To obtain a better understanding of the cellular and molecular mechanisms involved in disease pathogenesis and enable the discovery of therapeutic targets, here we profile the transcriptomes of more than 100,000 single human cells, yielding molecular definitions for non-parenchymal cell types that are found in healthy and cirrhotic human liver. We identify a scar-associated TREM2+CD9+ subpopulation of macrophages, which expands in liver fibrosis, differentiates from circulating monocytes and is pro-fibrogenic. We also define ACKR1+ and PLVAP+ endothelial cells that expand in cirrhosis, are topographically restricted to the fibrotic niche and enhance the transmigration of leucocytes. Multi-lineage modelling of ligand and receptor interactions between the scar-associated macrophages, endothelial cells and PDGFRα+ collagen-producing mesenchymal cells reveals intra-scar activity of several pro-fibrogenic pathways including TNFRSF12A, PDGFR and NOTCH signalling. Our work dissects unanticipated aspects of the cellular and molecular basis of human organ fibrosis at a single-cell level, and provides a conceptual framework for the discovery of rational therapeutic targets in liver cirrhosis.Includes Wellcome, BHF, MRC, BBSRC and NIHR

Air Force Institute of Technology Research Report 2009

This report summarizes the research activities of the Air Force Institute of Technology’s Graduate School of Engineering and Management. It describes research interests and faculty expertise; lists student theses/dissertations; identifies research sponsors and contributions; and outlines the procedures for contacting the school. Included in the report are: faculty publications, conference presentations, consultations, and funded research projects. Research was conducted in the areas of Aeronautical and Astronautical Engineering, Electrical Engineering and Electro-Optics, Computer Engineering and Computer Science, Systems and Engineering Management, Operational Sciences, Mathematics, Statistics and Engineering Physics