Image tag completion by local learning

The problem of tag completion is to learn the missing tags of an image. In this paper, we propose to learn a tag scoring vector for each image by local linear learning. A local linear function is used in the neighborhood of each image to predict the tag scoring vectors of its neighboring images. We construct a unified objective function for the learning of both tag scoring vectors and local linear function parame- ters. In the objective, we impose the learned tag scoring vectors to be consistent with the known associations to the tags of each image, and also minimize the prediction error of each local linear function, while reducing the complexity of each local function. The objective function is optimized by an alternate optimization strategy and gradient descent methods in an iterative algorithm. We compare the proposed algorithm against different state-of-the-art tag completion methods, and the results show its advantages

Magnetic resonance multitasking for motion-resolved quantitative cardiovascular imaging.

Quantitative cardiovascular magnetic resonance (CMR) imaging can be used to characterize fibrosis, oedema, ischaemia, inflammation and other disease conditions. However, the need to reduce artefacts arising from body motion through a combination of electrocardiography (ECG) control, respiration control, and contrast-weighting selection makes CMR exams lengthy. Here, we show that physiological motions and other dynamic processes can be conceptualized as multiple time dimensions that can be resolved via low-rank tensor imaging, allowing for motion-resolved quantitative imaging with up to four time dimensions. This continuous-acquisition approach, which we name cardiovascular MR multitasking, captures - rather than avoids - motion, relaxation and other dynamics to efficiently perform quantitative CMR without the use of ECG triggering or breath holds. We demonstrate that CMR multitasking allows for T1 mapping, T1-T2 mapping and time-resolved T1 mapping of myocardial perfusion without ECG information and/or in free-breathing conditions. CMR multitasking may provide a foundation for the development of setup-free CMR imaging for the quantitative evaluation of cardiovascular health

Optical imaging methods for the study of disease models from the nano to the mesoscale

The visualisation of disease phenotypes allows scientists to study fundamental mechanisms of disease. Optical imaging methods are useful not only to observe anatomical features of biological samples, but also to infer interactions between molecular species using fluorescence labelling. This thesis presents the development of imaging and analysis tools to study biological questions in three models of disease, with samples ranging from the sub-cellular to the organ scale. First, the role of the alpha-synuclein (a-syn) protein, whose dysfunction is a hallmark of Parkinson’s Disease, was studied with respect to vesicle trafficking at the synapse. Synaptic vesicles are ∼40 nm in diameter; imaging vesicles therefore requires methods with resolution below the diffraction limit. Single-molecule localisation microscopy (SMLM), which circumvents the diffraction limit by separating fluorophore emission in time to localise individual molecules in space with ∼20 nm precision, was thus implemented to study a-syn in purified synaptic boutons. A software package was developed to analyse the colocalisation of a-syn with internalised vesicles, and the clustering of a-syn under differing synaptic calcium levels. The colocalisation of a-syn and internalised vesicles was found to be temperature independent, suggesting that a-syn is involved in non-canonical trafficking mechanisms. Ground truth simulations from a synaptosome model were used to benchmark two cluster analysis methods. Both methods applied on the experimental data showed that a-syn becomes less clustered at low synaptic calcium levels. Second, the spatiotemporal association of ESCRT-II, a protein complex whose role in the budding of the human immunodeficiency virus (HIV) was previously considered dispensable, and the HIV polyprotein Gag was studied during viral egress using novel image analysis tools. A nearest-neighbour analysis showed the ESCRT-II protein EAP45 colocalises with Gag similarly to ALIX, a protein well known to be involved in HIV budding. However, upon deletion of EAP45’s N-terminus, its colocalisation with Gag was significantly impaired, highlighting the importance of this EAP45 domain in linking to Gag. Single particle tracking was used to trace the trajectories of EAP45 and Gag in live cells, and an algorithm was developed to visualise the simultaneous motion of two particles; these analyses revealed three types of potential dynamic interaction between EAP45 and Gag. Finally, an open-source instrument to visualise phenotypes from large organs in 3D was developed for the study of chronic obstructive pulmonary disease (COPD) models. The instrument implements Optical Projection Tomography, a technique which can reconstruct cross-sectional slices of a transparent object from its orthographic projections, using off-the- shelf components and novel ImageJ plugins for artefact correction and volume reconstructions. Excised and cleared mouse lungs were imaged in which high order airways can be discerned with 50 μm resolution. The raw lung data, instructions for building the instrument, the free ImageJ plugins, and a detailed software manual are available in an online repository to encourage the widespread use of OPT for imaging large samples.Gates Cambridg

3D exemplar-based image inpainting in electron microscopy

In electron microscopy (EM) a common problem is the non-availability of data, which causes artefacts in reconstructions. In this thesis the goal is to generate artificial data where missing in EM by using exemplar-based inpainting (EBI). We implement an accelerated 3D version tailored to applications in EM, which reduces reconstruction times from days to minutes. We develop intelligent sampling strategies to find optimal data as input for reconstruction methods. Further, we investigate approaches to reduce electron dose and acquisition time. Sparse sampling followed by inpainting is the most promising approach. As common evaluation measures may lead to misinterpretation of results in EM and falsify a subsequent analysis, we propose to use application driven metrics and demonstrate this in a segmentation task. A further application of our technique is the artificial generation of projections in tiltbased EM. EBI is used to generate missing projections, such that the full angular range is covered. Subsequent reconstructions are significantly enhanced in terms of resolution, which facilitates further analysis of samples. In conclusion, EBI proves promising when used as an additional data generation step to tackle the non-availability of data in EM, which is evaluated in selected applications. Enhancing adaptive sampling methods and refining EBI, especially considering the mutual influence, promotes higher throughput in EM using less electron dose while not lessening quality.Ein häufig vorkommendes Problem in der Elektronenmikroskopie (EM) ist die Nichtverfügbarkeit von Daten, was zu Artefakten in Rekonstruktionen führt. In dieser Arbeit ist es das Ziel fehlende Daten in der EM künstlich zu erzeugen, was durch Exemplar-basiertes Inpainting (EBI) realisiert wird. Wir implementieren eine auf EM zugeschnittene beschleunigte 3D Version, welche es ermöglicht, Rekonstruktionszeiten von Tagen auf Minuten zu reduzieren. Wir entwickeln intelligente Abtaststrategien, um optimale Datenpunkte für die Rekonstruktion zu erhalten. Ansätze zur Reduzierung von Elektronendosis und Aufnahmezeit werden untersucht. Unterabtastung gefolgt von Inpainting führt zu den besten Resultaten. Evaluationsmaße zur Beurteilung der Rekonstruktionsqualität helfen in der EM oft nicht und können zu falschen Schlüssen führen, weswegen anwendungsbasierte Metriken die bessere Wahl darstellen. Dies demonstrieren wir anhand eines Beispiels. Die künstliche Erzeugung von Projektionen in der neigungsbasierten Elektronentomographie ist eine weitere Anwendung. EBI wird verwendet um fehlende Projektionen zu generieren. Daraus resultierende Rekonstruktionen weisen eine deutlich erhöhte Auflösung auf. EBI ist ein vielversprechender Ansatz, um nicht verfügbare Daten in der EM zu generieren. Dies wird auf Basis verschiedener Anwendungen gezeigt und evaluiert. Adaptive Aufnahmestrategien und EBI können also zu einem höheren Durchsatz in der EM führen, ohne die Bildqualität merklich zu verschlechtern

Sparse algorithms for decoding and identification of neural circuits

The brain, as an information processing machine, surpasses any man-made computational device, both in terms of its capabilities and its efficiency. Neuroscience research has made great strides since the foundational works of Cajal and Golgi. However, we still have very little understanding about the algorithmic underpinnings of the brain as an information processor. Identifying mechanistic models of the functional building blocks of the brain will have significant impact not just on neuroscience, but also on artificial computational systems. This provides the main motivation for the work presented in this thesis, summarily i) biologically-inspired algorithms that can be efficiently implemented in silico, ii) functional identification of the processing in certain types of neural circuits, and iii) a collaborative ecosystem for brain research in a model organism, towards the synergistic goal of understanding functional mechanisms employed by the brain. First, this thesis provides a highly parallelizable, biologically-inspired, motion detection algorithm that is based upon the temporal processing of the local (spatial) phase of a visual stimulus. The relation of the phase based motion detector to the widely studied Reichardt detector model, is discussed. Examples are provided comparing the performance of the proposed algorithm with the Reichardt detector as well as the optic flow algorithm, which is the workhorse for motion detection in computer vision. Further, it is shown through examples that the phase based motion detection model provides intuitive explanations for reverse-phi based illusory motion percepts. Then, tractable algorithms are presented for decoding with and identification of neural circuits, comprised of processing that can be described by a second-order Volterra kernel (quadratic filter). It is shown that the Reichardt detector, as well as models of cortical complex cells, can be described by this structure. Examples are provided for decoding of visual stimuli encoded by a population of Reichardt detector cells and complex cells, as well as their identification from observed spike times. Further, the phase based motion detection model is shown to be equivalent to a second-order Volterra kernel acting on two normalized inputs. Subsequently, a general model that computes the ratio of two non-linear functionals, each comprising linear (first order Volterra kernel) and quadratic (second-order Volterra kernel) filters, is proposed. It is shown that, even under these highly non-linear operations, a population of cells can encode stimuli faithfully using a number of measurements that are proportional to the bandwidth of the input stimulus. Tractable algorithms are devised to identify the divisive normalization model and examples of identification are provided for both simulated and biological data. Additionally, an extended framework, comprising parallel channels of divisively normalized cells each subjected to further divisive normalization from lateral feedback connections, is proposed. An algorithm is formulated for identifying all the components in this extended framework from controlled stimulus presentation and observed outputs samples. Finally, the thesis puts forward the Fruit Fly Brain Observatory (FFBO), an initiative to enable a collaborative ecosystem for fruit fly brain research. Key applications in FFBO, and the software and computational infrastructure enabling them, are described along with case studies

Advances in Transmission Electron Microscopy for the Study of Soft and Hard Matter

This book provides readers with some examples of advanced applications of electron microscopy on organic and inorganic specimens, highlighting out how new original approaches could provide a deeper understanding of the properties of matter and how a transmission electron microscope is not only a microscope but also a flexible tool for tailoring experiments, properly suited, to the issue of interest

Sparsity Analysis for Computer Vision Applications

Ph.DDOCTOR OF PHILOSOPH