Deep-learning-based segmentation of small extracellular vesicles in transmission electron microscopy images

Small extracellular vesicles (sEVs) are cell-derived vesicles of nanoscale size (~30-200 nm) that function as conveyors of information between cells, reflecting the cell of their origin and its physiological condition in their content. Valuable information on the shape and even on the composition of individual sEVs can be recorded using transmission electron microscopy (TEM). Unfortunately, sample preparation for TEM image acquisition is a complex procedure, which often leads to noisy images and renders automatic quantification of sEVs an extremely difficult task. We present a completely deep-learning-based pipeline for the segmentation of sEVs in TEM images. Our method applies a residual convolutional neural network to obtain fine masks and use the Radon transform for splitting clustered sEVs. Using three manually annotated datasets that cover a natural variability typical for sEV studies, we show that the proposed method outperforms two different state-of-the-art approaches in terms of detection and segmentation performance. Furthermore, the diameter and roundness of the segmented vesicles are estimated with an error of less than 10%, which supports the high potential of our method in biological applications.We want to acknowledge the support of NVIDIA Corporation with the donation of the Titan X (Pascal) GPU used for this research. This work was supported by the Spanish Ministry of Economy and Competitiveness (TEC2013-48552-C2-1-R, TEC2015-73064-EXP, TEC2016-78052-R) (EGM-AMB), a 2017 Leonardo Grant for Researchers and Cultural Creators, BBVA Foundation (EGM-AMB), and the Czech Science Foundation (GA17-05048S)(MM-PM) and (GJ17-11776Y) (AK-VP)

Deep learning based domain adaptation for mitochondria segmentation on EM volumes.

[EN] BACKGROUND AND OBJECTIVE: Accurate segmentation of electron microscopy (EM) volumes of the brain is essential to characterize neuronal structures at a cell or organelle level. While supervised deep learning methods have led to major breakthroughs in that direction during the past years, they usually require large amounts of annotated data to be trained, and perform poorly on other data acquired under similar experimental and imaging conditions. This is a problem known as domain adaptation, since models that learned from a sample distribution (or source domain) struggle to maintain their performance on samples extracted from a different distribution or target domain. In this work, we address the complex case of deep learning based domain adaptation for mitochondria segmentation across EM datasets from different tissues and species. METHODS: We present three unsupervised domain adaptation strategies to improve mitochondria segmentation in the target domain based on (1) state-of-the-art style transfer between images of both domains; (2) self-supervised learning to pre-train a model using unlabeled source and target images, and then fine-tune it only with the source labels; and (3) multi-task neural network architectures trained end-to-end with both labeled and unlabeled images. Additionally, to ensure good generalization in our models, we propose a new training stopping criterion based on morphological priors obtained exclusively in the source domain. The code and its documentation are publicly available at https://github.com/danifranco/EM_domain_adaptation. RESULTS: We carried out all possible cross-dataset experiments using three publicly available EM datasets. We evaluated our proposed strategies and those of others based on the mitochondria semantic labels predicted on the target datasets. CONCLUSIONS: The methods introduced here outperform the baseline methods and compare favorably to the state of the art. In the absence of validation labels, monitoring our proposed morphology-based metric is an intuitive and effective way to stop the training process and select in average optimal models.I. Arganda-Carreras would like to acknowledge the support of the 2020 Leonardo Grant for Researchers and Cultural Creators, BBVA Foundation. This work is supported in part by the University of the Basque Country UPV/EHU grant GIU19/027 and by Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación, under grant PID2019-109820RB-I00, MCIN/AEI /10.13039/501100011033/, cofinanced by European Regional Development Fund (ERDF), “A way of making Europe.

Deep learning-assisted high-throughput analysis of freeze-fracture replica images applied to glutamate receptors and calcium channels at hippocampal synapses

The molecular anatomy of synapses defines their characteristics in transmission and plasticity. Precise measurements of the number and distribution of synaptic proteins are important for our understanding of synapse heterogeneity within and between brain regions. Freeze–fracture replica immunogold electron microscopy enables us to analyze them quantitatively on a two-dimensional membrane surface. Here, we introduce Darea software, which utilizes deep learning for analysis of replica images and demonstrate its usefulness for quick measurements of the pre- and postsynaptic areas, density and distribution of gold particles at synapses in a reproducible manner. We used Darea for comparing glutamate receptor and calcium channel distributions between hippocampal CA3-CA1 spine synapses on apical and basal dendrites, which differ in signaling pathways involved in synaptic plasticity. We found that apical synapses express a higher density of α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptors and a stronger increase of AMPA receptors with synaptic size, while basal synapses show a larger increase in N-methyl-D-aspartate (NMDA) receptors with size. Interestingly, AMPA and NMDA receptors are segregated within postsynaptic sites and negatively correlated in density among both apical and basal synapses. In the presynaptic sites, Cav2.1 voltage-gated calcium channels show similar densities in apical and basal synapses with distributions consistent with an exclusion zone model of calcium channel-release site topography

Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

Recent studies have supported the relation between mitochondrial functions and degenerative disorders related to ageing, such as Alzheimer's and Parkinson's diseases. Since these studies have exposed the need for detailed and high-resolution analysis of physical alterations in mitochondria, it is necessary to be able to perform segmentation and 3D reconstruction of mitochondria. However, due to the variety of mitochondrial structures, automated mitochondria segmentation and reconstruction in electron microscopy (EM) images have proven to be a difficult and challenging task. This paper puts forward an effective and automated pipeline based on deep learning to realize mitochondria segmentation in different EM images. The proposed pipeline consists of three parts: (1) utilizing image registration and histogram equalization as image pre-processing steps to maintain the consistency of the dataset; (2) proposing an effective approach for 3D mitochondria segmentation based on a volumetric, residual convolutional and deeply supervised network; and (3) employing a 3D connection method to obtain the relationship of mitochondria and displaying the 3D reconstruction results. To our knowledge, we are the first researchers to utilize a 3D fully residual convolutional network with a deeply supervised strategy to improve the accuracy of mitochondria segmentation. The experimental results on anisotropic and isotropic EM volumes demonstrate the effectiveness of our method, and the Jaccard index of our segmentation (91.8% in anisotropy, 90.0% in isotropy) and F1 score of detection (92.2% in anisotropy, 90.9% in isotropy) suggest that our approach achieved state-of-the-art results. Our fully automated pipeline contributes to the development of neuroscience by providing neurologists with a rapid approach for obtaining rich mitochondria statistics and helping them elucidate the mechanism and function of mitochondria

Stable deep neural network architectures for mitochondria segmentation on electron microscopy volumes

Electron microscopy (EM) allows the identification of intracellular organelles such as mitochondria, providing insights for clinical and scientific studies. In recent years, a number of novel deep learning architectures have been published reporting superior performance, or even human-level accuracy, compared to previous approaches on public mitochondria segmentation datasets. Unfortunately, many of these publications do not make neither the code nor the full training details public to support the results obtained, leading to reproducibility issues and dubious model comparisons. For that reason, and following a recent code of best practices for reporting experimental results, we present an extensive study of the state-of-the-art deep learning architectures for the segmentation of mitochondria on EM volumes, and evaluate the impact in performance of different variations of 2D and 3D U-Net-like models for this task. To better understand the contribution of each component, a common set of pre- and post-processing operations has been implemented and tested with each approach. Moreover, an exhaustive sweep of hyperparameters values for all architectures have been performed and each configuration has been run multiple times to report the mean and standard deviation values of the evaluation metrics. Using this methodology, we found very stable architectures and hyperparameter configurations that consistently obtain state-of-the-art results in the well-known EPFL Hippocampus mitochondria segmentation dataset. Furthermore, we have benchmarked our proposed models on two other available datasets, Lucchi++ and Kasthuri++, where they outperform all previous works. The code derived from this research and its documentation are publicly available

Yüz analizine dayalı derin öğrenme tabanlı bir ilgi tespit sisteminin gerçekleştirilmesi

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Pazarlama alanında, en heyecan verici, yenilikçi ve gelecek vaat eden konulardan biri müşteri ilgisinin ölçülmesidir. Müşteri ilgisini ölçmek için geleneksel bir yaklaşım olan müşteri memnuniyet anketleri, günümüzde müşteriyi rahatsız edici bir yöntem olarak değerlendirilmektedir. Diğer bir müşteri ilgisi ölçme yöntemi de bir insan gözlemcinin müşteri davranışlarını izleyip kaydetmesi şeklinde olabilir ancak bu da deneyimli ve yetenekli insan gerektirir. Ayrıca her gözlemci, insan davranışlarını farklı yorumlayabileceğinden, sonuçlar tarafsız olamayabilir. Bu nedenle müşteri davranışlarını izlemek için, rahatsız edici olmayan, nicel, tarafsız ve otomatik sonuçlar üretebilen sistemlere ihtiyaç vardır. Bu tez çalışması ile müşteri davranışının bilgisayar aracılığı ile izlenmesi ve bir ürüne ya da reklama ilgi duyan müşterilerin belirlenmesi için derin öğrenme tabanlı bir sistem önerilmektedir. Bu sistem ilk olarak müşterinin dikkatini baş yönelimi tahminiyle ölçer. Baş pozisyonları reklama veya ilgilenilen ürüne yönelik olan müşteriler için, sistem yüz ifadelerini analiz eder ve yüz ifadesine dayalı olarak müşterilerin ürünlere veya reklamlara olan ilgisini tahmin eder. Sistem ön yüz görüntülerinin algılanmasıyla çalışmaya başlar, ardından yüz ifadesi tespiti için önemli olan ağız, göz ve kaş bileşenleri tespit edilip yüz üzerinde bölütlenir ve bölütlenmiş bir yüz görüntüsü oluşturulur. Son olarak, ham yüz görüntüleri ile birlikte, elde edilen bölütlenmiş yüz görüntülerine ait güven değerleri kullanılarak yüz ifadeleri tespit edilir. İki aşamalı olan bu yüz ifadesi tespit yöntemi, parça tabanlı özellikler ile bütünsel yüz özelliklerini birleştirerek daha güçlü bir yüz ifadesi sistemi sunar. Sistemde ayrıca müşteri yüzleri etiketlenerek video çerçevesi boyunca takip edilir. Her müşteriye ait yüz ifadeleri belirli bir süre boyunca depolanır ve bu süre sonunda müşterinin ürüne ilgili olup olmadığı ile ilgili sonuç bildirilir. Önerilen sistem müşteri davranışlarının izlenmesine ek olarak, farklı odak gruplarının çeşitli fikirlere, resimlere, seslere, kelimelere ve diğer uyaranlara duygusal tepkisini izlemek için de kullanılabilir.In the marketing research, one of the most exciting, innovative, and promising trends is quantification of customer interest. The customer satisfaction survey, which is a traditional approach to quantify customer interest, has come to be considered as an invasive method in recent years. Recording customer interest by a salesperson who observes customers' behavior during the advertisement watching or shopping phase is another approach. However, this task requires specific skills for every salesperson, and each observer may interpret customer behaviors differently. Consequently, there is a critical need to develop non-invasive, objective, and quantitative tools for monitoring customer interest. This study presents a deep learning-based system for monitoring customer behavior specifically for detection of interest. The proposed system first measures customer attention through head pose estimation. For those customers whose heads are oriented toward the advertisement or the product of interest, the system further analyzes the facial expressions and reports customers' interest. The proposed system starts by detecting frontal face poses; facial components important for facial expression recognition are then segmented and an iconized face image is generated; finally, facial expressions are analyzed using the confidence values of obtained iconized face image combined with the raw facial images. This approach fuses local part-based features with holistic facial information for robust facial expression recognition. The system is also tracked human faces along the video frame by labeling the faces. The facial expressions of each customer are stored for a certain period of time; at the end of this period, the result of whether the customer is related to the product or advirtesement is notified. With the proposed processing pipeline, using a basic imaging device, such as a webcam, head pose estimation and facial expression recognition is possible. The proposed pipeline can be used to monitor emotional response of focus groups to various ideas, pictures, sounds, words, and other stimuli

Kısmi ve tam yüz görüntüleri üzerinde makine öğrenmesi yöntemleriyle yüz ifadesi tespiti

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Yüz ifadeleri insanlar arası iletişimin önemli bir parçası olduğu gibi insan makine etkileşiminde de önemli rol oynamaktadır. Suçlu tespiti, sürücü dikkatinin izlenmesi, hasta takibi gibi önemli konularda karar vermede yüz ifadesi tespiti kullanılmaktadır. Bu sebeple, yüz ifadelerinin sistemler aracılığı ile otomatik tespiti popüler bir makine öğrenmesi çalışma alanıdır. Bu tez çalışmasında yüz ifadesi sınıflandırma çalışmaları yapılmıştır. Yapılan yüz ifadesi tespiti uygulamaları genel olarak iki başlık altında toplanabilir. Bunlardan ilki kısmi yüz görüntülerinin klasik makine öğrenmesi yöntemleriyle analizi ve ikincisi ise tüm yüz görüntülerinin derin öğrenme yöntemleri ile analiz edilmesidir. Geliştirilen ilk uygulamada, yüz görüntülerinden duygu tespiti için literatürdeki çalışmalardan farklı olarak sadece göz ve kaşların bulunduğu bölgeler kullanılarak sınıflandırma yapılmış ve yüksek başarım elde edilmiştir. Önerilen bu yöntem sayesinde yüz ifadesi tespitleri alt yüz kapanmalarından veya ağız hareketlerinden etkilenmeyecek, gürbüz özniteliklerin seçimi ile daha az öznitelikle sınırlı kaynaklara sahip cihazlarda çalışabilecek niteliktedir. Ayrıca önerilen sistemin genelleme yeteneğinin yüksek olduğu karşılaştırmalı olarak deneysel çalışmalarla ortaya konulmuştur. Tez kapsamında yapılan diğer yüz ifadesi sınıflandırma çalışmaları tüm yüz görüntüleri kullanılarak derin öğrenme yöntemleri ile gerçeklenmiştir. Önerilen yaklaşımlardan birisi yüz bölütleme çalışmasıdır. Bu çalışmalar ile elde edilen bölütlenmiş görüntüde yüz ifadesi ile ilgili öznitelikler korunmakta, kişisel herhangi bir veri saklanmamakta ve böylece kişisel gizlilik de korunmuş olmaktadır. Ayrıca bölütlenmiş görüntü ile orijinal yüz görüntüsünün birleşimi; yüz ifadesi için önemli olan kaş, göz ve ağız bölgelerine odaklanılarak yüz ifadelerinin tanınma başarımının arttırılması sağlamıştır.Facial expressions are important for interpersonal communication also play an important role in human machine interaction. Facial expressions are used in many areas such as criminal detection, driver attention monitoring, patient monitoring. Therefore, automatic facial expression recognition systems are a popular machine learning problem. In this thesis study, facial expression recognition studies are performed. In general, the applications of facial expression recognition can be grouped under two topic in this thesis: analysis of partial facial images with classical machine learning methods and analysis of whole facial images with deep learning methods. In the first application, classification of the facial expressions from facial images was performed using only eye and eyebrows regions. This approach is different from the studies which are studied facial expression recognition in the literature and high success rate was achieved. With this approach, proposed system is more robust for under facial occlusions and mouth motion during speech. Further, according to our experiments, the generalization ability of the proposed system is high. In this thesis, the rest of the facial expression recognition applications was developed with whole face images using deep learning techniques. One of the proposed methods is segmentation of facial parts with CNN. After segmentation process, facial segmented images were obtained. With this segmented images, personal privacy is protected because the segmented images don't include any personal information. Also, the success rate of the classification was increased with combining original raw image and segmented image. Because; eyes, eyebrows and mouth are crucial for facial expression recognition and segmented images have these areas. Therefore, the proposed CNN architecture for classification forces the earlier layers of the CNN system to learn to detect and localize the facial regions, thus providing decoupled and guided training

Detección de mitocondrias en células mediante Deep Learning

Este proyecto se basa en la realización de la segmentación de mitocondrias en imágenes celulares obtenidas mediante microscopia electrónica. Para ello se hace uso principalmente de las redes neuronales, y concretamente una arquitectura de red convolucional llamada U-Net. Se presenta un estudio pormenorizado para la búsqueda de la mejor combinación de parámetros y técnicas que permitan obtener una buena segmentación. En el estudio, se han probado diferentes conjuntos de datos públicos utilizados por la comunidad, donde finalmente se han alcanzado resultados comparables al estado del arte