497 research outputs found

    An Automated Images-to-Graphs Framework for High Resolution Connectomics

    Get PDF
    Reconstructing a map of neuronal connectivity is a critical challenge in contemporary neuroscience. Recent advances in high-throughput serial section electron microscopy (EM) have produced massive 3D image volumes of nanoscale brain tissue for the first time. The resolution of EM allows for individual neurons and their synaptic connections to be directly observed. Recovering neuronal networks by manually tracing each neuronal process at this scale is unmanageable, and therefore researchers are developing automated image processing modules. Thus far, state-of-the-art algorithms focus only on the solution to a particular task (e.g., neuron segmentation or synapse identification). In this manuscript we present the first fully automated images-to-graphs pipeline (i.e., a pipeline that begins with an imaged volume of neural tissue and produces a brain graph without any human interaction). To evaluate overall performance and select the best parameters and methods, we also develop a metric to assess the quality of the output graphs. We evaluate a set of algorithms and parameters, searching possible operating points to identify the best available brain graph for our assessment metric. Finally, we deploy a reference end-to-end version of the pipeline on a large, publicly available data set. This provides a baseline result and framework for community analysis and future algorithm development and testing. All code and data derivatives have been made publicly available toward eventually unlocking new biofidelic computational primitives and understanding of neuropathologies.Comment: 13 pages, first two authors contributed equally V2: Added additional experiments and clarifications; added information on infrastructure and pipeline environmen

    Challenges and Opportunities of End-to-End Learning in Medical Image Classification

    Get PDF
    Das Paradigma des End-to-End Lernens hat in den letzten Jahren die Bilderkennung revolutioniert, aber die klinische Anwendung hinkt hinterher. Bildbasierte computergestützte Diagnosesysteme basieren immer noch weitgehend auf hochtechnischen und domänen-spezifischen Pipelines, die aus unabhängigen regelbasierten Modellen bestehen, welche die Teilaufgaben der Bildklassifikation wiederspiegeln: Lokalisation von auffälligen Regionen, Merkmalsextraktion und Entscheidungsfindung. Das Versprechen einer überlegenen Entscheidungsfindung beim End-to-End Lernen ergibt sich daraus, dass domänenspezifische Zwangsbedingungen von begrenzter Komplexität entfernt werden und stattdessen alle Systemkomponenten gleichzeitig, direkt anhand der Rohdaten, und im Hinblick auf die letztendliche Aufgabe optimiert werden. Die Gründe dafür, dass diese Vorteile noch nicht den Weg in die Klinik gefunden haben, d.h. die Herausforderungen, die sich bei der Entwicklung Deep Learning-basierter Diagnosesysteme stellen, sind vielfältig: Die Tatsache, dass die Generalisierungsfähigkeit von Lernalgorithmen davon abhängt, wie gut die verfügbaren Trainingsdaten die tatsächliche zugrundeliegende Datenverteilung abbilden, erweist sich in medizinische Anwendungen als tiefgreifendes Problem. Annotierte Datensätze in diesem Bereich sind notorisch klein, da für die Annotation eine kostspielige Beurteilung durch Experten erforderlich ist und die Zusammenlegung kleinerer Datensätze oft durch Datenschutzauflagen und Patientenrechte erschwert wird. Darüber hinaus weisen medizinische Datensätze drastisch unterschiedliche Eigenschaften im Bezug auf Bildmodalitäten, Bildgebungsprotokolle oder Anisotropien auf, und die oft mehrdeutige Evidenz in medizinischen Bildern kann sich auf inkonsistente oder fehlerhafte Trainingsannotationen übertragen. Während die Verschiebung von Datenverteilungen zwischen Forschungsumgebung und Realität zu einer verminderten Modellrobustheit führt und deshalb gegenwärtig als das Haupthindernis für die klinische Anwendung von Lernalgorithmen angesehen wird, wird dieser Graben oft noch durch Störfaktoren wie Hardwarelimitationen oder Granularität von gegebenen Annotation erweitert, die zu Diskrepanzen zwischen der modellierten Aufgabe und der zugrunde liegenden klinischen Fragestellung führen. Diese Arbeit untersucht das Potenzial des End-to-End-Lernens in klinischen Diagnosesystemen und präsentiert Beiträge zu einigen der wichtigsten Herausforderungen, die derzeit eine breite klinische Anwendung verhindern. Zunächst wird der letzten Teil der Klassifikations-Pipeline untersucht, die Kategorisierung in klinische Pathologien. Wir demonstrieren, wie das Ersetzen des gegenwärtigen klinischen Standards regelbasierter Entscheidungen durch eine groß angelegte Merkmalsextraktion gefolgt von lernbasierten Klassifikatoren die Brustkrebsklassifikation im MRT signifikant verbessert und eine Leistung auf menschlichem Level erzielt. Dieser Ansatz wird weiter anhand von kardiologischer Diagnose gezeigt. Zweitens ersetzen wir, dem Paradigma des End-to-End Lernens folgend, das biophysikalische Modell, das für die Bildnormalisierung in der MRT angewandt wird, sowie die Extraktion handgefertigter Merkmale, durch eine designierte CNN-Architektur und liefern eine eingehende Analyse, die das verborgene Potenzial der gelernten Bildnormalisierung und einen Komplementärwert der gelernten Merkmale gegenüber den handgefertigten Merkmalen aufdeckt. Während dieser Ansatz auf markierten Regionen arbeitet und daher auf manuelle Annotation angewiesen ist, beziehen wir im dritten Teil die Aufgabe der Lokalisierung dieser Regionen in den Lernprozess ein, um eine echte End-to-End-Diagnose baserend auf den Rohbildern zu ermöglichen. Dabei identifizieren wir eine weitgehend vernachlässigte Zwangslage zwischen dem Streben nach der Auswertung von Modellen auf klinisch relevanten Skalen auf der einen Seite, und der Optimierung für effizientes Training unter Datenknappheit auf der anderen Seite. Wir präsentieren ein Deep Learning Modell, das zur Auflösung dieses Kompromisses beiträgt, liefern umfangreiche Experimente auf drei medizinischen Datensätzen sowie eine Serie von Toy-Experimenten, die das Verhalten bei begrenzten Trainingsdaten im Detail untersuchen, und publiziren ein umfassendes Framework, das unter anderem die ersten 3D-Implementierungen gängiger Objekterkennungsmodelle umfasst. Wir identifizieren weitere Hebelpunkte in bestehenden End-to-End-Lernsystemen, bei denen Domänenwissen als Zwangsbedingung dienen kann, um die Robustheit von Modellen in der medizinischen Bildanalyse zu erhöhen, die letztendlich dazu beitragen sollen, den Weg für die Anwendung in der klinischen Praxis zu ebnen. Zu diesem Zweck gehen wir die Herausforderung fehlerhafter Trainingsannotationen an, indem wir die Klassifizierungskompnente in der End-to-End-Objekterkennung durch Regression ersetzen, was es ermöglicht, Modelle direkt auf der kontinuierlichen Skala der zugrunde liegenden pathologischen Prozesse zu trainieren und so die Robustheit der Modelle gegenüber fehlerhaften Trainingsannotationen zu erhöhen. Weiter adressieren wir die Herausforderung der Input-Heterogenitäten, mit denen trainierte Modelle konfrontiert sind, wenn sie an verschiedenen klinischen Orten eingesetzt werden, indem wir eine modellbasierte Domänenanpassung vorschlagen, die es ermöglicht, die ursprüngliche Trainingsdomäne aus veränderten Inputs wiederherzustellen und damit eine robuste Generalisierung zu gewährleisten. Schließlich befassen wir uns mit dem höchst unsystematischen, aufwendigen und subjektiven Trial-and-Error-Prozess zum Finden von robusten Hyperparametern für einen gegebene Aufgabe, indem wir Domänenwissen in ein Set systematischer Regeln überführen, die eine automatisierte und robuste Konfiguration von Deep Learning Modellen auf einer Vielzahl von medizinischen Datensetzen ermöglichen. Zusammenfassend zeigt die hier vorgestellte Arbeit das enorme Potenzial von End-to-End Lernalgorithmen im Vergleich zum klinischen Standard mehrteiliger und hochtechnisierter Diagnose-Pipelines auf, und präsentiert Lösungsansätze zu einigen der wichtigsten Herausforderungen für eine breite Anwendung unter realen Bedienungen wie Datenknappheit, Diskrepanz zwischen der vom Modell behandelten Aufgabe und der zugrunde liegenden klinischen Fragestellung, Mehrdeutigkeiten in Trainingsannotationen, oder Verschiebung von Datendomänen zwischen klinischen Standorten. Diese Beiträge können als Teil des übergreifende Zieles der Automatisierung von medizinischer Bildklassifikation gesehen werden - ein integraler Bestandteil des Wandels, der erforderlich ist, um die Zukunft des Gesundheitswesens zu gestalten

    Dissecting the neuronal basis of threat responding in mice

    Get PDF
    Environmental threats demand adaptive defensive responses of an organism that ensure its survival. Extreme stressors, however, can unbalance stress homeostasis and lead to long-term changes that impair appropriate defensive behaviors and emotional responses. In my thesis, I assessed (1) the interaction of two stress-related neuromodulatory systems, (2) the effects of a traumatic incident on brain volume and hyperarousal, and (3) sonic vocalization as a defensive behavior in mice, and discussed the topics in three independent studies.In the first study, I evaluated the interaction of two regulatory systems with respect to fear, anxiety, and trauma-related behaviors. Although the endocannabinoid and the corticotropin-releasing factor (CRF) systems are well described in modulating stressrelatedresponses, the direct interaction of both systems remained poorly understood. The generation of a new conditional knockout mouse line that selectively lacked the expression of the cannabinoid type 1 (CB1) receptor in CRF-positive neurons presented no differences in various tests of fear and anxiety-related behaviors under basal conditions or after a traumatic event. Also stress hormone levels were unaffected. However, male knockout animals exhibited a significantly increased acoustic startle response thus suggesting a specific involvement of CB1-CRF interactions in controlling arousal.In the second study, I assessed the consequences of a traumatic experience on behavior and grey matter volume in mice. Whole-brain deformation-based morphometry (DBM) by means of magnetic resonance imaging (MRI) after incubation of a traumatic incident showed changes in the dorsal hippocampus and the reticular nucleus. Using the severity of hyperarousal as regressor for cross-sectional volumetric differences between traumatized mice and controls revealed a negative correlation with the dorsal hippocampus. Further, longitudinal analysis including volumetric measurements before and after the traumatic incident showed that volume reductions in the globus pallidus reflect trauma-related changes in hyperarousal severity.In the third study, I characterized sonic vocalization as a defensive behavior in mice. Mice bred for high anxiety-related behavior (HAB) were found to have a high disposition to emit audible squeaks when taken by the tail which was not the case for any of the other five mouse lines tested. The calls emitted had a fundamental frequency of 3.8 kHz and were shown to be sensitive to anxiolytic but not panicolytic compounds. Manganese-enhanced MRI (MEMRI) scans pointed towards an increased tonic activity, among others, in the periaqueductal grey (PAG). Inhibition of the dorsal PAG by muscimol not only completely abolished sonic vocalization, but also reduced anxiety-like behavior. This suggests that sonic vocalization of mice is related to anxiety and controlled by the PAG. To explore the ecological relevance of defensive vocalization, I performed playback experiments with conspecifics and putative predators. Squeaks turned out to be aversive to HAB mice but became appetitive to both mice and rats when a stimulus mouse was present during playback.Collectively, the results of this thesis provide novel insights into fear and anxiety-related behaviors and shine light onto their mechanistic basis and ecological relevance

    Characterization of aminergic neurons controlling behavioral persistence and motivation in Drosophila melanogaster

    Get PDF
    Deprivation is at odds with survival. To obliterate their condition of hunger animals engage in costly foraging behavior. This conundrum demands unceasing integration of external sensory processing and internal metabolic monitors. Unsurprisingly, such critical behaviors are translated to strong impulses. If unchecked, however, impulsivity can trap animals in unfavorable behavioral states and prevent them from exploiting other valuable opportunities. Categorically, motivational mechanisms have been proposed as the conduit to comply with or decline a response to a strong impulse. Thus, motivation emerges as a critical determinant for observed animal behavioral variability at a given time. Although neuronal circuit diagrams may be deceptively static, neuromodulation can implement behavioral variability in the nervous systems. Bioamines, such as dopamine and norepinephrine, mediate modulatory impact on intrinsic motivational circuits that govern feeding and reward. Across model organisms, however, how animals integrate and update decision-making based on the current motivational and internal states are still poorly understood at the molecular and circuitry levels. Due to its extensive toolbox and amenable miniature nervous systems, Drosophila melanogaster is poised to enrich the current perspective for these concepts. For Drosophila melanogaster, certain odors are salient cues for long distance foraging events. To explore how starved flies make goal-directed decisions, I developed a novel spherical treadmill paradigm. Through the utilization of high-resolution behavioral analyses and tight control of, otherwise highly turbulent, odor delivery, I found that food-deprived flies tracked vinegar persistently even in the repeated absence of a food reward. Combining this behavioral paradigm with immediate neuronal manipulations revealed that this innate persistence recruited circuits that are traditionally linked with learning and memory in an internal state-dependent manner. TH+ cluster dopaminergic neurons, operators of punishment learning, and Dop1R2 signaling enabled this olfactory-driven persistence. Downstream of these dopaminergic neurons, a single mushroom body output neuron, MVP2 was crucial for persistence. MVP2 was necessary and sufficient to integrate hunger state as the underlying motivational drive for food-seeking persistence. Furthermore, I investigated how this strong impulse is counteracted when a fly reaches its goal, nutritious food. A change from odor tracking to food consumption demands the coordination of different sensory systems and motor control subunits. Norepinephrine is implemented in such global switches; such as fight or flight transitions. Using optogenetic manipulation, I demonstrated that the food-seeking drive was suppressed by, an insect norepinephrine analog, octopaminergic input, via VPM4 neurons. Being connected to MVP2 synaptically, which we showed using high-resolution tracing techniques, and a surrogate for feeding at the neuronal level, VPM4 neurons acted as the inhibitory brake on persistent odor tracking to allow feeding related behavior. As a culmination of novel paradigm development, thermo/optogenetic neuronal manipulations and connectomics, this work presents a neuronal microcircuit that recapitulates the alterations of animal behavior faithfully from odor tracking to olfactory suppression during feeding. Specific subsets of dopaminergic and octopaminergic neurons are found to be mediators of motivationally driven events. My findings provide fresh mechanistic insights on how multimodal integration can occur in the brain, how such systems are prone to the internal states, and offers several plausible explanations on how persistence emerges. Finally, this work might serve as a template to better understand the roles and the functional diversity of mammalian aminergic neurons

    Characterization of aminergic neurons controlling behavioral persistence and motivation in Drosophila melanogaster

    Get PDF
    Deprivation is at odds with survival. To obliterate their condition of hunger animals engage in costly foraging behavior. This conundrum demands unceasing integration of external sensory processing and internal metabolic monitors. Unsurprisingly, such critical behaviors are translated to strong impulses. If unchecked, however, impulsivity can trap animals in unfavorable behavioral states and prevent them from exploiting other valuable opportunities. Categorically, motivational mechanisms have been proposed as the conduit to comply with or decline a response to a strong impulse. Thus, motivation emerges as a critical determinant for observed animal behavioral variability at a given time. Although neuronal circuit diagrams may be deceptively static, neuromodulation can implement behavioral variability in the nervous systems. Bioamines, such as dopamine and norepinephrine, mediate modulatory impact on intrinsic motivational circuits that govern feeding and reward. Across model organisms, however, how animals integrate and update decision-making based on the current motivational and internal states are still poorly understood at the molecular and circuitry levels. Due to its extensive toolbox and amenable miniature nervous systems, Drosophila melanogaster is poised to enrich the current perspective for these concepts. For Drosophila melanogaster, certain odors are salient cues for long distance foraging events. To explore how starved flies make goal-directed decisions, I developed a novel spherical treadmill paradigm. Through the utilization of high-resolution behavioral analyses and tight control of, otherwise highly turbulent, odor delivery, I found that food-deprived flies tracked vinegar persistently even in the repeated absence of a food reward. Combining this behavioral paradigm with immediate neuronal manipulations revealed that this innate persistence recruited circuits that are traditionally linked with learning and memory in an internal state-dependent manner. TH+ cluster dopaminergic neurons, operators of punishment learning, and Dop1R2 signaling enabled this olfactory-driven persistence. Downstream of these dopaminergic neurons, a single mushroom body output neuron, MVP2 was crucial for persistence. MVP2 was necessary and sufficient to integrate hunger state as the underlying motivational drive for food-seeking persistence. Furthermore, I investigated how this strong impulse is counteracted when a fly reaches its goal, nutritious food. A change from odor tracking to food consumption demands the coordination of different sensory systems and motor control subunits. Norepinephrine is implemented in such global switches; such as fight or flight transitions. Using optogenetic manipulation, I demonstrated that the food-seeking drive was suppressed by, an insect norepinephrine analog, octopaminergic input, via VPM4 neurons. Being connected to MVP2 synaptically, which we showed using high-resolution tracing techniques, and a surrogate for feeding at the neuronal level, VPM4 neurons acted as the inhibitory brake on persistent odor tracking to allow feeding related behavior. As a culmination of novel paradigm development, thermo/optogenetic neuronal manipulations and connectomics, this work presents a neuronal microcircuit that recapitulates the alterations of animal behavior faithfully from odor tracking to olfactory suppression during feeding. Specific subsets of dopaminergic and octopaminergic neurons are found to be mediators of motivationally driven events. My findings provide fresh mechanistic insights on how multimodal integration can occur in the brain, how such systems are prone to the internal states, and offers several plausible explanations on how persistence emerges. Finally, this work might serve as a template to better understand the roles and the functional diversity of mammalian aminergic neurons

    Dissecting a neuron network: FIB-SEM-based 3D-reconstruction of the visual neuropils in the sea spider Achelia langi (Dohrn, 1881) (Pycnogonida)

    Get PDF
    Background: The research field of connectomics arose just recently with the development of new three-dimensional- electron microscopy (EM) techniques and increasing computing power. So far, only a few model species (for example, mouse, the nematode Caenorhabditis elegans, and the fruit fly Drosophila melanogaster) have been studied using this approach. Here, we present a first attempt to expand this circle to include pycnogonids, which hold a key position for the understanding of arthropod evolution. The visual neuropils in Achelia langi are studied using a focused ion beam-scanning electron microscope (FIB-SEM) crossbeam-workstation, and a three-dimensional serial reconstruction of the connectome is presented. Results: The two eyes of each hemisphere of the sea spider's eye tubercle are connected to a first and a second visual neuropil. The first visual neuropil is subdivided in two hemineuropils, each responsible for one eye and stratified into three layers. Six different neuron types postsynaptic to the retinula (R-cells) axons are characterized by their morphology: five types of descending unipolar neurons and one type of ascending neurons. These cell types are also identified by Golgi impregnations. Mapping of all identifiable chemical synapses indicates that the descending unipolar neurons are postsynaptic to the R-cells and, hence, are second-order neurons. The ascending neurons are predominantly presynaptic and sometimes postsynaptic to the R-cells and may play a feedback role. Conclusions: Comparing these results with the compound eye visual system of crustaceans and insects - the only arthropod visual system studied so far in such detail - we found striking similarities in the morphology and synaptic organization of the different neuron types. Hence, the visual system of pycnogonids shows features of both chelicerate median and mandibulate lateral eyes

    An original framework for understanding human actions and body language by using deep neural networks

    Get PDF
    The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour. By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way. These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively. While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements; both are essential tasks in many computer vision applications, including event recognition, and video surveillance. In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided. The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements. All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods
    • …
    corecore