Deep learning long-range information in undirected graphs with wave networks

Graph algorithms are key tools in many fields of science and technology. Some of these algorithms depend on propagating information between distant nodes in a graph. Recently, there have been a number of deep learning architectures proposed to learn on undirected graphs. However, most of these architectures aggregate information in the local neighborhood of a node, and therefore they may not be capable of efficiently propagating long-range information. To solve this problem we examine a recently proposed architecture, wave, which propagates information back and forth across an undirected graph in waves of nonlinear computation. We compare wave to graph convolution, an architecture based on local aggregation, and find that wave learns three different graph-based tasks with greater efficiency and accuracy. These three tasks include (1) labeling a path connecting two nodes in a graph, (2) solving a maze presented as an image, and (3) computing voltages in a circuit. These tasks range from trivial to very difficult, but wave can extrapolate from small training examples to much larger testing examples. These results show that wave may be able to efficiently solve a wide range of problems that require long-range information propagation across undirected graphs. An implementation of the wave network, and example code for the maze problem are included in the tflon deep learning toolkit (https://bitbucket.org/mkmatlock/tflon)

Structure-based control of complex networks with nonlinear dynamics

What can we learn about controlling a system solely from its underlying network structure? Here we adapt a recently developed framework for control of networks governed by a broad class of nonlinear dynamics that includes the major dynamic models of biological, technological, and social processes. This feedback-based framework provides realizable node overrides that steer a system towards any of its natural long term dynamic behaviors, regardless of the specific functional forms and system parameters. We use this framework on several real networks, identify the topological characteristics that underlie the predicted node overrides, and compare its predictions to those of structural controllability in control theory. Finally, we demonstrate this framework's applicability in dynamic models of gene regulatory networks and identify nodes whose override is necessary for control in the general case, but not in specific model instances.Comment: Includes main text and supporting informatio

Convolutional neural networks automate detection for tracking of submicron scale particles in 2D and 3D

Particle tracking is a powerful biophysical tool that requires conversion of large video files into position time series, i.e. traces of the species of interest for data analysis. Current tracking methods, based on a limited set of input parameters to identify bright objects, are ill-equipped to handle the spectrum of spatiotemporal heterogeneity and poor signal-to-noise ratios typically presented by submicron species in complex biological environments. Extensive user involvement is frequently necessary to optimize and execute tracking methods, which is not only inefficient but introduces user bias. To develop a fully automated tracking method, we developed a convolutional neural network for particle localization from image data, comprised of over 6,000 parameters, and employed machine learning techniques to train the network on a diverse portfolio of video conditions. The neural network tracker provides unprecedented automation and accuracy, with exceptionally low false positive and false negative rates on both 2D and 3D simulated videos and 2D experimental videos of difficult-to-track species

Exosomes regulate neurogenesis and circuit assembly.

Exosomes are thought to be released by all cells in the body and to be involved in intercellular communication. We tested whether neural exosomes can regulate the development of neural circuits. We show that exosome treatment increases proliferation in developing neural cultures and in vivo in dentate gyrus of P4 mouse brain. We compared the protein cargo and signaling bioactivity of exosomes released by hiPSC-derived neural cultures lacking MECP2, a model of the neurodevelopmental disorder Rett syndrome, with exosomes released by isogenic rescue control neural cultures. Quantitative proteomic analysis indicates that control exosomes contain multiple functional signaling networks known to be important for neuronal circuit development. Treating MECP2-knockdown human primary neural cultures with control exosomes rescues deficits in neuronal proliferation, differentiation, synaptogenesis, and synchronized firing, whereas exosomes from MECP2-deficient hiPSC neural cultures lack this capability. These data indicate that exosomes carry signaling information required to regulate neural circuit development

Cell identification in whole-brain multiview images of neural activation

We present a scalable method for brain cell identification in multiview confocal light sheet microscopy images. Our algorithmic pipeline includes a hierarchical registration approach and a novel multiview version of semantic deconvolution that simultaneously enhance visibility of fluorescent cell bodies, equalize their contrast, and fuses adjacent views into a single 3D images on which cell identification is performed with mean shift. We present empirical results on a whole-brain image of an adult Arc-dVenus mouse acquired at 4micron resolution. Based on an annotated test volume containing 3278 cells, our algorithm achieves an $F_1$ measure of 0.89

Differentiation and Characterization of Excitatory and Inhibitory Synapses by Cryo-electron Tomography and Correlative Microscopy.

As key functional units in neural circuits, different types of neuronal synapses play distinct roles in brain information processing, learning, and memory. Synaptic abnormalities are believed to underlie various neurological and psychiatric disorders. Here, by combining cryo-electron tomography and cryo-correlative light and electron microscopy, we distinguished intact excitatory and inhibitory synapses of cultured hippocampal neurons, and visualized the in situ 3D organization of synaptic organelles and macromolecules in their native state. Quantitative analyses of >100 synaptic tomograms reveal that excitatory synapses contain a mesh-like postsynaptic density (PSD) with thickness ranging from 20 to 50 nm. In contrast, the PSD in inhibitory synapses assumes a thin sheet-like structure ∼12 nm from the postsynaptic membrane. On the presynaptic side, spherical synaptic vesicles (SVs) of 25-60 nm diameter and discus-shaped ellipsoidal SVs of various sizes coexist in both synaptic types, with more ellipsoidal ones in inhibitory synapses. High-resolution tomograms obtained using a Volta phase plate and electron filtering and counting reveal glutamate receptor-like and GABAA receptor-like structures that interact with putative scaffolding and adhesion molecules, reflecting details of receptor anchoring and PSD organization. These results provide an updated view of the ultrastructure of excitatory and inhibitory synapses, and demonstrate the potential of our approach to gain insight into the organizational principles of cellular architecture underlying distinct synaptic functions.SIGNIFICANCE STATEMENT To understand functional properties of neuronal synapses, it is desirable to analyze their structure at molecular resolution. We have developed an integrative approach combining cryo-electron tomography and correlative fluorescence microscopy to visualize 3D ultrastructural features of intact excitatory and inhibitory synapses in their native state. Our approach shows that inhibitory synapses contain uniform thin sheet-like postsynaptic densities (PSDs), while excitatory synapses contain previously known mesh-like PSDs. We discovered "discus-shaped" ellipsoidal synaptic vesicles, and their distributions along with regular spherical vesicles in synaptic types are characterized. High-resolution tomograms further allowed identification of putative neurotransmitter receptors and their heterogeneous interaction with synaptic scaffolding proteins. The specificity and resolution of our approach enables precise in situ analysis of ultrastructural organization underlying distinct synaptic functions

A deep convolutional neural network approach for astrocyte detection

Astrocytes are involved in various brain pathologies including trauma, stroke, neurodegenerative disorders such as Alzheimer's and Parkinson's diseases, or chronic pain. Determining cell density in a complex tissue environment in microscopy images and elucidating the temporal characteristics of morphological and biochemical changes is essential to understand the role of astrocytes in physiological and pathological conditions. Nowadays, manual stereological cell counting or semi-automatic segmentation techniques are widely used for the quantitative analysis of microscopy images. Detecting astrocytes automatically is a highly challenging computational task, for which we currently lack efficient image analysis tools. We have developed a fast and fully automated software that assesses the number of astrocytes using Deep Convolutional Neural Networks (DCNN). The method highly outperforms state-of-the-art image analysis and machine learning methods and provides precision comparable to those of human experts. Additionally, the runtime of cell detection is significantly less than that of other three computational methods analysed, and it is faster than human observers by orders of magnitude. We applied our DCNN-based method to examine the number of astrocytes in different brain regions of rats with opioid-induced hyperalgesia/tolerance (OIH/OIT), as morphine tolerance is believed to activate glia. We have demonstrated a strong positive correlation between manual and DCNN-based quantification of astrocytes in rat brain.Peer reviewe

AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

살아있는 뉴런과 동물에서 mRNA 관찰에 대한 연구

학위논문(박사) -- 서울대학교대학원 : 자연과학대학 물리·천문학부(물리학전공), 2021.8. 이병훈.mRNA는 유전자 발현의 첫번째 산물이면서, 리보솜과 함께 단백질을 합성한다. 특히 뉴런에서, 몇몇 RNA들은 자극에 의해 만들어지고, 뉴런의 특정 부분으로 수송되어 국소적으로 단백질 양을 조절할 수 있게 한다. 최근 mRNA 표지 기술의 발전으로 살아있는 세포에서 단일 mRNA를 관찰하는 것이 가능해졌다. 이 연구에서, 우리는 RNA 이미징 기술을 이용해, 기억 형성과 상기할 때 활성화된 뉴런의 집합을 찾는 것 뿐 아니라, 뉴런의 축삭돌기에서 mRNA가 어떻게 수송되는지를 관찰했다. 이 논문의 첫 부분에서 우리는 신경 자극에 반응해서 만들어지는 것으로 알려진, Arc 유전자의 전사를 관찰하였다. 기억은 engram 혹은 기억 흔적 (memory trace)라고 불리는 뉴런들의 집합에 저장되어 있다고 생각된다. 그러나, 시간에 따라서 이런 기억 흔적세포들의 집합이 어떻게 변하고, 변화하면서도 어떻게 정보를 유지할 수 있는지 잘 알려져 있지 않다. 또한, 살아있는 동물에서, 기억 흔적세포를 긴 시간 동안 여러 번 찾아내는 것은 어려운 일이었다. 이 연구에서는 genetically-encoded RNA indicator (GERI) 기술을 사용해, 기억 흔적세포의 표식으로 널리 사용되는 Arc mRNA의 전사과정을 살아있는 쥐에서 관찰하였다. GERI를 이용함으로써, 기존 방법들의 한계점이었던 시간 제약 없이, 실시간으로 Arc를 발현하는 뉴런들을 찾아낼 수 있었다. 쥐에게 공간 공포 기억을 주고 나서 여러 번 기억을 상기시키는 행동실험 후에 Arc를 발현하는 세포를 식별했을 때, CA1에서는 Arc를 발현하는 세포가 이틀 후에는 더 이상 활성화되지 않았으나, RSC의 경우 4퍼센트의 뉴런들이 계속해서 활성화하는 것을 관찰했다. 신경활동과 유전자 발현을 같이 조사하기 위해, 쥐가 가상 환경을 탐험하고 있을 때 GERI와 칼슘 이미징을 동시에 진행하였다. 그 결과, 기억을 형성할 때와 상기시킬 때 Arc를 발현했던 뉴런들이 기억을 표상하는 것을 알아낼 수 있었다. 이처럼 GERI 기술을 이용해 살아있는 동물에서 유전자 발현된 세포를 찾아내는 방식은 다양한 학습 및 기억 과정에서 기억 흔적세포의 dynamics에 대해 알아낼 수 있을 것으로 기대된다. 이 논문의 두번째 부분에서, 우리는 세포 골격의 기본 구성 단위가 되는 β-actin의 mRNA를 축삭돌기에서 관찰하였다. mRNA의 국소화 (localization)를 통한 국소 단백질 합성은 축삭돌기 (axon)의 성장과 재생에 중요한 역할이 있다고 알려져 있다. 하지만, 아직 mRNA의 국소화가 축삭돌기에서 어떻게 조절되고 있는지 잘 알려져 있지 않다. 이 문제를 해결하기 위해서, 우리는 모든 β-actin mRNA가 형광으로 표지된 유전자 변형 쥐를 이용해, 살아있는 축삭돌기에서 β-actin mRNA를 관찰하였다. 이 쥐의 뉴런을 축삭을 구분해 줄 수 있는 미세유체 장치 (microfluidic device)에 배양한 뒤에, β-actin mRNA를 관찰하고 추적을 진행했다. 축삭은 세포 몸통으로부터 길게 자라기 때문에 mRNA가 먼 거리를 수송되어야 함에도 불구하고, 대부분의 mRNA가 수상돌기에 비해 덜 움직이고 작은 영역에서 움직이는 것을 보았다. 우리는 β-actin mRNA가 주로 축삭돌기의 가지가 될 수 있는 filopodia 근처와, 시냅스가 만들어지는 bouton 근처에 국소화되는 것을 관찰했다. Filopodia와 bouton이 actin이 풍부한 부분으로 알려져 있기 때문에, 우리는 액틴 필라멘트와 β-actin mRNA의 움직임간에 연관성을 조사했다. 흥미롭게도, 우리는 β-actin mRNA가 액틴 필라멘트와 같이 국소화 되고, β-actin mRNA가 액틴 필라멘트 안에서 sub-diffusive한 움직임을 보였으며, 먼 거리를 움직이던 mRNA도 액틴 필라멘트에 고정되는 모습도 확인할 수 있었다. 축삭에서 β-actin mRNA 움직임을 본 이번 관찰은 mRNA 수송 및 국소화에 대한 생물물리학 적 메커니즘의 기반이 될 수 있을 것이다.mRNA is the first product of the gene expression and facilitates the protein synthesis. Especially in neurons, some RNAs are transcribed in response to stimuli and transported to the specific region, altering local proteome for neurons to function normally. Recent advances of mRNA labeling techniques allowed us to observe the single mRNAs in live cells. In this thesis, we applied RNA imaging technique not only to identify the neuronal ensemble that activated during memory formation and retrieval, but also to traffic mRNAs transported to the axon. In the first part of the thesis, we observed the transcription site of Arc gene, one of the immediate-early gene, which is rapidly transcribed upon the neural stimuli. Because of the characteristic of expressing in response to stimuli, Arc is widely used as a marker for memory trace cells thought to store memories. However, little is known about the ensemble dynamics of these cells because it has been challenging to observe them repeatedly over long periods of time in vivo. To overcome this limitation, we present a genetically-encoded RNA indicator (GERI) technique for intravital chronic imaging of endogenous Arc mRNA. We used our GERI to identify Arc-positive neurons in real time without the time lag associated with reporter protein expression in conventional approaches. We found that Arc-positive neuronal populations rapidly turned over within two days in CA1, whereas ~4% of neurons in the retrosplenial cortex consistently expressed Arc upon contextual fear conditioning and repeated memory retrievals. Dual imaging of GERI and calcium indicator in CA1 of mice navigating a virtual reality environment revealed that only the overlapping population of neurons expressing Arc during encoding and retrieval exhibited relatively high calcium activity in a context-specific manner. This in vivo RNA imaging approach has potential to unravel the dynamics of engram cells underlying various learning and memory processes. In the second part of this thesis, we imaged β-actin mRNAs, which can generate a cytoskeletal protein, β-actin, through translation. Local protein synthesis has a critical role in axonal guidance and regeneration. Yet it is not clearly understood how the mRNA localization is regulated in axons. To address these questions, we investigated mRNA motion in live axons using a transgenic mouse that expresses fluorescently labeled endogenous β-actin mRNA. By culturing hippocampal neurons in a microfluidic device that allows separation of axons from dendrites, we performed single particle tracking of β-actin mRNA selectively in axons. Although axonal mRNAs need to travel a long distance, we observed that most axonal mRNAs show much less directed motion than dendritic mRNAs. We found that β-actin mRNAs frequently localize at the neck of filopodia which can grow as axon collateral branches and at varicosities where synapses typically occur. Since both filopodia and varicosities are known as actin-rich areas, we investigated the dynamics of actin filaments and β-actin mRNAs simultaneously by using high-speed dual-color imaging. We found that axonal mRNAs colocalize with actin filaments and show sub-diffusive motion within the actin-rich regions. The novel findings on the dynamics of β-actin mRNA will shed important light on the biophysical mechanisms of mRNA transport and localization in axons.1. INTRODUCTION, 1 1.1. Neuronal ensemble, 1 1.2. Immediate-early Gene (IEG), 3 1.3. Methods for IEG-positive neurons, 3 1.4. Two-photon microscope, 5 1.5. References, 7 2. IMAGING ARC mRNA TRANSCRIPTION SITES IN LIVE MICE, 9 2.1. Introduction, 9 2.2. Materials and Methods, 10 2.3. Results and Discussion, 18 2.4. References, 26 3. NEURONS EXPRESSING ARC mRNA DURING REPEATED MEMORY RETRIEVALS, 28 3.1. Introduction, 28 3.2. Results and Discussion, 28 3.3. References, 35 4. NEURAL ACTIVITIES OF ARC+ NEURONS, 36 4.1. Introduction, 36 4.2. Materials and Methods, 37 4.3. Results and Discussion, 38 4.4. References, 52 5. AXONAL mRNA DYNAMICS IN LIVE NEURONS, 54 5.1. Introduction, 54 5.2. Materials and Methods, 55 5.3. Results and Discussion, 59 5.4. References, 70 6. CONCLUSION AND OUTLOOK, 72 ABSTRACT IN KOREAN (국문초록), 76박

Biomedical applications of three‐dimensional bioprinted craniofacial tissue engineering

Abstract Anatomical complications of the craniofacial regions often present considerable challenges to the surgical repair or replacement of the damaged tissues. Surgical repair has its own set of limitations, including scarcity of the donor tissues, immune rejection, use of immune suppressors followed by the surgery, and restriction in restoring the natural aesthetic appeal. Rapid advancement in the field of biomaterials, cell biology, and engineering has helped scientists to create cellularized skeletal muscle‐like structures. However, the existing method still has limitations in building large, highly vascular tissue with clinical application. With the advance in the three‐dimensional (3D) bioprinting technique, scientists and clinicians now can produce the functional implants of skeletal muscles and bones that are more patient‐specific with the perfect match to the architecture of their craniofacial defects. Craniofacial tissue regeneration using 3D bioprinting can manage and eliminate the restrictions of the surgical transplant from the donor site. The concept of creating the new functional tissue, exactly mimicking the anatomical and physiological function of the damaged tissue, looks highly attractive. This is crucial to reduce the donor site morbidity and retain the esthetics. 3D bioprinting can integrate all three essential components of tissue engineering, that is, rehabilitation, reconstruction, and regeneration of the lost craniofacial tissues. Such integration essentially helps to develop the patient‐specific treatment plans and damage site‐driven creation of the functional implants for the craniofacial defects. This article is the bird's eye view on the latest development and application of 3D bioprinting in the regeneration of the skeletal muscle tissues and their application in restoring the functional abilities of the damaged craniofacial tissue. We also discussed current challenges in craniofacial bone vascularization and gave our view on the future direction, including establishing the interactions between tissue‐engineered skeletal muscle and the peripheral nervous system