An Efficient Algorithm for Deep Stochastic Contextual Bandits

In stochastic contextual bandit (SCB) problems, an agent selects an action based on certain observed context to maximize the cumulative reward over iterations. Recently there have been a few studies using a deep neural network (DNN) to predict the expected reward for an action, and the DNN is trained by a stochastic gradient based method. However, convergence analysis has been greatly ignored to examine whether and where these methods converge. In this work, we formulate the SCB that uses a DNN reward function as a non-convex stochastic optimization problem, and design a stage-wise stochastic gradient descent algorithm to optimize the problem and determine the action policy. We prove that with high probability, the action sequence chosen by this algorithm converges to a greedy action policy respecting a local optimal reward function. Extensive experiments have been performed to demonstrate the effectiveness and efficiency of the proposed algorithm on multiple real-world datasets.Comment: Accepted by AAAI 202

HyperLink: Virtual Machine Introspection and Memory Forensic Analysis without Kernel Source Code

Virtual Machine Introspection (VMI) is an approach to inspecting and analyzing the software running inside a virtual machine from the hypervisor. Similarly, memory forensics analyzes the memory snapshots or dumps to understand the runtime state of a physical or virtual machine. The existing VMI and memory forensic tools rely on up-to-date kernel information of the target operating system (OS) to work properly, which often requires the availability of the kernel source code. This requirement prevents these tools from being widely deployed in real cloud environments. In this paper, we present a VMI tool called HyperLink that partially retrieves running process information from a guest virtual machine without its source code. While current introspection and memory forensic solutions support only one or a limited number of kernel versions of the target OS, HyperLink is a one-for-many introspection and forensic tool, i.e., it supports most, if not all, popular OSes regardless of their versions. We implement both online and offline versions of HyperLink.We validate the efficacy of HyperLink under different versions of Linux, Windows, FreeBSD, and Mac OS X. For all the OSes we tested, HyperLink can successfully retrieve the process information in one minute or several seconds. Through online and offline analyses, we demonstrate that HyperLink can help users detect real-world kernel rootkits and play an important role in intrusion detection. Due to its version-agnostic property, HyperLink could become the first introspection and forensic tool that works well in autonomic cloud computing environments

Leveraging Inlier Correspondences Proportion for Point Cloud Registration

In feature-learning based point cloud registration, the correct correspondence construction is vital for the subsequent transformation estimation. However, it is still a challenge to extract discriminative features from point cloud, especially when the input is partial and composed by indistinguishable surfaces (planes, smooth surfaces, etc.). As a result, the proportion of inlier correspondences that precisely match points between two unaligned point clouds is beyond satisfaction. Motivated by this, we devise several techniques to promote feature-learning based point cloud registration performance by leveraging inlier correspondences proportion: a pyramid hierarchy decoder to characterize point features in multiple scales, a consistent voting strategy to maintain consistent correspondences and a geometry guided encoding module to take geometric characteristics into consideration. Based on the above techniques, We build our Geometry-guided Consistent Network (GCNet), and challenge GCNet by indoor, outdoor and object-centric synthetic datasets. Comprehensive experiments demonstrate that GCNet outperforms the state-of-the-art methods and the techniques used in GCNet is model-agnostic, which could be easily migrated to other feature-based deep learning or traditional registration methods, and dramatically improve the performance. The code is available at https://github.com/zhulf0804/NgeNet

Dynamic Cycling of t-SNARE Acylation Regulates Platelet Exocytosis

Platelets regulate vascular integrity by secreting a host of molecules that promote hemostasis and its sequelae. Given the importance of platelet exocytosis, it is critical to understand how it is controlled. The t-SNAREs, SNAP-23 and syntaxin-11, lack classical transmembrane domains (TMDs), yet both are associated with platelet membranes and redistributed into cholesterol-dependent lipid rafts when platelets are activated. Using metabolic labeling and hydroxylamine (HA)/HCl treatment, we showed that both contain thioester-linked acyl groups. Mass spectrometry mapping further showed that syntaxin-11 was modified on cysteine 275, 279, 280, 282, 283, and 285, and SNAP-23 was modified on cysteine 79, 80, 83, 85, and 87. Interestingly, metabolic labeling studies showed incorporation of [3H]palmitate into the t-SNAREs increased although the protein levels were unchanged, suggesting that acylation turns over on the two t-SNAREs in resting platelets. Exogenously added fatty acids did compete with [3H]palmitate for t-SNARE labeling. To determine the effects of acylation, we measured aggregation, ADP/ATP release, as well as P-selectin exposure in platelets treated with the acyltransferase inhibitor cerulenin or the thioesterase inhibitor palmostatin B. We found that cerulenin pretreatment inhibited t-SNARE acylation and platelet function in a dose- and time-dependent manner whereas palmostatin B had no detectable effect. Interestingly, pretreatment with palmostatin B blocked the inhibitory effects of cerulenin, suggesting that maintaining the acylation state is important for platelet function. Thus, our work shows that t-SNARE acylation is actively cycling in platelets and suggests that the enzymes regulating protein acylation could be potential targets to control platelet exocytosis in vivo

Motor neuron apoptosis and neuromuscular junction perturbation are prominent features in a Drosophila model of Fus-mediated ALS

<p>Abstract</p> <p>Backgound</p> <p>Amyotrophic lateral sclerosis (ALS) is progressive neurodegenerative disease characterized by the loss of motor function. Several ALS genes have been identified as their mutations can lead to familial ALS, including the recently reported RNA-binding protein fused in sarcoma (Fus). However, it is not clear how mutations of Fus lead to motor neuron degeneration in ALS. In this study, we present a <it>Drosophila </it>model to examine the toxicity of Fus, its <it>Drosophila </it>orthologue Cabeza (Caz), and the ALS-related Fus mutants.</p> <p>Results</p> <p>Our results show that the expression of wild-type Fus/Caz or FusR521G induced progressive toxicity in multiple tissues of the transgenic flies in a dose- and age-dependent manner. The expression of Fus, Caz, or FusR521G in motor neurons significantly impaired the locomotive ability of fly larvae and adults. The presynaptic structures in neuromuscular junctions were disrupted and motor neurons in the ventral nerve cord (VNC) were disorganized and underwent apoptosis. Surprisingly, the interruption of Fus nuclear localization by either deleting its nuclear localization sequence (NLS) or adding a nuclear export signal (NES) blocked Fus toxicity. Moreover, we discovered that the loss of <it>caz </it>in <it>Drosophila </it>led to severe growth defects in the eyes and VNCs, caused locomotive disability and NMJ disruption, but did not induce apoptotic cell death.</p> <p>Conclusions</p> <p>These data demonstrate that the overexpression of Fus/Caz causes <it>in vivo </it>toxicity by disrupting neuromuscular junctions (NMJs) and inducing apoptosis in motor neurons. In addition, the nuclear localization of Fus is essential for Fus to induce toxicity. Our findings also suggest that Fus overexpression and gene deletion can cause similar degenerative phenotypes but the underlying mechanisms are likely different.</p

Intron gain and loss in segmentally duplicated genes in rice

BACKGROUND: Introns are under less selection pressure than exons, and consequently, intronic sequences have a higher rate of gain and loss than exons. In a number of plant species, a large portion of the genome has been segmentally duplicated, giving rise to a large set of duplicated genes. The recent completion of the rice genome in which segmental duplication has been documented has allowed us to investigate intron evolution within rice, a diploid monocotyledonous species. RESULTS: Analysis of segmental duplication in rice revealed that 159 Mb of the 371 Mb genome and 21,570 of the 43,719 non-transposable element-related genes were contained within a duplicated region. In these duplicated regions, 3,101 collinear paired genes were present. Using this set of segmentally duplicated genes, we investigated intron evolution from full-length cDNA-supported non-transposable element-related gene models of rice. Using gene pairs that have an ortholog in the dicotyledonous model species Arabidopsis thaliana, we identified more intron loss (49 introns within 35 gene pairs) than intron gain (5 introns within 5 gene pairs) following segmental duplication. We were unable to demonstrate preferential intron loss at the 3' end of genes as previously reported in mammalian genomes. However, we did find that the four nucleotides of exons that flank lost introns had less frequently used 4-mers. CONCLUSION: We observed that intron evolution within rice following segmental duplication is largely dominated by intron loss. In two of the five cases of intron gain within segmentally duplicated genes, the gained sequences were similar to transposable elements

Frontotemporal Dementia Nonsense Mutation of Progranulin Rescued by Aminoglycosides

Frontotemporal dementia (FTD) is an early onset dementia and is characterized by progressive atrophy of the frontal and/or temporal lobes. FTD is highly heritable with mutations in progranulin accounting for 5-26% of cases in different populations. Progranulin is involved in endocytosis, secretion and lysosomal processes, but its function under physiological and pathological conditions remains to be defined. Many FTD-causing nonsense progranulin mutations contain a premature termination codon (PTC), thus progranulin haploinsufficiency has been proposed as a major disease mechanism. Currently, there is no effective FTD treatment or therapy. Aminoglycosides are a class of antibiotics that possess a less known function to induce eukaryotic ribosomal readthrough of PTCs to produce a full-length protein. The aminoglycoside-induced readthrough strategy has been utilized to treat multiple human diseases caused by PTCs. In this study, we tested the only clinically approved readthrough small molecule PTC124 and eleven aminoglycosides in a cell culture system on four PTCs responsible for FTD or a related neurodegenerative disease amyotrophic lateral sclerosis. We found that the aminoglycosides G418 and gentamicin B1 rescued the expression of the progranulin R493X mutation. G418 was more effective than gentamicin B1 (~50% rescue vs \u3c 10%), and the effect was dose and time-dependent. The proganulin readthrough protein displayed similar subcellular localization as the wild-type proganulin protein. These data provide an exciting proof-of-concept that aminoglycosides or other readthrough-promoting compounds are a therapeutic avenue for familial FTD caused by proganulin PTC mutations

Motor neuron apoptosis and neuromuscular junction perturbation are prominent features in a \u3cem\u3eDrosophila\u3c/em\u3e model of Fus-mediated ALS

BACKGROUND: Amyotrophic lateral sclerosis (ALS) is progressive neurodegenerative disease characterized by the loss of motor function. Several ALS genes have been identified as their mutations can lead to familial ALS, including the recently reported RNA-binding protein fused in sarcoma (Fus). However, it is not clear how mutations of Fus lead to motor neuron degeneration in ALS. In this study, we present a Drosophila model to examine the toxicity of Fus, its Drosophila orthologue Cabeza (Caz), and the ALS-related Fus mutants. RESULTS: Our results show that the expression of wild-type Fus/Caz or FusR521G induced progressive toxicity in multiple tissues of the transgenic flies in a dose- and age-dependent manner. The expression of Fus, Caz, or FusR521G in motor neurons significantly impaired the locomotive ability of fly larvae and adults. The presynaptic structures in neuromuscular junctions were disrupted and motor neurons in the ventral nerve cord (VNC) were disorganized and underwent apoptosis. Surprisingly, the interruption of Fus nuclear localization by either deleting its nuclear localization sequence (NLS) or adding a nuclear export signal (NES) blocked Fus toxicity. Moreover, we discovered that the loss of caz in Drosophila led to severe growth defects in the eyes and VNCs, caused locomotive disability and NMJ disruption, but did not induce apoptotic cell death. CONCLUSIONS: These data demonstrate that the overexpression of Fus/Caz causes in vivo toxicity by disrupting neuromuscular junctions (NMJs) and inducing apoptosis in motor neurons. In addition, the nuclear localization of Fus is essential for Fus to induce toxicity. Our findings also suggest that Fus overexpression and gene deletion can cause similar degenerative phenotypes but the underlying mechanisms are likely different

ALS Mutations of FUS Suppress Protein Translation and Disrupt the Regulation of Nonsense-Mediated Decay

Amyotrophic lateral sclerosis (ALS) is an incurable neurodegenerative disease characterized by preferential motor neuron death. Approximately 15% of ALS cases are familial, and mutations in the fused in sarcoma (FUS) gene contribute to a subset of familial ALS cases. FUS is a multifunctional protein participating in many RNA metabolism pathways. ALS-linked mutations cause a liquid–liquid phase separation of FUS protein in vitro, inducing the formation of cytoplasmic granules and inclusions. However, it remains elusive what other proteins are sequestered into the inclusions and how such a process leads to neuronal dysfunction and degeneration. In this study, we developed a protocol to isolate the dynamic mutant FUS-positive cytoplasmic granules. Proteomic identification of the protein composition and subsequent pathway analysis led us to hypothesize that mutant FUS can interfere with protein translation. We demonstrated that the ALS mutations in FUS indeed suppressed protein translation in N2a cells expressing mutant FUS and fibroblast cells derived from FUS ALS cases. In addition, the nonsense-mediated decay (NMD) pathway, which is closely related to protein translation, was altered by mutant FUS. Specifically, NMD-promoting factors UPF1 and UPF3b increased, whereas a negative NMD regulator, UPF3a, decreased, leading to the disruption of NMD autoregulation and the hyperactivation of NMD. Alterations in NMD factors and elevated activity were also observed in the fibroblast cells of FUS ALS cases. We conclude that mutant FUS suppresses protein biosynthesis and disrupts NMD regulation, both of which likely contribute to motor neuron death

Decay of high-energy electron bound states in crystals

High-energy electrons that are used as a probe of specimens in transmission electron microscopy exhibit a complex and rich behavior due to multiple scattering. Among other things, understanding the dynamical effects is needed for a quantitative analysis of atomic-resolution images and spectroscopic data. In this study, state-correlation functions are computed within the multislice approach that allow to elucidate behaviors of transversely bound states in crystals. These states play an important role as a large fraction of current density can be coupled into them via focused electron probes. We show that bound states are generically unstable and decay monoexponentially with crystal depth. Their attenuation is accompanied by a resonant intensity transfer to Bessel-like wavefunctions that appear as Laue rings in the far-field diffraction patterns. Behaviors of bound states are also quantified when thermal effects are included, as well as point defects. This approach helps to bridge the Bloch wave and multisliced electron propagation pictures of dynamical scattering providing new insights into fundamental solutions of the wave equation, and may assist in developing quantitative STEM/TEM imaging techniques