50 research outputs found

    On the Convergence and Sample Complexity Analysis of Deep Q-Networks with ϵ\epsilon-Greedy Exploration

    Full text link
    This paper provides a theoretical understanding of Deep Q-Network (DQN) with the ε\varepsilon-greedy exploration in deep reinforcement learning. Despite the tremendous empirical achievement of the DQN, its theoretical characterization remains underexplored. First, the exploration strategy is either impractical or ignored in the existing analysis. Second, in contrast to conventional Q-learning algorithms, the DQN employs the target network and experience replay to acquire an unbiased estimation of the mean-square Bellman error (MSBE) utilized in training the Q-network. However, the existing theoretical analysis of DQNs lacks convergence analysis or bypasses the technical challenges by deploying a significantly overparameterized neural network, which is not computationally efficient. This paper provides the first theoretical convergence and sample complexity analysis of the practical setting of DQNs with ϵ\epsilon-greedy policy. We prove an iterative procedure with decaying ϵ\epsilon converges to the optimal Q-value function geometrically. Moreover, a higher level of ϵ\epsilon values enlarges the region of convergence but slows down the convergence, while the opposite holds for a lower level of ϵ\epsilon values. Experiments justify our established theoretical insights on DQNs

    Meta-Analysis of the Alzheimer\u27s Disease Human Brain Transcriptome and Functional Dissection in Mouse Models.

    Get PDF
    We present a consensus atlas of the human brain transcriptome in Alzheimer\u27s disease (AD), based on meta-analysis of differential gene expression in 2,114 postmortem samples. We discover 30 brain coexpression modules from seven regions as the major source of AD transcriptional perturbations. We next examine overlap with 251 brain differentially expressed gene sets from mouse models of AD and other neurodegenerative disorders. Human-mouse overlaps highlight responses to amyloid versus tau pathology and reveal age- and sex-dependent expression signatures for disease progression. Human coexpression modules enriched for neuronal and/or microglial genes broadly overlap with mouse models of AD, Huntington\u27s disease, amyotrophic lateral sclerosis, and aging. Other human coexpression modules, including those implicated in proteostasis, are not activated in AD models but rather following other, unexpected genetic manipulations. Our results comprise a cross-species resource, highlighting transcriptional networks altered by human brain pathophysiology and identifying correspondences with mouse models for AD preclinical studies

    INVESTIGATING THE PRINCIPLES OF CHEMICAL MODULATION ON ION CHANNELS

    No full text
    Modulation of ion channels by small molecule drugs or chemicals provides both opportunities and challenges in the drug development process. The molecular interaction between a channel protein and a drug is the basis for both mediating the therapeutic effects on the intended targets; and on the other hand adverse drug reactions due to unintended off-target effects. My thesis research is focused on investigating the basic principles of chemical modulation of ion channels and the physiological consequences of these modulations in native systems such as patient specific derived cardiomyocytes. In my first thesis project, I explored whether small molecule potassium channel activators could play a role in normalizing physiological characteristics of cardiomyocytes from Long QT syndrome (LQTS) patients. LQTS is a genetic disease characterized by a prolonged QT interval in electrocardiogram (ECG) and induced by the reduction of depolarization capacity. I applied quantitative Hodgkin-Huxley modeling analyses to test the effectiveness of different normalization strategies and the modeling predicted results guided our selection process for compound identification. The identified compound was then validated in recombinant systems and eventually evaluated in patient specific induced pluripotent stem cells (iPSCs) based LQTS disease model. My study suggests that gating-specific modulation of hERG (the human Ether-à-go-go-Related Gene) potassium channels could reverse the disease phenotypes of KCNQ1 mutation induced LQT1 patient derived cardiomyocytes. In my second thesis project, I explored the pharmacological profiles of voltage gated sodium channels (Navs ), which are essential for membrane excitability and are validated therapeutic targets for cardiac arrhythmias, seizure disorders, pain syndromes and neuromuscular diseases. My investigation was specifically focused on molecular mechanism for potential pharmacological promiscuity. In this study, I developed specific mutant channels of Nav1.5 and Nav1.4 to confer persistent calcium permeability, where genetically encoded calcium indicators with different wavelength emission spectra were used for simultaneously reporting of compound activity and site dependence. This study reveals a F1760 residue dependent promiscuity in Nav1.5, Nav1.4 and potentially all the eukaryotic Nav channels. This under appreciated promiscuity of blockade of Navs may deepen our understanding of general molecular mechanisms of drug target promiscuity

    INVESTIGATING THE PRINCIPLES OF CHEMICAL MODULATION ON ION CHANNELS

    No full text
    Modulation of ion channels by small molecule drugs or chemicals provides both opportunities and challenges in the drug development process. The molecular interaction between a channel protein and a drug is the basis for both mediating the therapeutic effects on the intended targets; and on the other hand adverse drug reactions due to unintended off-target effects. My thesis research is focused on investigating the basic principles of chemical modulation of ion channels and the physiological consequences of these modulations in native systems such as patient specific derived cardiomyocytes. In my first thesis project, I explored whether small molecule potassium channel activators could play a role in normalizing physiological characteristics of cardiomyocytes from Long QT syndrome (LQTS) patients. LQTS is a genetic disease characterized by a prolonged QT interval in electrocardiogram (ECG) and induced by the reduction of depolarization capacity. I applied quantitative Hodgkin-Huxley modeling analyses to test the effectiveness of different normalization strategies and the modeling predicted results guided our selection process for compound identification. The identified compound was then validated in recombinant systems and eventually evaluated in patient specific induced pluripotent stem cells (iPSCs) based LQTS disease model. My study suggests that gating-specific modulation of hERG (the human Ether-à-go-go-Related Gene) potassium channels could reverse the disease phenotypes of KCNQ1 mutation induced LQT1 patient derived cardiomyocytes. In my second thesis project, I explored the pharmacological profiles of voltage gated sodium channels (Navs ), which are essential for membrane excitability and are validated therapeutic targets for cardiac arrhythmias, seizure disorders, pain syndromes and neuromuscular diseases. My investigation was specifically focused on molecular mechanism for potential pharmacological promiscuity. In this study, I developed specific mutant channels of Nav1.5 and Nav1.4 to confer persistent calcium permeability, where genetically encoded calcium indicators with different wavelength emission spectra were used for simultaneously reporting of compound activity and site dependence. This study reveals a F1760 residue dependent promiscuity in Nav1.5, Nav1.4 and potentially all the eukaryotic Nav channels. This under appreciated promiscuity of blockade of Navs may deepen our understanding of general molecular mechanisms of drug target promiscuity

    Learning and generalization of one-hidden-layer neural networks, going beyond standard Gaussian data

    Full text link
    This paper analyzes the convergence and generalization of training a one-hidden-layer neural network when the input features follow the Gaussian mixture model consisting of a finite number of Gaussian distributions. Assuming the labels are generated from a teacher model with an unknown ground truth weight, the learning problem is to estimate the underlying teacher model by minimizing a non-convex risk function over a student neural network. With a finite number of training samples, referred to the sample complexity, the iterations are proved to converge linearly to a critical point with guaranteed generalization error. In addition, for the first time, this paper characterizes the impact of the input distributions on the sample complexity and the learning rate

    Numerical Study on the Aeroacoustic Performance of Different Diversion Strategies in the Pantograph Area of High-Speed Trains at 400 km/h

    No full text
    The speed increase in high-speed trains is a critical procedure in the promotion of high-speed railway technology. As an indispensable and complex structure of high-speed trains, the pantograph’s aerodynamic drag and noise is a significant limitation in the speed increase process of high-speed trains. In the present study, the hybrid method of large eddy simulation (LES) and Ffowcs Williams-Hawkings (FW-H) acoustic analogy is applied to analyze the aerodynamic and aeroacoustic performances of pantograph installed in different ways, i.e., sinking platform and fairing. The results of simulation show that the application of pantograph fairing can reduce the aerodynamic drag greatly. In addition, compared with the pantographs installed alone on the train roof, the installation of the sinking platform brings about 2 dBA reduction in sound pressure level (SPL). Meanwhile, the utilization of the pantograph fairing mainly decreases the noise in the frequency band above 1000 Hz and the largest SPL reduction is up to 3 dBA among the monitoring points. Further analysis shows that the influence of different diversion strategies on the spectral characteristics actually attenuates the dominant frequency of the panhead. In the horizontal plane, the noise directivity of the pantograph installed with a fairing is similar to the pantograph installed alone on the train roof

    Optically Controlled Oscillators in an Engineered Bioelectric Tissue

    No full text
    Complex electrical dynamics in excitable tissues occur throughout biology, but the roles of individual ion channels can be difficult to determine due to the complex nonlinear interactions in native tissue. Here, we ask whether we can engineer a tissue capable of basic information storage and processing, where all functional components are known and well understood. We develop a cell line with four transgenic components: two to enable collective propagation of electrical waves and two to enable optical perturbation and optical readout of membrane potential. We pattern the cell growth to define simple cellular ring oscillators that run stably for > 2 h ( ~ 10[superscript 4]  cycles) and that can store data encoded in the direction of electrical circulation. Using patterned optogenetic stimulation, we probe the biophysical attributes of this synthetic excitable tissue in detail, including dispersion relations, curvature-dependent wave front propagation, electrotonic coupling, and boundary effects. We then apply the biophysical characterization to develop an optically reconfigurable bioelectric oscillator. These results demonstrate the feasibility of engineering bioelectric tissues capable of complex information processing with optical input and output.United States. Office of Naval Research (Grant N000141110-549)National Institutes of Health (U.S.) (Grant 1-R01-EB012498- 01 and New Innovator Grant1-DP2-OD007428)Howard Hughes Medical Institut

    Enhancement of Damping Capbility of MnCu Alloy by High Magnetic Field

    No full text
    The directionally solidified MnCuNiFe alloy was prepared under high magnetic field. The microstructure, composition distribution, phase transformation behavior and damping capacity of the alloy were studied by means of metallographic microscope, scanning electron microscope, transmission electron microscope, X-ray diffraction, differential scanning calorimetry, thermal expansion analysis and dynamic mechanical analysis. It is revealed that magnetic field has definite effect on the refinement of dendrite microstructure as well as the enrichment of Ni element, and thus induces the occurrence of martensitic transformation at about 300 K. The preferred (111) orientation modulated by high magnetic field, especially the induced fct1 → fcc martensitic transformation, together with the twin boundary relaxation, ensure that the directionally solidified MnCuNiFe alloy prepared under high magnetic field owns high-damping capacity in a wide-temperature range from 200 K to 320 K

    Determination of oligosaccharides and monosaccharides in Hakka rice wine by precolumn derivation high-performance liquid chromatography

    No full text
    This article presents a precolumn derivatization procedure with 1-phenyl-3-methyl-5-pyrazolone (PMP) reagent to detect oligosaccharides and monosaccharides in Hakka rice wine. The subsequent separation of the derivatized glucose–PMP also was performed using a mobile phase consisting of the molar ratio of acetonitrile to ammonium acetate buffer (0.1M) of 22:78 at a flow rate of 1.0 mL/min with the column temperature of 35°C, and the pH of ammonium acetate buffer at 5.5. The optimum derivation conditions were as follows: reaction temperature, 70°C; reaction time, 30 minutes; molar ratio of PMP to glucose, 10:1 (v/v); molar ratio of sodium hydroxide to glucose, 3:1 (v/v). The recovery rates were between 93.13% and 102.08% with relative standard deviation of 0.96–2.48%. The established method provides sufficient sensitivity with values of limit of detection of 0.09–0.26 mg/L and limit of quantification of 0.27–0.87 mg/L for determination of oligosaccharides and monosaccharides
    corecore