3,950 research outputs found
Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics
Value-based reinforcement-learning algorithms provide state-of-the-art
results in model-free discrete-action settings, and tend to outperform
actor-critic algorithms. We argue that actor-critic algorithms are limited by
their need for an on-policy critic. We propose Bootstrapped Dual Policy
Iteration (BDPI), a novel model-free reinforcement-learning algorithm for
continuous states and discrete actions, with an actor and several off-policy
critics. Off-policy critics are compatible with experience replay, ensuring
high sample-efficiency, without the need for off-policy corrections. The actor,
by slowly imitating the average greedy policy of the critics, leads to
high-quality and state-specific exploration, which we compare to Thompson
sampling. Because the actor and critics are fully decoupled, BDPI is remarkably
stable, and unusually robust to its hyper-parameters. BDPI is significantly
more sample-efficient than Bootstrapped DQN, PPO, and ACKTR, on discrete,
continuous and pixel-based tasks. Source code:
https://github.com/vub-ai-lab/bdpi.Comment: Accepted at the European Conference on Machine Learning 2019 (ECML
Can oral infection be a risk factor for Alzheimer’s disease?
Alzheimer’s disease (AD) is a scourge of longevity that will drain enormous resources from public health budgets in the future. Currently, there is no diagnostic biomarker and/or treatment for this most common form of dementia in humans. AD can be of early familial-onset or sporadic with a late-onset. Apart from the two main hallmarks, amyloid-beta and neurofibrillary tangles, inflammation is a characteristic feature of AD neuropathology. Inflammation may be caused by a local central nervous system insult and/or by peripheral infections. Numerous microorganisms are suspected in AD brains ranging from bacteria (mainly oral and non-oral Treponema species), viruses (Herpes simplex type I) and yeasts (Candida species). A causal relationship between periodontal pathogens/non-oral Treponema species of bacteria has been proposed via the amyloid-beta and inflammatory links. Periodontitis constitutes a peripheral oral infection that can provide the brain with intact bacteria and virulence factors and inflammatory mediators due to daily, transient bacteraemias. If and when genetic risk factors meet environmental risk factors in the brain, disease is expressed, in which neurocognition may be impacted, leading to the development of dementia. To achieve the goal of finding a diagnostic biomarker and possible prophylactic treatment for AD, there is an initial need to solve the etiological puzzle contributing to its pathogenesis. This review therefore addresses oral infection as the plausible aetiology of late onset AD (LOAD)
Roles of TRPV1 and neuropeptidergic receptors in dorsal root reflex-mediated neurogenic inflammation induced by intradermal injection of capsaicin
<p>Abstract</p> <p>Background</p> <p>Acute cutaneous neurogenic inflammation initiated by activation of transient receptor potential vanilloid-1 (TRPV<sub>1</sub>) receptors following intradermal injection of capsaicin is mediated mainly by dorsal root reflexes (DRRs). Inflammatory neuropeptides are suggested to be released from primary afferent nociceptors participating in inflammation. However, no direct evidence demonstrates that the release of inflammatory substances is due to the triggering of DRRs and how activation of TRPV<sub>1 </sub>receptors initiates neurogenic inflammation via triggering DRRs.</p> <p>Results</p> <p>Here we used pharmacological manipulations to analyze the roles of TRPV<sub>1 </sub>and neuropeptidergic receptors in the DRR-mediated neurogenic inflammation induced by intradermal injection of capsaicin. The degree of cutaneous inflammation in the hindpaw that followed capsaicin injection was assessed by measurements of local blood flow (vasodilation) and paw-thickness (edema) of the foot skin in anesthetized rats. Local injection of capsaicin, calcitonin gene-related peptide (CGRP) or substance P (SP) resulted in cutaneous vasodilation and edema. Removal of DRRs by either spinal dorsal rhizotomy or intrathecal administration of the GABA<sub>A </sub>receptor antagonist, bicuculline, reduced dramatically the capsaicin-induced vasodilation and edema. In contrast, CGRP- or SP-induced inflammation was not significantly affected after DRR removal. Dose-response analysis of the antagonistic effect of the TRPV<sub>1 </sub>receptor antagonist, capsazepine administered peripherally, shows that the capsaicin-evoked inflammation was inhibited in a dose-dependent manner, and nearly completely abolished by capsazepine at doses between 30–150 μg. In contrast, pretreatment of the periphery with different doses of CGRP<sub>8–37 </sub>(a CGRP receptor antagonist) or spantide I (a neurokinin 1 receptor antagonist) only reduced the inflammation. If both CGRP and NK<sub>1 </sub>receptors were blocked by co-administration of CGRP<sub>8–37 </sub>and spantide I, a stronger reduction in the capsaicin-initiated inflammation was produced.</p> <p>Conclusion</p> <p>Our data suggest that 1) the generation of DRRs is critical for driving the release of neuropeptides antidromically from primary afferent nociceptors; 2) activation of TRPV<sub>1 </sub>receptors in primary afferent nociceptors following intradermal capsaicin injection initiates this process; 3) the released CGRP and SP participate in neurogenic inflammation.</p
Topoisomer Differentiation of Molecular Knots by FTICR MS: Lessons from Class II Lasso Peptides
Lasso peptides constitute a class of bioactive peptides sharing a knotted
structure where the C-terminal tail of the peptide is threaded through and
trapped within an N-terminalmacrolactamring. The structural characterization of
lasso structures and differentiation from their unthreaded topoisomers is not
trivial and generally requires the use of complementary biochemical and
spectroscopic methods. Here we investigated two antimicrobial peptides
belonging to the class II lasso peptide family and their corresponding
unthreaded topoisomers: microcin J25 (MccJ25), which is known to yield
two-peptide product ions specific of the lasso structure under collisioninduced
dissociation (CID), and capistruin, for which CID does not permit to
unambiguously assign the lasso structure. The two pairs of topoisomers were
analyzed by electrospray ionization Fourier transform ion cyclotron resonance
mass spectrometry (ESI-FTICR MS) upon CID, infrared multiple photon
dissociation (IRMPD), and electron capture dissociation (ECD). CID and
ECDspectra clearly permitted to differentiate MccJ25 from its non-lasso
topoisomer MccJ25-Icm, while for capistruin, only ECD was informative and
showed different extent of hydrogen migration (formation of c\bullet/z from
c/z\bullet) for the threaded and unthreaded topoisomers. The ECD spectra of the
triply-charged MccJ25 and MccJ25-lcm showed a series of radical b-type product
ions {\eth}b0In{\TH}. We proposed that these ions are specific of
cyclic-branched peptides and result from a dual c/z\bullet and y/b
dissociation, in the ring and in the tail, respectively. This work shows the
potentiality of ECD for structural characterization of peptide topoisomers, as
well as the effect of conformation on hydrogen migration subsequent to electron
capture
PageRank without hyperlinks: Reranking with PubMed related article networks for biomedical text retrieval
Graph analysis algorithms such as PageRank and HITS have been successful in Web environments because they are able to extract important inter-document relationships from manually-created hyperlinks. We consider the application of these algorithms to related document networks comprised of automatically-generated content-similarity links. Specifically, this work tackles the problem of document retrieval in the biomedical domain, in the context of the PubMed search engine. A series of reranking experiments demonstrate that incorporating evidence extracted from link structure yields significant improvements in terms of standard ranked retrieval metrics. These results extend the applicability of link analysis algorithms to different environments
- …