9,350 research outputs found
Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics
Value-based reinforcement-learning algorithms provide state-of-the-art
results in model-free discrete-action settings, and tend to outperform
actor-critic algorithms. We argue that actor-critic algorithms are limited by
their need for an on-policy critic. We propose Bootstrapped Dual Policy
Iteration (BDPI), a novel model-free reinforcement-learning algorithm for
continuous states and discrete actions, with an actor and several off-policy
critics. Off-policy critics are compatible with experience replay, ensuring
high sample-efficiency, without the need for off-policy corrections. The actor,
by slowly imitating the average greedy policy of the critics, leads to
high-quality and state-specific exploration, which we compare to Thompson
sampling. Because the actor and critics are fully decoupled, BDPI is remarkably
stable, and unusually robust to its hyper-parameters. BDPI is significantly
more sample-efficient than Bootstrapped DQN, PPO, and ACKTR, on discrete,
continuous and pixel-based tasks. Source code:
https://github.com/vub-ai-lab/bdpi.Comment: Accepted at the European Conference on Machine Learning 2019 (ECML
Physical activity, smoking, and the incidence of clinically diagnosed insomnia
OBJECTIVE: This study was designed to examine the independent and combined associations of physical activity and smoking on the incidence of doctor-diagnosed insomnia using a nationally representative sample over seven years, taking into account other relevant covariates. METHODS: Participants aged 18 years or older in the 2005 Taiwan National Health Interview Survey (NHIS) with links to National Health Insurance (NHI) claim data between 2005 and 2012 and without diagnosed insomnia before 2005, were selected into this study (n = 12,728). Participants were classified as having insomnia with International Classification of Diseases, Ninth Revision (ICD-9) CM codes 307.41, 307.42, or 780.52. Self-reported smoking status and frequency, duration, and types of leisure-time and non−leisure-time physical activities were collected. Metabolic equivalent (MET) intensity levels for each activity were assigned, and weekly energy expenditure of each activity was calculated and summed. RESULTS: Inactive participants had a higher risk of incident insomnia [hazard ratio (HR) = 1.22, 95% confidence interval (CI) = 1.06–1.42, p = 0.007] than the active group, and ever-smokers were more likely to have incident insomnia than never smokers (HR = 1.45, 95% CI = 1.20–1.76, p < 0.001). Compared with the nonsmoker/active group, the ever-smoker/inactive group had a higher risk of incident insomnia (HR = 1.78, 95% CI = 1.41–2.25, p < 0.001). Sensitivity analyses excluding individuals diagnosed with other sleep disorders or mental disorders yielded similar results, with the ever-smoker/inactive group having the highest risk of insomnia. CONCLUSIONS: Inactive adults and smokers are at higher risk for incident insomnia, highlighting the importance of a healthy lifestyle and pointing to strategies such as encouraging smoking cessation and physical activity to avoid insomnia among adults
Prospective association between late-life physical activity and hospital care utilisation: a 7-year nationwide follow-up study
BACKGROUND: It is still equivocal whether there is a potential role of late-life physical activity in ameliorating the challenges of increasing healthcare expenditure due to the consequence of global population ageing. OBJECTIVE: this study aimed to examine the prospective association between physical activity and subsequent hospital care utilisation in older adults and to explore the optimal dose of physical activity required to reduce hospital care utilisation. DESIGN: this was a prospective cohort study based on the data from the Taiwan 2005 National Health Interview Survey, which were linked to the 2005-12 claims data from the National Health Insurance system. PARTICIPANTS: 1,760 older adults aged 65 or more. METHODS: the frequency, duration and intensity for physical activity were assessed, and total physical activity energy expenditure was estimated. The average annualised hospital care utilisation for the period 2006 through 2012, including number of hospitalisations, number of days in hospital and the costs of hospitalisation, were calculated. RESULTS: older adults engaging in at least moderate volume of physical activity (≥1,000 kcal/week) experienced fewer subsequent hospital admissions and fewer days in hospital than did sedentary individuals, after adjusting for covariates. Trends for reduced hospitalisation costs were also found. These associations persisted in sensitivity analyses, including tests of reverse causation. CONCLUSION: this study has provided evidence that older adults who are at least moderately active may minimise utilisation of hospital care services. The findings highlight the importance of maintaining a physically active lifestyle in later life
A computational study of astrocytic glutamate influence on post-synaptic neuronal excitability
<p><b>Postsynaptic activity due to synaptic and intrinsic currents</b>, triggered by (a) synaptic glutamate [Glu]<sub>syn</sub> (b-d) simulation with [Glu]<sub>ast,eq</sub> = 1.5mM, 5mM, and 10mM respectively, synaptic currents (I<sub>syn</sub>) combined AMPA- and NMDA-mediated currents in response to synaptic glutamate, membrane potential (V<sub>m</sub>) of postsynaptic neuron resulting from combination of I<sub>syn</sub> and voltage-gated currents (Na<sup>+</sup>, K<sup>+</sup> and leak). Prolonged time course of synaptic glutamate leads to enhanced synaptic currents (I<sub>syn</sub>) and higher frequency postsynaptic firing response (V<sub>m</sub> depolarisations) as [Glu]<sub>ast,eq</sub> increases.</p
A Computational Study of Astrocytic GABA Release at the Glutamatergic Synapse: EAAT-2 and GAT-3 Coupled Dynamics
Neurotransmitter dynamics within neuronal synapses can be controlled by astrocytes and reflect key contributors to neuronal activity. In particular, Glutamate (Glu) released by activated neurons is predominantly removed from the synaptic space by perisynaptic astrocytic transporters EAAT-2 (GLT-1). In previous work, we showed that the time course of Glu transport is affected by ionic concentration gradients either side of the astrocytic membrane and has the propensity for influencing postsynaptic neuronal excitability. Experimental findings co-localize GABA transporters GAT-3 with EAAT-2 on the perisynaptic astrocytic membrane. While these transporters are unlikely to facilitate the uptake of synaptic GABA, this paper presents simulation results which demonstrate the coupling of EAAT-2 and GAT-3, giving rise to the ionic-dependent reversed transport of GAT-3. The resulting efflux of GABA from the astrocyte to the synaptic space reflects an important astrocytic mechanism for modulation of hyperexcitability. Key results also illustrate an astrocytic-mediated modulation of synaptic neuronal excitation by released GABA at the glutamatergic synapse
Most vital segment barriers
We study continuous analogues of "vitality" for discrete network flows/paths,
and consider problems related to placing segment barriers that have highest
impact on a flow/path in a polygonal domain. This extends the graph-theoretic
notion of "most vital arcs" for flows/paths to geometric environments. We give
hardness results and efficient algorithms for various versions of the problem,
(almost) completely separating hard and polynomially-solvable cases
Determinants of Bone and Blood Lead Levels among Minorities Living in the Boston Area
We measured blood and bone lead levels among minority individuals who live in some of Boston’s neighborhoods with high minority representation. Compared with samples of predominantly white subjects we had studied before, the 84 volunteers in this study (33:67 male:female ratio; 31–72 years of age) had similar educational, occupational, and smoking profiles and mean blood, tibia, and patella lead levels (3 μg/dL, 11.9 μg/g, and 14.2 μg/g, respectively) that were also similar. The slopes of the univariate regressions of blood, tibia, and patella lead versus age were 0.10 μg/dL/year (p < 0.001), 0.45 μg/g/year (p < 0.001), and 0.73 μg/g/year (p < 0.001), respectively. Analyses of smoothing curves and regression lines for tibia and patella lead suggested an inflection point at 55 years of age, with slopes for subjects ≥ 55 years of age that were not only steeper than those of younger subjects but also substantially steeper than those observed for individuals > 55 years of age in studies of predominantly white participants. This apparent racial disparity at older ages may be related to differences in historic occupational and/or environmental exposures, or possibly the lower rates of bone turnover that are known to occur in postmenopausal black women. The higher levels of lead accumulation seen in this age group are of concern because such levels have been shown in other studies to predict elevated risks of chronic disease such as hypertension and cognitive dysfunction. Additional research on bone lead levels in minorities and their socioeconomic and racial determinants is needed
Identification and validation of oncologic miRNA biomarkers for Luminal A-like breast cancer
Introduction: Breast cancer is a common disease with distinct tumor subtypes phenotypically characterized by ER and HER2/neu receptor status. MiRNAs play regulatory roles in tumor initiation and progression, and altered miRNA expression has been demonstrated in a variety of cancer states presenting the potential for exploitation as cancer biomarkers. Blood provides an excellent medium for biomarker discovery. This study investigated systemic miRNAs differentially expressed in Luminal A-like (ER+PR+HER2/neu-) breast cancer and their effectiveness as oncologic biomarkers in the clinical setting. Methods: Blood samples were prospectively collected from patients with Luminal A-like breast cancer (n=54) and controls (n=56). RNA was extracted, reverse transcribed and subjected to microarray analysis (n=10 Luminal A-like; n=10 Control). Differentially expressed miRNAs were identified by artificial neural network (ANN) data-mining algorithms. Expression of specific miRNAs was validated by RQ-PCR (n=44 Luminal A; n=46 Control) and potential relationships between circulating miRNA levels and clinicopathological features of breast cancer were investigated. Results: Microarray analysis identified 76 differentially expressed miRNAs. ANN revealed 10 miRNAs for further analysis ( miR-19b, miR-29a, miR-93, miR-181a, miR-182, miR-223, miR-301a, miR-423-5p, miR-486-5 and miR-652 ). The biomarker potential of 4 miRNAs ( miR-29a, miR-181a , miR-223 and miR-652 ) was confirmed by RQ-PCR, with significantly reduced expression in blood of women with Luminal A-like breast tumors compared to healthy controls (p=0.001, 0.004, 0.009 and 0.004 respectively). Binary logistic regression confirmed that combination of 3 of these miRNAs ( miR-29a, miR-181a and miR-652 ) could reliably differentiate between cancers and controls with an AUC of 0.80. Conclusion: This study provides insight into the underlying molecular portrait of Luminal A-like breast cancer subtype. From an initial 76 miRNAs, 4 were validated with altered expression in the blood of women with Luminal A-like breast cancer. The expression profiles of these 3 miRNAs, in combination with mammography, has potential to facilitate accurate subtype- specific breast tumor detection
Electrically controlled long-distance spin transport through an antiferromagnetic insulator
Spintronics uses spins, the intrinsic angular momentum of electrons, as an
alternative for the electron charge. Its long-term goal is in the development
of beyond-Moore low dissipation technology devices. Recent progress
demonstrated the long-distance transport of spin signals across ferromagnetic
insulators. Antiferromagnetically ordered materials are however the most common
class of magnetic materials with several crucial advantages over ferromagnetic
systems. In contrast to the latter, antiferromagnets exhibit no net magnetic
moment, which renders them stable and impervious to external fields. In
addition, they can be operated at THz frequencies. While fundamentally their
properties bode well for spin transport, previous indirect observations
indicate that spin transmission through antiferromagnets is limited to short
distances of a few nanometers. Here we demonstrate the long-distance, over tens
of micrometers, propagation of spin currents through hematite (\alpha-Fe2O3),
the most common antiferromagnetic iron oxide, exploiting the spin Hall effect
for spin injection. We control the spin current flow by the interfacial
spin-bias and by tuning the antiferromagnetic resonance frequency with an
external magnetic field. This simple antiferromagnetic insulator is shown to
convey spin information parallel to the compensated moment (N\'eel order) over
distances exceeding tens of micrometers. This newly-discovered mechanism
transports spin as efficiently as the net magnetic moments in the best-suited
complex ferromagnets. Our results pave the way to ultra-fast, low-power
antiferromagnet-insulator-based spin-logic devices that operate at room
temperature and in the absence of magnetic fields
- …