Finite Horizon Q-learning: Stability, Convergence, Simulations and an application on Smart Grids

Q-learning is a popular reinforcement learning algorithm. This algorithm has however been studied and analysed mainly in the infinite horizon setting. There are several important applications which can be modeled in the framework of finite horizon Markov decision processes. We develop a version of Q-learning algorithm for finite horizon Markov decision processes (MDP) and provide a full proof of its stability and convergence. Our analysis of stability and convergence of finite horizon Q-learning is based entirely on the ordinary differential equations (O.D.E) method. We also demonstrate the performance of our algorithm on a setting of random MDP as well as on an application on smart grids

Antitubercular therapy decreases nitric oxide production in HIV/TB coinfected patients

BACKGROUND: Nitric oxide (NO) production is increased among patients with human immunodeficiency virus (HIV) infection and also among those with tuberculosis (TB). In this study we sought to determine if there was increased NO production among patients with HIV/TB coinfection and the effect of four weeks chemotherapy on this level. METHODS: 19 patients with HIV/TB coinfection were studied. They were treated with standard four drug antitubercular therapy and sampled at baseline and four weeks. 20 patients with HIV infection, but no opportunistic infections, were disease controls and 20 individuals were healthy controls. Nitrite and citrulline, surrogate markers for NO, were measured spectrophotometrically. RESULTS: The mean age of HIV/TB patients was 28.4 ± 6.8 years and CD4 count was 116 ± 36.6/mm. Mean nitrite level among HIV/TB coinfected was 207.6 ± 48.8 nmol/ml. This was significantly higher than 99.7 ± 26.5 nmol/ml, the value for HIV infected without opportunistic infections and 46.4 ± 16.2 nmol/ml, the value for healthy controls (p value < 0.01). The level of HIV/TB coinfected NO in patients declined to 144.5 ± 34.4 nmol/ml at four weeks of therapy (p value < 0.05). Mean citrulline among HIV/TB coinfected was 1446.8 ± 468.8 nmol/ml. This was significantly higher than 880.8 ± 434.8 nmol/ml, the value for HIV infected without opportunistic infections and 486.6 ± 212.5 nmol/ml, the value for healthy controls (p value < 0.01). Levels of citrolline in HIV/TB infected declined to 1116.2 ± 388.6 nmol/ml at four weeks of therapy (p value < 0.05). CONCLUSIONS: NO production is elevated among patients with HIV infection, especially so among HIV/TB coinfected patients, but declines significantly following 4 weeks of antitubercular therapy

Differential Specificity of Endocrine FGF19 and FGF21 to FGFR1 and FGFR4 in Complex with KLB

Background: Recent studies suggest that betaKlotho (KLB) and endocrine FGF19 and FGF21 redirect FGFR signaling to regulation of metabolic homeostasis and suppression of obesity and diabetes. However, the identity of the predominant metabolic tissue in which a major FGFR-KLB resides that critically mediates the differential actions and metabolism effects of FGF19 and FGF21 remain unclear. Methodology/Principal Findings: We determined the receptor and tissue specificity of FGF21 in comparison to FGF19 by using direct, sensitive and quantitative binding kinetics, and downstream signal transduction and expression of early response gene upon administration of FGF19 and FGF21 in mice. We found that FGF21 binds FGFR1 with much higher affinity than FGFR4 in presence of KLB; while FGF19 binds both FGFR1 and FGFR4 in presence of KLB with comparable affinity. The interaction of FGF21 with FGFR4-KLB is very weak even at high concentration and could be negligible at physiological concentration. Both FGF19 and FGF21 but not FGF1 exhibit binding affinity to KLB. The binding of FGF1 is dependent on where FGFRs are present. Both FGF19 and FGF21 are unable to displace the FGF1 binding, and conversely FGF1 cannot displace FGF19 and FGF21 binding. These results indicate that KLB is an indispensable mediator for the binding of FGF19 and FGF21 to FGFRs that is not required for FGF1. Although FGF19 can predominantly activate the responses of the liver and to a less extent the adipose tissue, FGF21 can do so significantly only in the adipose tissue an

Scintillation light detection in the 6-m drift-length ProtoDUNE Dual Phase liquid argon TPC

DUNE is a dual-site experiment for long-baseline neutrino oscillation studies, neutrino astrophysics and nucleon decay searches. ProtoDUNE Dual Phase (DP) is a 6  ×  6  ×  6 m 3 liquid argon time-projection-chamber (LArTPC) that recorded cosmic-muon data at the CERN Neutrino Platform in 2019-2020 as a prototype of the DUNE Far Detector. Charged particles propagating through the LArTPC produce ionization and scintillation light. The scintillation light signal in these detectors can provide the trigger for non-beam events. In addition, it adds precise timing capabilities and improves the calorimetry measurements. In ProtoDUNE-DP, scintillation and electroluminescence light produced by cosmic muons in the LArTPC is collected by photomultiplier tubes placed up to 7 m away from the ionizing track. In this paper, the ProtoDUNE-DP photon detection system performance is evaluated with a particular focus on the different wavelength shifters, such as PEN and TPB, and the use of Xe-doped LAr, considering its future use in giant LArTPCs. The scintillation light production and propagation processes are analyzed and a comparison of simulation to data is performed, improving understanding of the liquid argon properties

The DUNE Far Detector Interim Design Report, Volume 3: Dual-Phase Module

The DUNE IDR describes the proposed physics program and technical designs of the DUNE far detector modules in preparation for the full TDR to be published in 2019. It is intended as an intermediate milestone on the path to a full TDR, justifying the technical choices that flow down from the high-level physics goals through requirements at all levels of the Project. These design choices will enable the DUNE experiment to make the ground-breaking discoveries that will help to answer fundamental physics questions. Volume 3 describes the dual-phase module's subsystems, the technical coordination required for its design, construction, installation, and integration, and its organizational structure

The DUNE Far Detector Interim Design Report, Volume 2: Single-Phase Module

The DUNE IDR describes the proposed physics program and technical designs of the DUNE far detector modules in preparation for the full TDR to be published in 2019. It is intended as an intermediate milestone on the path to a full TDR, justifying the technical choices that flow down from the high-level physics goals through requirements at all levels of the Project. These design choices will enable the DUNE experiment to make the ground-breaking discoveries that will help to answer fundamental physics questions. Volume 2 describes the single-phase module's subsystems, the technical coordination required for its design, construction, installation, and integration, and its organizational structure