Mitigating Off-Policy Bias in Actor-Critic Methods with One-Step Q-learning: A Novel Correction Approach

Compared to on-policy counterparts, off-policy model-free deep reinforcement learning can improve data efficiency by repeatedly using the previously gathered data. However, off-policy learning becomes challenging when the discrepancy between the underlying distributions of the agent's policy and collected data increases. Although the well-studied importance sampling and off-policy policy gradient techniques were proposed to compensate for this discrepancy, they usually require a collection of long trajectories and induce additional problems such as vanishing/exploding gradients or discarding many useful experiences, which eventually increases the computational complexity. Moreover, their generalization to either continuous action domains or policies approximated by deterministic deep neural networks is strictly limited. To overcome these limitations, we introduce a novel policy similarity measure to mitigate the effects of such discrepancy in continuous control. Our method offers an adequate single-step off-policy correction that is applicable to deterministic policy networks. Theoretical and empirical studies demonstrate that it can achieve a "safe" off-policy learning and substantially improve the state-of-the-art by attaining higher returns in fewer steps than the competing methods through an effective schedule of the learning rate in Q-learning and policy optimization

Hemoglobin is inversely related to flow-mediated dilatation in chronic kidney disease

The microcirculation is regulated by oxygen gradients and by endothelial release of nitric oxide, which can react with hemoglobin to form S-nitroso derivatives. Here we induced flow-mediated dilatation of the brachial artery in response to ischemia in 141 non-diabetic patients with stage 3–4 chronic kidney disease who had no history of smoking, cardiovascular events or use of erythropoietin-based agents. Patients with hemoglobin concentrations above the cohort median of 11.6 g/dl were found to have significant reductions in flow-mediated dilatation compared to those below the median. This inverse relationship remained significant after adjustment for potential confounders, including insulin sensitivity, glomerular filtration rate, proteinuria, body mass index, serum urate, etiology of underlying renal disease, treatment with anti-hypertensive drugs, and traditional Framingham risk factors. Given that hemoglobin can act as an important nitric oxide carrier and buffer, our studies suggest that the mechanism by which hemoglobin influences the endothelium-dependent microcirculation requires its nitrosylation; however, more direct studies need to be performed

FGF-23 and vascular dysfunction in patients with stage 3 and 4 chronic kidney disease

Studies in animals show that fibroblast growth factor (FGF)-23 interferes with vascular reactivity induced by the nitric oxide (NO) system. To investigate the relationship between circulating FGF-23 levels and the response of forearm blood flow to ischemia (flow-mediated vasodilatation, FMD) and nitroglycerin, we tested 183 patients with stage 3-4 chronic kidney disease (CKD). None of them had cardiovascular complications or were taking drugs interfering with vascular function. Patients with FGF-23 levels above the median had significantly lower glomerular filtration rate, FMD, and fetuin-A levels (an anti-inflammatory molecule and potent inhibitor of calcification). They also had higher proteinuria and phosphate levels when compared to patients whose FGF-23 levels were below the median. The response to nitroglycerin was not different between the two groups. Multiple regression analysis showed that the relationship between FGF-23 and FMD was only modestly sensitive to adjustment for classical risk factors, biomarkers of bone mineral metabolism, high-sensitivity C-reactive protein, and homeostatic model assessment index. Adjustment for asymmetrical dimethyl arginine (ADMA) weakened the strength of this link; however, it remained highly significant. There was no independent association between FGF-23 and nitroglycerin. Thus, attenuation of FMD by ADMA suggests that this endogenous inhibitor of NO synthase may, in part, mediate the vascular effects of FGF-23 in patients with CKD. © 2010 International Society of Nephrology

Endothelial function in patients with familial Mediterranean fever-related amyloidosis and association with cardiovascular events

Objectives. Secondary amyloidosis is the most important complication of FMF and endothelial function is more severely impaired. Elevated asymmetric dimethyl arginine (ADMA) may mediate the excess cardiovascular disease (CVD) risk of this group. We aimed to compare endothelial function characteristics, including ADMA, in patients with FMF-related amyloidosis and primary glomerulopathies and to define risk factors for a CVD event. Methods. We undertook a cross-sectional study with prospective follow-up including consecutive patients with FMF-related amyloidosis (n = 98) or other non-diabetic glomerulopathies (n = 102). All patients had nephrotic-range proteinuria and normal glomerular filtration rate. Flow-mediated dilatation (FMD) was assessedand ADMA levels, CRP and pentraxin 3 (PTX3) were determined. Patients were followed for cardiovascular events. Results. Amyloidosis patients secondary to FMF showed higher levels of ADMA, CRP and PTX3 and lower FMD as compared with patients with other glomerulopathies. Cardiovascular events (n = 54) were registered during 3 years of follow-up. Increased ADMA levels and lower FMD were observed in patients with cardiovascular risk in both groups, but especially in individuals with amyloidosis.Conclusion. Patients with FMF-related amyloidosis have increased CVD event risk, probably related to the high ADMA levels, elevated inflammatory markers and decreased FMD measures observed in these patients

Štetni učinci pušenja tijekom trudnoće na DNA i razine reaktivnih oblika kisika (ROS) u krvi majke i novorođenčeta

Some of the genotoxic/carcinogenic substances or metabolites in cigarette smoke are capable of passing through the placenta and harming a newborn’s health. Smoking is also known as a factor in the formation of oxidative damage and the main mechanism involved in the carcinogenic process. Predetermining this genotoxic risk can be successfully achieved by measuring certain parameters of oxidative stress. The comet assay is considered an important biomarker for the evaluation of genotoxic substances and is effective for detecting DNA damage caused by smoking. This study examined third trimester bloods and the cord blood of 28 actively smoking and 22 non-smoking mothers in terms of DNA damage and oxidative stress parameters. Cu/Zn superoxide dismutase (CuZn-SOD), malondialdehyde (MDA), catalase (CAT), plasma nitrite/nitrates (NO2•/NO3•), selenium-dependent glutathione peroxidase (Se-GPx), Cu, and Zn levels were measured as indicators of oxidative damage. There were no significant increases in DNA damage of the actively smoking pregnant group in comparison with the non-smoking pregnant group, either in the third trimester or cord blood. Oxidative stress parameters of smoker and non-smoker groups were statistically different for MDA (p<0.05), CuZn-SOD (p<0.01), Se-GPx (p<0.05) values while the difference was not significant for NO2•/NO3•, CAT, Zn, and Cu values. The same values were also investigated in cord blood, and only NO2•/NO3• (p<0.01), Se-GPx (p<0.01 and CAT (p<0.001) values were found statistically different. Smoking mothers may have been exposed to more oxidative stress than non-smoking mothers.Pojedine genotoksične/kancerogene supstancije ili metaboliti u cigaretnom dimu mogu proći kroz posteljicu i naštetiti zdravlju novorođenčeta. Pušenje je također poznato kao čimbenik pri nastanku oksidacijskog oštećenja DNA i u procesu kancerogeneze. Ovaj genotoksični rizik može se uspješno odrediti mjerenjem određenih parametara oksidacijskog stresa. Komet-test smatra se važnim biološkim biljegom pri evaluaciji genotoksičnih supstancija i iznimno učinkovitim sredstvom pronalaženja oštećenja DNA uzrokovanih pušenjem. Ova studija proučava krv trudnica u trećem tromjesečju trudnoće i fetalnu krv 28 majki aktivnih pušačica te 22-ju majki nepušačica vezano za oksidacijska oštećenja DNA i parametre oksidacijskog stresa. Razine Cu/Zn superoksidne dismutaze (CuZn-SOD), malondialdehida (MDA), katalaze (CAT), nitrita/nitrata u plazmi (NO2-/NO3-), selenijeve glutation peroksidaze (Se-GPx), Cu i Zn mjerene su kao pokazatelji oksidacijskog oštećenja. Nije bilo značajnih povećanja oštećenja DNA u skupini trudnica aktivnih pušačica u usporedbi sa skupinom trudnica nepušačica, ni u krvi iz trećeg tromjesečja ni u fetalnoj krvi. Parametri oksidacijskog stresa pušačke i nepušačke skupine bili su statistički različiti za vrijednosti MDA (p<0,05), CuZn-SOD (p<0,01), Se-GPx (P<0,05), dok razlika nije bila značajna za vrijednosti NO2-/NO3-, CAT, Zn i Cu. Iste su vrijednosti ispitane i u fetalnoj krvi, a jedino su vrijednosti NO2-/NO3- (p<0,01), Se-GPx (p<0,01) i CAT (p<0,001) bile statistički različite. Vjerojatno je da su majke pušačice bile izložene većem oksidacijskom stresu od majki nepušačica

Traits associated with winter wheat grain yield in Central and West Asia

WOS: 000339112800006PubMed ID: 24456121Improved adaptation of winter wheat (Triticum aestivum L.) to drought and heat may be influenced by days to heading, plant height, biomass, canopy temperature (CT) at grain filling, and rate of senescence. This study shows that, under supplemental irrigation or rainfed conditions, days to heading and plant height together explain up to 68% of grain yield (GY) variation, and these associations were further confirmed in several locations across West and Central Asia. Days to heading can be slightly reduced below that of check line Karahan to further improve GY while avoiding the effect of late frosts. Plant height has been decreased in recent germplasm, but further reductions below that of check line Karahan could still improve GY in a wide range of environments. However, in Iranian sites, taller genotypes showed better adaptation with higher biomass and increased reserves for grain filling. Canopy temperature and rate senescence were not associated with GY. A normalized difference vegetation index, used to estimate biomass (Feekes stages 4-5), had intermediate heritability across environments and correlated positively with GY under low plant density and should be explored further as a tool for early selection.Mexican Ministry of Agriculture, Livestock, Rural Development, Fisheries and Food (SAGARPA) via MasAgro Seeds of DiscoveryWe would like to thank Professor Dr Sultan Gobanoglu, University of Ankara, Agricultural Faculty, Plant Protection Department, for establishing contact with the Turkish National Meteorological Institute, Ankara, and sharing meteorological data. Drs Alex Morgounov, Beyhan Akin, Mesut Kesser, and Nurberdy Gummadov are also acknowledged for sharing GY data for trials conducted in 2011-2012 in Konya (Bahri Dagdas Uluslararasi Tarimsal Arastirma Institute) and Eskisehir (Eskisehir Anadolu Tarimsal Arastirma Institute). The Mexican Ministry of Agriculture, Livestock, Rural Development, Fisheries and Food (SAGARPA) via MasAgro Seeds of Discovery is acknowledged for financial support

The Evaluation of Individuals who applied to Family Medicine Center for Medical Marriage Report

Marriage is a spiritual and physical union of two people, supposed to last for a lifetime. According to Turkish Civil Law article 136, couples are obliged by municipal authorities to get a medical report in order to get permission to marry. In this context, although it varies among regions, some biochemical evaluations are required from couples at the time of admission. This study evaluates the laboratory parameters of the couples who admitted to outpatient clinic of Family Medicine at Istanbul Medeniyet University Goztepe Training and Research Hospital between years 2012-2013 in order to get medical report. Among 290 applicants evaluated 148 were women. There was no clinical or laboratory finding for VDRL, AntiHCV or radiological tuberculosis positivity. Two applicants had positivity for AntiHIV and 7 applicants had positive HBsAg. In accordance with these findings, the importance of giving counseling is obvious rather than only evaluating the health situations of these couples. It is obvious that efficient counseling provides the generation of a healthy population by helping the couples avoid genetically or sexually transmitted diseases. Therefore 'to protect the unborn offspring and achieve a healthier population' is both the responsibility and the 'chance' of the primary care system and the physician