Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning

Deep reinforcement learning (DRL) breaks through the bottlenecks of traditional reinforcement learning (RL) with the help of the perception capability of deep learning and has been widely applied in real-world problems.While model-free RL, as a class of efficient DRL methods, performs the learning of state representations simultaneously with policy learning in an end-to-end manner when facing large-scale continuous state and action spaces. However, training such a large policy model requires a large number of trajectory samples and training time. On the other hand, the learned policy often fails to generalize to large-scale action spaces, especially for the continuous action spaces. To address this issue, in this paper we propose an efficient policy learning method in latent state and action spaces. More specifically, we extend the idea of state representations to action representations for better policy generalization capability. Meanwhile, we divide the whole learning task into learning with the large-scale representation models in an unsupervised manner and learning with the small-scale policy model in the RL manner.The small policy model facilitates policy learning, while not sacrificing generalization and expressiveness via the large representation model. Finally,the effectiveness of the proposed method is demonstrated by MountainCar,CarRacing and Cheetah experiments

Productivity prediction of fractured horizontal wells with low permeability flow characteristics

Horizontal well and large-scale fracturing are revolutionary technologies in  petroleum industry. The technologies bring obvious economic benefits to exploiting unconventional oil and gas reservoirs with low permeability, ultra-low permeability and shale gas. With the increasingly extensive application of these technologies, other correlated technologies have also gained great development. However, low-permeability reservoirs exhibit complicated features and horizontal well fractures have complex shape. The existing methods for the productivity prediction of fractured horizontal well in low-permeability reservoirs rarely consider the influencing factors in a comprehensive manner. In this paper, a horizontal well seepage model of casing fracturing completion was established according to the superposition principle of low-permeability reservoir and the relationship between potential and pressure, by which model the seepage characteristics of low-permeability reservoirs could be fully described. Based on the established new seepage model, a new targeted model with coupling seepage and wellbore flow was established for the productivity prediction of low-permeability fractured horizontal well. Finally, the new targeted model was verified through field experiment. The experimental results confirmed the reliability of productivity prediction by the proposed model. Sensitivity analysis was then performed on the parameters in the proposed model

Impaired large-scale cortico–hippocampal network connectivity, including the anterior temporal and posterior medial systems, and its associations with cognition in patients with first-episode schizophrenia

Background and objectiveThe cortico–hippocampal network is an emerging neural framework with striking evidence that it supports cognition in humans, especially memory; this network includes the anterior temporal (AT) system, the posterior medial (PM) system, the anterior hippocampus (aHIPPO), and the posterior hippocampus (pHIPPO). This study aimed to detect aberrant patterns of functional connectivity within and between large-scale cortico–hippocampal networks in first-episode schizophrenia patients compared with a healthy control group via resting-state functional magnetic resonance imaging (rs-fMRI) and to explore the correlations of these aberrant patterns with cognition.MethodsA total of 86 first-episode, drug-naïve schizophrenia patients and 102 healthy controls (HC) were recruited to undergo rs-fMRI examinations and clinical evaluations. We conducted large-scale edge-based network analysis to characterize the functional architecture of the cortico–hippocampus network and investigate between-group differences in within/between-network functional connectivity. Additionally, we explored the associations of functional connectivity (FC) abnormalities with clinical characteristics, including scores on the Positive and Negative Syndrome Scale (PANSS) and cognitive scores.ResultsCompared with the HC group, schizophrenia patients exhibited widespread alterations to within-network FC of the cortico–hippocampal network, with decreases in FC involving the precuneus (PREC), amygdala (AMYG), parahippocampal cortex (PHC), orbitofrontal cortex (OFC), perirhinal cortex (PRC), retrosplenial cortex (RSC), posterior cingulate cortex (PCC), angular gyrus (ANG), aHIPPO, and pHIPPO. Schizophrenia patients also showed abnormalities in large-scale between-network FC of the cortico–hippocampal network, in the form of significantly decreased FC between the AT and the PM, the AT and the aHIPPO, the PM and the aHIPPO, and the aHIPPO and the pHIPPO. A number of these signatures of aberrant FC were correlated with PANSS score (positive, negative, and total score) and with scores on cognitive test battery items, including attention/vigilance (AV), working memory (WM), verbal learning and memory (Verb_Lrng), visual learning and memory (Vis_Lrng), reasoning and problem-solving (RPS), and social cognition (SC).ConclusionSchizophrenia patients show distinct patterns of functional integration and separation both within and between large-scale cortico–hippocampal networks, reflecting a network imbalance of the hippocampal long axis with the AT and PM systems, which regulate cognitive domains (mainly Vis_Lrng, Verb_Lrng, WM, and RPS), and particularly involving alterations to FC of the AT system and the aHIPPO. These findings provide new insights into the neurofunctional markers of schizophrenia

Impaired Toll-like receptor 2-mediated Th1 and Th17/22 cytokines secretion in human peripheral blood mononuclear cells from patients with atopic dermatitis

Background: Impaired Toll-like receptor 2 (TLR2) function has been associated with the pathogenesis of atopic dermatitis (AD). However, there are only few studies reporting on the TLR2-induced immunological responses of circulating leucocytes of AD patients. We thus investigated the expression and secretion of Th1, Th2 and Th17/22 cytokines triggered by TLR2 ligands in human peripheral blood mononuclear cells (PBMCs) from AD patients. Expression of TLR2, 1, 6 and high-affinity receptor for IgE (Fc epsilon RI) were further investigated to evaluate the outcome of immune response in AD. Methods: Expression of TLR2, 1, 6 and Fc epsilon RI in PBMCs from AD patients and healthy individuals were measured by qPCR. Subsequent to stimulation with TLR2 ligands PGN and Pam3CSK4, expression and secretion of Th1, Th2 and Th17/22 cytokines were investigated by qPCR and ELISA. Results: The levels of TLR2, 1, 6 mRNA were not altered in both groups of subjects while that of Fc epsilon RI was increased in AD patients. Subsequent to the activation by TLR2 ligands, PBMCs from AD patients significantly released less IFN-gamma, IL-17F and IL-22 than those from healthy controls while no detectable level of release was observed with the other cytokines. In contrast, significantly higher levels of mRNA expression for TNF-alpha, IL5, IL-17A and IL-22 were observed in TLR2 activated PBMCs of AD patients than those of healthy control. Conclusions: PBMCs from AD patients are defective in the secretion of Th1 and Th17/22 cytokines in response to TLR2 ligands. The inconsistent increased expression of the mRNA for the corresponding Th1 cytokines and the Th2 cytokines IL-5 suggested that there may be alterations of downstream signaling events in the cytokine release mechanisms of PBMCs that are associated with the development of AD.National Natural Science Foundation of China [81401299, 81271755]; Natural Science Foundation of Guangdong Province [2014A030313711]; Shenzhen Research Grant [JCYJ20130329110752139, JCYJ20130329110752142]SCI(E)[email protected]; [email protected]

Second-line therapy for patients with steroid-refractory aGVHD: systematic review and meta-analysis of randomized controlled trials

ObjectiveSteroids-refractory (SR) acute graft-versus-host disease (aGVHD) is a life-threatening condition in patients undergoing allogeneic hematopoietic stem cell transplantation (allo-HSCT), but the optimal second-line therapy still has not been established. We aimed to perform a systematic review and meta-analysis of randomized controlled trials (RCTs) to compare the efficacy and safety of different second-line therapy regimens.MethodsLiterature search in MEDLINE, Embase, Cochrane Library and China Biology Medicine databases were performed to retrieve RCTs comparing the efficacy and safety of different therapy regimens for patients with SR aGVHD. Meta-analysis was conducted with Review Manager version 5.3. The primary outcome is the overall response rate (ORR) at day 28. Pooled relative risk (RR) and 95% confidence interval (CI) were calculated with the Mantel-Haenszel method.ResultsEight eligible RCTs were included, involving 1127 patients with SR aGVHD and a broad range of second-line therapy regimens. Meta-analysis of 3 trials investigating the effects of adding mesenchymal stroma cells (MSCs) to other second-line therapy regimens suggested that the addition of MSCs is associated with significantly improvement in ORR at day 28 (RR = 1.15, 95% CI = 1.01–1.32, P = 0.04), especially in patients with severe (grade III–IV or grade C–D) aGVHD (RR = 1.26, 95% CI = 1.04–1.52, P = 0.02) and patients with multiorgan involved (RR = 1.27, 95% CI = 1.05–1.55, P = 0.01). No significant difference was observed betwwen the MSCs group and control group in consideration of overall survival and serious adverse events. Treatment outcomes of the other trials were comprehensively reviewed, ruxolitinib showed significantly higher ORR and complete response rate at day 28, higher durable overall response at day 56 and longer failure-free survival in comparison with other regimens; inolimomab shows similar 1-year therapy success rate but superior long-term overall survial in comparison with anti-thymocyte globulin, other comparisons did not show significant differences in efficacy.ConclusionsAdding MSCs to other second-line therapy regimens is associated with significantly improved ORR, ruxolitinib showed significantly better efficacy outcomes in comparison with other regimens in patients with SR aGVHD. Further well-designed RCTs and integrated studies are required to determine the optimal treatment.Systematic review registrationhttps://www.crd.york.ac.uk/PROSPERO/, identifier CRD42022342487

Impacts of climate change and fruit tree expansion on key hydrological components at different spatial scales

Assessing how fruit tree expansion and climate variability affect hydrological components (e.g., water yield, surface runoff, underground runoff, soil water, evapotranspiration, and infiltration) at different spatial scales is crucial for the management and protection of watersheds, ecosystems, and engineering design. The Jiujushui watershed (259.32 km2), which experienced drastic forest changes over the past decades, was selected to explore the response mechanisms of hydrological components to fruit tree expansion and climate variability at different spatial scales (whole basin and subbasin scale). Specifically, we set up two change scenarios (average temperature increase of 0.5°C and fruit tree area expansion of 18.97%) in the SWAT model by analyzing historical data (1961∼2011). Results showed that climate change reduced water yield, surface runoff, and underground runoff by 6.75, 0.37, and 5.91 mm, respectively. By contrast, the expansion of fruit trees increased surface runoff and water yield by 2.81 and 4.10 mm, respectively, but decreased underground runoff by 1 mm. Interestingly, the sub-basins showed different intensities and directions of response under climate change and fruit tree expansion scenarios. However, the downstream response was overall more robust than the upstream response. These results suggest that there may be significant differences in the hydrological effects of climate change and fruit tree expansion at different spatial scales, thus any land disturbance measures should be carefully considered

PIP5K1α promotes myogenic differentiation via AKT activation and calcium release

Abstract Background Skeletal muscle satellite cell-derived myoblasts are mainly responsible for postnatal muscle growth and injury-induced regeneration. Many intracellular signaling pathways are essential for myogenic differentiation, while a number of kinases are involved in this modulation process. Type I phosphatidylinositol 4-phosphate 5-kinase (PIP5KI) was identified as one of the key kinases involved in myogenic differentiation, but the underlying molecular mechanism is still unclear. Methods PIP5K1α was quantified by quantitative reverse transcriptase PCR and western blot assay. Expression levels of myogenin and myosin heavy chain, which showed significant downregulation in PIP5K1α siRNA-mediated knockdown cells in western blot analysis, were confirmed by immunostaining. Phosphatidylinositol 4,5-bisphosphate in PIP5K1α siRNA-mediated knockdown cells was also measured by the PI(4,5)P2 Mass ELISA Kit. C2C12 cells were overexpressed with different forms of AKT, followed by western blot analysis on myogenin and myosin heavy chain, which reveals their function in myogenic differentiation. FLIPR assays are used to test the release of calcium in PIP5K1α siRNA-mediated knockdown cells after histamine or bradykinin treatment. Statistical significances between groups were determined by two-tailed Student’s t test. Results Since PIP5K1α was the major form in skeletal muscle, knockdown of PIP5K1α consistently inhibited myogenic differentiation while overexpression of PIP5K1α promoted differentiation and rescued the inhibitory effect of the siRNA. PIP5K1α was found to be required for AKT activation and calcium release, both of which were important for skeletal muscle differentiation. Conclusions Taken together, these results suggest that PIP5K1α is an important regulator in myoblast differentiation