Reinforcement Learning Experience Reuse with Policy Residual Representation

Experience reuse is key to sample-efficient reinforcement learning. One of the critical issues is how the experience is represented and stored. Previously, the experience can be stored in the forms of features, individual models, and the average model, each lying at a different granularity. However, new tasks may require experience across multiple granularities. In this paper, we propose the policy residual representation (PRR) network, which can extract and store multiple levels of experience. PRR network is trained on a set of tasks with a multi-level architecture, where a module in each level corresponds to a subset of the tasks. Therefore, the PRR network represents the experience in a spectrum-like way. When training on a new task, PRR can provide different levels of experience for accelerating the learning. We experiment with the PRR network on a set of grid world navigation tasks, locomotion tasks, and fighting tasks in a video game. The results show that the PRR network leads to better reuse of experience and thus outperforms some state-of-the-art approaches.Comment: Conference version appears in IJCAI 201

Tris(1,10-phenanthroline-κ2 N,N′)zinc(II) chloride 2-phenyl-4-selenazole-5-car­box­yl­ate decahydrate

The asymmetric unit of the title salt, [Zn(C12H8N2)3](C10H6NO2Se)Cl·10H2O, contains a [Zn(phen)3]2+ cation (phen is 1,10-phenanthroline), uncoordinated chloride and 2-phenyl-4-selenazole-5-carboxyl­ate anions and ten uncoord­in­ated water mol­ecules. The central ZnII ion is six-coordinated by six N atoms from three phen ligands in a distorted octa­hedral geometry. An extensive O—H⋯O, O—H⋯N and O—H⋯Cl hydrogen-bonding network stabilizes the crystal structure

2-Phenyl-1,3-selenazole-4-carb­oxy­lic acid

In the title compound, C10H7NO2Se, the two rings are twisted, making a dihedral angle of 12.42 (9)°. In the crystal, pairs of mol­ecules are disposed about an inversion center, generating O—H⋯O hydrogen-bonded dimers

Protectin conjugates in tissue regeneration 1 alleviates sepsis-induced acute lung injury by inhibiting ferroptosis

Background: Acute lung injury (ALI) is a common and serious complication of sepsis with high mortality. Ferroptosis, categorized as programmed cell death, contributes to the development of lung injury. Protectin conjugates in tissue regeneration 1 (PCTR1) is an endogenous lipid mediator that exerts protective effects against multiorgan injury. However, the role of PCTR1 in the ferroptosis of sepsis-related ALI remains unknown. Methods: A pulmonary epithelial cell line and a mouse model of ALI stimulated with lipopolysaccharide (LPS) were established in vitro and in vivo. Ferroptosis biomarkers, including ferrous (Fe2+), glutathione (GSH), malondialdehyde (MDA) and 4-Hydroxynonenal (4-HNE), were assessed by relevant assay kits. Glutathione peroxidase 4 (GPX4) and prostaglandin-endoperoxide synthase 2 (PTGS2) protein levels were determined by western blotting. Lipid peroxides were examined by fluorescence microscopy and flow cytometry. Cell viability was determined by a CCK-8 assay kit. The ultrastructure of mitochondria was observed with transmission electron microscopy. Morphology and inflammatory cytokine levels predicted the severity of lung injury. Afterward, related inhibitors were used to explore the potential mechanism by which PCTR1 regulates ferroptosis. Results: PCTR1 treatment protected mice from LPS-induced lung injury, which was consistent with the effect of the ferroptosis inhibitor ferrostatin-1. PCTR1 treatment decreased Fe2+, PTGS2 and lipid reactive oxygen species (ROS) contents, increased GSH and GPX4 levels and ameliorated mitochondrial ultrastructural injury. Administration of LPS or the ferroptosis agonist RSL3 resulted in reduced cell viability, which was rescued by PCTR1. Mechanistically, inhibition of the PCTR1 receptor lipoxin A4 (ALX), protein kinase A (PKA) and transcription factor cAMP-response element binding protein (CREB) partly decreased PCTR1 upregulated GPX4 expression and a CREB inhibitor blocked the effects ofPCTR1 on ferroptosis inhibition and lung protection. Conclusion: This study suggests that PCTR1 suppresses LPS-induced ferroptosis via the ALX/PKA/CREB signaling pathway, which may offer promising therapeutic prospects in sepsis-related ALI

Potential Nexus of Non-alcoholic Fatty Liver Disease and Type 2 Diabetes Mellitus: Insulin Resistance Between Hepatic and Peripheral Tissues

The liver is the central metabolic organ and plays a pivotal role in regulating homeostasis of glucose and lipid metabolism. Aberrant liver metabolism promotes insulin resistance, which is reported to be a common characteristic of metabolic diseases such as non-alcoholic fatty liver disease (NAFLD) and type 2 diabetes mellitus (T2DM). There is a complex and bidirectional relationship between NAFLD and T2DM. NAFLD patients with hepatic insulin resistance generally share a high risk of impaired fasting glucose associated with early diabetes; most patients with T2DM experience non-alcoholic fatty liver (NAFL), non-alcoholic steatohepatitis (NASH), and other more severe liver complications such as cirrhosis and hepatocellular carcinoma (HCC). Additionally, hepatic insulin resistance, which is caused by diacylglycerol-mediated activation of protein kinase C epsilon (PKC), may be the critical pathological link between NAFLD and T2DM. Therefore, this review aims to illuminate current insights regarding the complex and strong association between NAFLD and T2DM and summarize novel and emerging targets for the treatment of hepatic insulin resistance based on established mechanistic knowledge

Cross-sectional study of the relationship of peripheral blood cell profiles with severity of infection by adenovirus type 55

BACKGROUND: The immunologic profiles of patients with human adenovirus serotype 55 (HAdV-55) infections were characterized in subjects diagnosed with silent infections (n = 30), minor infections (n = 27), severe infections (n = 34), and healthy controls (n = 30) during a recent outbreak among Chinese military trainees. METHODS: Blood was sampled at the disease peak and four weeks later, and samples were analyzed to measure changes in leukocyte and platelet profiles in patients with different severities of disease. Differential lymphocyte subsets and cytokine profiles were measured by flow cytometry and Luminex xMAP®, and serum antibodies were analyzed by ELISA and immunofluorescence staining. RESULTS: Patients with severe HAdV infections had higher proportions of neutrophils and reduced levels of lymphocytes (p < 0.005 for both). Patients with minor and severe infections had significantly lower platelet counts (p < 0.005 for both) than those with silent infections. The silent and minor infection groups had higher levels of dendritic cells than the severe infection group. Relative to patients with silent infections, patients with severe infections had significantly higher levels of IL-17(+)CD4(+) cells, decreased levels of IL-17(+)CD8(+) cells, and higher levels of IFN-γ, IL-4, IL-10, and IFN-α2 (p < 0.001 for all comparisons). CONCLUSIONS: Patients with different severities of disease due to HAdV-55 infection had significantly different immune responses. These data provide an initial step toward the identification of patients at risk for more severe disease and the development of treatments against HAdV-55 infection

Charge order induced Dirac pockets in the nonsymmorphic crystal TaTe4_4

The interplay between charge order (CO) and nontrivial band topology has spurred tremendous interest in understanding topological excitations beyond the single-particle description. In a quasi-one-dimensional nonsymmorphic crystal TaTe$_4$, the (2a$\times$2b$\times$3c) charge ordered ground state drives the system into a space group where the symmetry indicator features the emergence of Dirac fermions and unconventional double Dirac fermions. Using angle-resolved photoemission spectroscopy and first-principles calculations, we provide evidence of the CO induced Dirac fermion-related bands near the Fermi level. Furthermore, the band folding at the Fermi level is compatible with the new periodicity dictated by the CO, indicating that the electrons near the Fermi level follow the crystalline symmetries needed to host double Dirac fermions in this system.Comment: 9 pages, 4 figures. Second version of the manuscript following the first submission in April 202

Microsurgical vasovasostomy for the treatment of intractable chronic scrotal pain after vasectomy

Dear Editor, We present herein two rare cases of intractable chronic scrotal pain after vasectomy. The patients were effectively treated with microsurgical vasovasostomy (MVV). We also discuss the possible aetiologies of the pain and other surgical options. Vasectomy was once the most common method of permanent contraception for men in both China and worldwide. One particularly distressing complication after vasectomy is chronic scrotal pain, which is defined as intermittent or constant, unilateral or bilateral scrotal pain for o3 months. The pain is intense enough to interfere with the patient&apos;s daily activities and prompts him to seek medical attention. 1 Although its aetiology remains unclear, epididymal congestion, painful sperm granulomas, vascular stasis and nerve impingement have been postulated as possible aetiologic factors. 2 Non-surgical options have been used successfully to treat chronic scrotal pain after vasectomy, including scrotal support, thermal therapy, limiting activity, non-steroidal anti-inflammatory drugs, narcotic analgesics, antibiotics, neuroleptics, spermatic cord nerve block, biofeedback and psychiatric evaluation. Surgical options include reversal of the vasectomy, microsurgical spermatic cord denervation, granuloma excision, epididymectomy and orchidectomy. The microsurgical techniques used for vasectomy reversal have changed significantly in the past decade, culminating in the standard surgical procedures used today, and its indications include a desire to have more children (remarriage or after the death of a child), treatment of post-vasectomy pain and treatment of obstructive azoospermia due to traumatic or iatrogenic injury of vas deferens. 3 To our knowledge, we report the first cases of the use of MVV for the treatment of intractable chronic scrotal pain after vasectomy in a Chinese hospital. The 72-year-old and 49-year-old men presented with a more than 20-year history of intractable, chronic scrotal pain after vasectomy. They had consulted various urologists and had undergone numerous attempted therapies in other hospitals. They reported a history of vasectomy more than 30 years and 20 years previously, respectively. They did not have any histories of haematuria, haematospermia, lower urinary tract symptoms, epididymitis, prostatitis or testicular trauma. Their physical examination was unremarkable, and both the secondary sexual characteristics and genital examination were normal. The testes were descended bilaterally and normal in size and consistency. The caput epididymides exhibited dilatation and tenderness. The vasa deferentia were palpated for painful lumps at the vasectomy sites. Digital rectal examination was unremarkable for prostatic abnormalities. Each patient underwent Doppler ultrasonography of the testes and urinary tract, urinalysis, urine culture and spermiogram to exclude primary or secondary causes of pain, including intratesticular infection, tumours and ureteral lithiasis. At our initial consultation, the patients were asked to complete a pain and psychological questionnaire, which included pain, depression and anxiety scores. The pain score (Visual Analogue Scale) was in the form of an 11-point numerical rating score with 0 representing &apos;no pain&apos; and 10 representing the &apos;worst possible pain&apos;. The patients&apos; preoperative pain scores were 5 and 6 points, respectively. The depression scores (Self-rating Depression Scale) were in the form of an 80-point numerical rating score; a score less than 50 indicated &apos;normal&apos;, and a score greater than 50 indicated &apos;depression&apos;. The depression scores of the two patients were 35 and 38 points, respectively. The anxiety scores (Self-rating Anxiety Scale) were in the form of an 80-point numerical rating score; scores less than 50 were considered to indicate &apos;normal&apos;, whereas scores greater than 50 indicated &apos;anxiety&apos;. The anxiety scores of the two patients were 33 and 32 points, respectively. Spermatic cord block was performed once for each patient with 6 ml of 1% lidocaine and 1 ml of methylprednisolone (40 mg). The patients had 3 and 7 days of complete pain relief after the blockade, respectively. The study protocol was approved by the Ethical Committee of the First Affiliated Hospital of Sun Yat-Sen University, and informed consent was signed by the patients. The patients were offered MVV as a more permanent solution in March and July 2012, respectively. Scrotal exploration was performed with the patients under combined spinal-epidural anaesthesia. The left-side incision (3 cm) of the scrotum through the tunica vaginalis was made, and the left vas deferens was delivered through this incision. The painful lumps and nerveimpinging tissue at the vasectomy site were thoroughly resected by electrocautery. Distal patency was confirmed by infusing diluted methylene blue through the abdominal side of the vas deferens, resulting in blue colouring of the urine. A 123 to 153 operating microscope (Leica Microsystems (Schweiz) AG, Heerbrugg, Switzerland) was use

Montmorillonite modified by CNx supported Pt for methanol oxidation

A composite support based on nature clay, i.e. montmorillonite (MMT), shows great promise as support materials for Pt electrocatalyst for the methanol oxidation reaction in fuel cell anodes. The reported composite support (CNx-MMT) was prepared via carbonizing MMT which was covered by N-contented polymer. X-ray diffraction and transmission electron microscopy results showed that Pt nanoparticles can be well-dispersed on the composite support with highly dispersed tiny crystal Pt nanoparticles. Cyclic voltammetry measurements showed that the Pt/CNx-MMT has the enhanced electrocatalytic activity in methanol oxidation reaction. The developed Pt catalyst supported on new composite support is catalytically more active for methanol electrooxidation than Pt supported on the conventional carbon support and shows good stability, offering promising potential for application of MMT as support for fuel cell electrocatalysis.Web of Scienc