A Survey on Transformers in Reinforcement Learning

Transformer has been considered the dominating neural architecture in NLP and CV, mostly under supervised settings. Recently, a similar surge of using Transformers has appeared in the domain of reinforcement learning (RL), but it is faced with unique design choices and challenges brought by the nature of RL. However, the evolution of Transformers in RL has not yet been well unraveled. In this paper, we seek to systematically review motivations and progress on using Transformers in RL, provide a taxonomy on existing works, discuss each sub-field, and summarize future prospects

Morphine Suppresses IFN Signaling Pathway and Enhances AIDS Virus Infection

Background: Opioids exert a profound influence on immunomodulation and enhance HIV infection and replication. However, the mechanism(s) of their action remains to be determined. We thus investigated the impact of morphine on the intracellular innate antiviral immunity. Methodology/Principal Findings: Seven-day-cultured macrophages were infected with equal amounts of cell-free HIV Bal or SIV DeltaB670 for 2 h at 37uC after 24 h of treatment with or without morphine. Effect of morphine on HIV/SIV infection and replication was evaluated by HIV/SIV RT activity assay and indirect immunofluorescence for HIV p24 or SIV p28 antigen. The mRNA expression of cellular factors suppressed or induced by morphine treatment was analyzed by the real-time RT-PCR. We demonstrated that morphine treatment of human blood monocyte-derived macrophages significantly inhibited the expression of interferons (IFN-a, IFN-b and IFN-l) and IFN-inducible genes (APOBEC3C/3F/3G and 3H). The further experiments showed that morphine suppressed the expression of several key elements (RIG-I and IRF-7) in IFN signaling pathway. In addition, morphine treatment induced the expression of suppressor of cytokine signaling protein-1, 2, 3 (SOCS-1, 2, 3) and protein inhibitors of activated STAT-1, 3, X, Y (PIAS-1, 3, X, Y), the key negative regulators of IFN signaling pathway. Conclusions: These findings indicate that morphine impairs intracellular innate antiviral mechanism(s) in macrophages

Upregulation of SOCS-3 and PIAS-3 Impairs IL-12-Mediated Interferon-Gamma Response in CD56+ T Cells in HCV-Infected Heroin Users

CD56(+) T cells are abundant in liver and play an important role in host innate immunity against viral infections, including hepatitis C virus (HCV) infection, a common infection among heroin abusers. We thus investigated the in vivo impact of heroin use or heroin use plus HCV infection on the CD56(+) T cell frequency and function.A total of 37 heroin users with (17) or without (20) HCV infection and 17 healthy subjects were included in the study. Although there was no significant difference in CD56(+) T cell frequency in PBMCs among three study groups, CD56(+) T cells isolated from the heroin users had significantly lower levels of constitutive interferon-gamma (IFN-gamma) expression than those from the normal subjects. In addition, when stimulated by interleukin (IL)-12, CD56(+) natural T cells from HCV-infected heroin users produced significantly lower levels of IFN-gamma than those from the normal subjects. This diminished ability to produce IFN-gamma by CD56(+) T cells was associated with the increased plasma HCV viral loads in the HCV-infected heroin users. Investigation of the mechanisms showed that although heroin use or heroin use plus HCV infection had little impact on the expression of the key positive regulators (IL-12 receptors, STAT-1, 3, 4, 5, JAK-2, and TYK-2) in IL-12 pathway, heroin use or heroin use plus HCV infection induced the expression of suppressor of cytokine signaling protein-3 (SOCS-3) and protein inhibitors of activated STAT-3 (PIAS-3), two key inhibitors of IL-12 pathway.These findings provide compelling in vivo evidence that heroin use or heroin use plus HCV infection impairs CD56(+) T cell-mediated innate immune function, which may account for HCV infection and persistence in liver

Graph neural network based approach to automatically assigning common weakness enumeration identifiers for vulnerabilities

Abstract Vulnerability reports are essential for improving software security since they record key information on vulnerabilities. In a report, CWE denotes the weakness of the vulnerability and thus helps quickly understand the cause of the vulnerability. Therefore, CWE assignment is useful for categorizing newly discovered vulnerabilities. In this paper, we propose an automatic CWE assignment method with graph neural networks. First, we prepare a dataset that contains 3394 real world vulnerabilities from Linux, OpenSSL, Wireshark and many other software programs. Then, we extract statements with vulnerability syntax features from these vulnerabilities and use program slicing to slice them according to the categories of syntax features. On top of slices, we represent these slices with graphs that characterize the data dependency and control dependency between statements. Finally, we employ the graph neural networks to learn the hidden information from these graphs and leverage the Siamese network to compute the similarity between vulnerability functions, thereby assigning CWE IDs for these vulnerabilities. The experimental results show that the proposed method is effective compared to existing methods

Vibration Induces BAFF Overexpression and Aberrant O-Glycosylation of IgA1 in Cultured Human Tonsillar Mononuclear Cells in IgA Nephropathy

Objective. To investigate the influence of in vitro vibratory stimulation of human tonsillar mononuclear cells (TMCs). Methods. Fourteen IgA nephropathy (IgAN) patients with chronic tonsillitis (CT) and 12 CT patients with no renal pathology were enrolled. Group A TMCs were collected after 24 hours of culture and used to determine baseline levels. TMCs in groups B, C, D, E, and F were exposed to vibratory stimulation (60 Hz) for 0 (as the control group), 1, 3, 5, and 10 minutes, respectively. Results. Baseline concentrations of B-cell-activation factor (BAFF) and IgA1, BAFF mRNA expression, and aberrant O-glycosylation IgA1 level were higher in the IgAN group as compared to that in the CT group, and all increased after vibratory stimulation. Baseline mRNA expressions of core β1,3-galactosyltransferase (C1GALT1) and core β1,3GalT-specific molecular chaperone (Cosmc) were lower in the IgAN group; the levels decreased further after vibratory stimulation. Conclusion. In patients with IgAN, vibratory stimulation of TMCs appears to induce IgA1 secretion through activation of BAFF release and to aberrant O-glycosylation IgA1 by suppressing C1GALT1 and Cosmc expression. In vitro vibratory stimulation of human TMCs mimics the vibratory simulation of palatine tonsils produced by vocal cords during phonation

Vibration Induces BAFF Overexpression and Aberrant O-Glycosylation of IgA1 in Cultured Human Tonsillar Mononuclear Cells in IgA Nephropathy

Objective. To investigate the influence of in vitro vibratory stimulation of human tonsillar mononuclear cells (TMCs). Methods. Fourteen IgA nephropathy (IgAN) patients with chronic tonsillitis (CT) and 12 CT patients with no renal pathology were enrolled. Group A TMCs were collected after 24 hours of culture and used to determine baseline levels. TMCs in groups B, C, D, E, and F were exposed to vibratory stimulation (60 Hz) for 0 (as the control group), 1, 3, 5, and 10 minutes, respectively. Results. Baseline concentrations of B-cell-activation factor (BAFF) and IgA1, BAFF mRNA expression, and aberrant O-glycosylation IgA1 level were higher in the IgAN group as compared to that in the CT group, and all increased after vibratory stimulation. Baseline mRNA expressions of core β1,3-galactosyltransferase (C1GALT1) and core β1,3GalT-specific molecular chaperone (Cosmc) were lower in the IgAN group; the levels decreased further after vibratory stimulation. Conclusion. In patients with IgAN, vibratory stimulation of TMCs appears to induce IgA1 secretion through activation of BAFF release and to aberrant O-glycosylation IgA1 by suppressing C1GALT1 and Cosmc expression. In vitro vibratory stimulation of human TMCs mimics the vibratory simulation of palatine tonsils produced by vocal cords during phonation