HARL: Hierarchical Adaptive Reinforcement Learning Based Auto Scheduler for Neural Networks

To efficiently perform inference with neural networks, the underlying tensor programs require sufficient tuning efforts before being deployed into production environments. Usually, enormous tensor program candidates need to be sufficiently explored to find the one with the best performance. This is necessary to make the neural network products meet the high demand of real-world applications such as natural language processing, auto-driving, etc. Auto-schedulers are being developed to avoid the need for human intervention. However, due to the gigantic search space and lack of intelligent search guidance, current auto-schedulers require hours to days of tuning time to find the best-performing tensor program for the entire neural network. In this paper, we propose HARL, a reinforcement learning (RL) based auto-scheduler specifically designed for efficient tensor program exploration. HARL uses a hierarchical RL architecture in which learning-based decisions are made at all different levels of search granularity. It also automatically adjusts exploration configurations in real-time for faster performance convergence. As a result, HARL improves the tensor operator performance by 22% and the search speed by 4.3x compared to the state-of-the-art auto-scheduler. Inference performance and search speed are also significantly improved on end-to-end neural networks

KRAS mutation: The booster of pancreatic ductal adenocarcinoma transformation and progression

Pancreatic ductal adenocarcinoma (PDAC) is the most common type of pancreatic cancer. It has a poor response to conventional therapy and has an extremely poor 5-year survival rate. PDAC is driven by multiple oncogene mutations, with the highest mutation frequency being observed in KRAS. The KRAS protein, which binds to GTP, has phosphokinase activity, which further activates downstream effectors. KRAS mutation contributes to cancer cell proliferation, metabolic reprogramming, immune escape, and therapy resistance in PDAC, acting as a critical driver of the disease. Thus, KRAS mutation is positively associated with poorer prognosis in pancreatic cancer patients. This review focus on the KRAS mutation patterns in PDAC, and further emphases its role in signal transduction, metabolic reprogramming, therapy resistance and prognosis, hoping to provide KRAS target therapy strategies for PDAC

Which Channel to Ask My Question? Personalized Customer Service Request Stream Routing using Deep Reinforcement Learning

Customer services are critical to all companies, as they may directly connect to the brand reputation. Due to a great number of customers, e-commerce companies often employ multiple communication channels to answer customers' questions, for example, chatbot and hotline. On one hand, each channel has limited capacity to respond to customers' requests, on the other hand, customers have different preferences over these channels. The current production systems are mainly built based on business rules, which merely considers tradeoffs between resources and customers' satisfaction. To achieve the optimal tradeoff between resources and customers' satisfaction, we propose a new framework based on deep reinforcement learning, which directly takes both resources and user model into account. In addition to the framework, we also propose a new deep-reinforcement-learning based routing method-double dueling deep Q-learning with prioritized experience replay (PER-DoDDQN). We evaluate our proposed framework and method using both synthetic and a real customer service log data from a large financial technology company. We show that our proposed deep-reinforcement-learning based framework is superior to the existing production system. Moreover, we also show our proposed PER-DoDDQN is better than all other deep Q-learning variants in practice, which provides a more optimal routing plan. These observations suggest that our proposed method can seek the trade-off where both channel resources and customers' satisfaction are optimal.Comment: 13 pages, 7 figure

Pathway and kinetics of cyhalothrin biodegradation by Bacillus thuringiensis strain ZS-19

10.1038/srep08784Scientific Reports

Expression of the Inhibitory Receptor TIGIT Is Up-Regulated Specifically on NK Cells With CD226 Activating Receptor From HIV-Infected Individuals

Natural killer (NK) cells are important for maintenance of innate immune system stability and serve as a first line of defense against tumors and virus infections; they can act either directly or indirectly and are regulated via co-operation between inhibitory and stimulatory surface receptors. The recently reported inhibitory receptor, TIGIT, can be expressed on the NK cell surface; however, the expression level and function of TIGIT on NK cells during HIV infection is unknown. In this study, for the first time, we investigated the expression and function of TIGIT in NK cells from HIV-infected individuals. Our data demonstrate that the level of TIGIT is higher on NK cells from patients infected with human immunodeficiency virus (HIV) compared with HIV-negative healthy controls. TIGIT expression is inversely correlated with CD4+ T cell counts and positively correlated with plasma viral loads. Additionally, levels of the TIGIT ligand, CD155, were higher on CD4+ T cells from HIV-infected individuals compared with those from healthy controls; however, there was no difference in the level of the activating receptor, CD226, which recognizes the same ligands as TIGIT. Furthermore, TIGIT was found to specifically up-regulated on CD226+ NK cells in HIV-infected individuals, and either rIL-10, or rIL-12 + rIL-15, could induce TIGIT expression on these cells. In addition, high TIGIT expression inhibited the production of interferon-gamma (IFN-γ) by NK cells, while TIGIT inhibition restored IFN-γ production. Overall, these results highlight the important role of TIGIT in NK cell function and suggest a potential new avenue for the development of therapeutic strategies toward a functional cure for HIV

NKG2C+NKG2A− Natural Killer Cells are Associated with a Lower Viral Set Point and may Predict Disease Progression in Individuals with Primary HIV Infection

Natural killer (NK) cells are the first line of defense against pathogens of the immune system and also play an important role in resistance against HIV. The activating receptor NKG2C and the inhibitory receptor NKG2A co-modulate the function of NK cells by recognizing the same ligand, HLA-E. However, the role of NKG2A and NKG2C on viral set point and the prediction of HIV disease progression have been rarely reported. In this study, we determined the expression of NKG2C or NKG2A on the surface of NK cells from 22 individuals with primary HIV infection (PHI) stage and 23 HIV-negative normal control (NC) subjects. The CD4+ T cell count and plasma level of HIV RNA in the infected individuals were longitudinally followed-up for about 720 days. The proportion of NKG2C+NKG2A− NK cells was higher in subjects from the low set point group and was negatively correlated with the viral load. In addition, strong anti-HIV activities were observed in NKG2C+ NK cells from the HIV-positive donors. Furthermore, a proportion of NKG2C+NKG2A− NK cells >35.45%, and a ratio of NKG2C/NKG2A >1.7 were predictive for higher CD4+ T cell counts 720 days after infection. Collectively, the experimental results allow us to draw the conclusion that NKG2C+ NK cells might exert an antiviral effect and that the proportion of NKG2C+NKG2A− NK cells, and the ratio of NKG2C/NKG2A, are potential biomarkers for predicting HIV disease progression

Configuration Synthesis and Performance Analysis of Finger Soft Actuator

Compared with the traditional rigid finger actuator, the soft actuator has the advantages of light weight and good compliance. This type of finger actuator can be used for data acquisition or finger rehabilitation training, and it has broad application prospects. The motion differences between the soft actuator and finger may cause extrusion deformation at the binding point, and the binding forces along nonfunctional direction may reduce drive efficiency. In order to reduce the negative deformation of soft structure and improve comfort, the configuration synthesis and performance analysis of the finger soft actuator were conducted for the present work. The configuration synthesis method for soft actuator was proposed based on the analysis of the physiological structure of the finger, and the soft actuator can make the human-machine closed-loop structure including n joints (n=1, 2, 3) meet the requirement of DOF (degrees of freedom). Then the typical feasible configurations were enumerated. The different typical configurations were analyzed and compared based on the establishment of mathematical models and simulation analysis. Results show that the configuration design method is feasible. This study offers a theoretical basis for designing the configuration of finger soft actuator