Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising

Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only need to estimate the relevance between the ads and user's interests, but most importantly require a strategic response with respect to other advertisers bidding in the market. In this paper, we formulate bidding optimization with multi-agent reinforcement learning. To deal with a large number of advertisers, we propose a clustering method and assign each cluster with a strategic bidding agent. A practical Distributed Coordinated Multi-Agent Bidding (DCMAB) has been proposed and implemented to balance the tradeoff between the competition and cooperation among advertisers. The empirical study on our industry-scaled real-world data has demonstrated the effectiveness of our methods. Our results show cluster-based bidding would largely outperform single-agent and bandit approaches, and the coordinated bidding achieves better overall objectives than purely self-interested bidding agents

Postoperative Radiotherapy and N2 Non-small Cell Lung Cancer Prognosis: A Retrospective Study Based on Surveillance, Epidemiology, and End Results Database

The purpose of this study is to clarify the significance of postoperative radiotherapy for N2 lung cancer. This study aimed to investigate the effect of postoperative radiotherapy on the survival and prognosis of patients with N2 lung cancer. Data from 12,000 patients with N2 lung cancer were extracted from the Surveillance, Epidemiology, and End Results database (2004-2012). Age at disease onset and 5-year survival rates were calculated. Survival curves were plotted using the Kaplan-Meier method. The univariate log-rank test was performed. Multivariate Cox regression were used to examine factors affecting survival. Patients’ median age was 67 years (mean 66.46 ± 10.03). The 5-year survival rate was 12.55%. Univariate analysis revealed age, sex, pathology, and treatment regimen as factors affecting prognosis. In multivariate analysis, when compared to postoperative chemotherapy, postoperative chemoradiotherapy was better associated with survival benefits (hazard ratio [HR]= 0.85, 95% confidence interval [CI]: 0.813-0.898, P <0.001). Propensity score matching revealed that patients who had received postoperative chemoradiotherapy had a better prognosis than did patients who had received postoperative chemotherapy (HR=0.869, 95% CI: 0.817-0.925, P <0.001). Female patients and patients aged <65 years had a better prognosis than did their counterparts. Patients with adenocarcinoma had a better prognosis than did patients with squamous cell carcinoma. Moreover, prognosis worsened with increasing disease T stage. Patients who had received postoperative chemoradiotherapy had a better prognosis than did patients who had received postoperative chemotherapy. Postoperative radiotherapy was an independent prognostic factor in this patient group

Blocking interaction between SHP2 and PD‐1 denotes a novel opportunity for developing PD‐1 inhibitors

Small molecular PD‐1 inhibitors are lacking in current immuno‐oncology clinic. PD‐1/PD‐L1 antibody inhibitors currently approved for clinical usage block interaction between PD‐L1 and PD‐1 to enhance cytotoxicity of CD8+ cytotoxic T lymphocyte (CTL). Whether other steps along the PD‐1 signaling pathway can be targeted remains to be determined. Here, we report that methylene blue (MB), an FDA‐approved chemical for treating methemoglobinemia, potently inhibits PD‐1 signaling. MB enhances the cytotoxicity, activation, cell proliferation, and cytokine‐secreting activity of CTL inhibited by PD‐1. Mechanistically, MB blocks interaction between Y248‐phosphorylated immunoreceptor tyrosine‐based switch motif (ITSM) of human PD‐1 and SHP2. MB enables activated CTL to shrink PD‐L1 expressing tumor allografts and autochthonous lung cancers in a transgenic mouse model. MB also effectively counteracts the PD‐1 signaling on human T cells isolated from peripheral blood of healthy donors. Thus, we identify an FDA‐approved chemical capable of potently inhibiting the function of PD‐1. Equally important, our work sheds light on a novel strategy to develop inhibitors targeting PD‐1 signaling axis

IL-10 plays a central regulatory role in the cytokines induced by hepatitis C virus core protein and polyinosinic acid:polycytodylic acid

Hepatitis C virus (HCV) can cause persistent infection and chronic liver disease, and viral factors are involved in HCV persistence. HCV core protein, a highly conserved viral protein, not only elicits an immunoresponse, but it also regulates it. In addition, HCV core protein interacts with toll-like receptors (TLRs) on monocytes, inducing them to produce cytokines. Polyinosinic acid:polycytodylic acid (polyI:C) is a synthetic analogue of double-stranded RNA that binds to TLR3 and can induce secretion of type I IFN from monocytes. Cytokine response against HCV is likely to affect the natural course of infection as well as HCV persistence. However, possible effects of cytokines induced by HCV core protein and polyI:C remain to be investigated. In this study, we isolated CD14+ monocytes from healthy donors, cultured them in the presence of HCV core protein and/or polyI:C, and characterized the induced cytokines, phenotypes and mechanisms. We demonstrated that HCV core protein- and polyI:C-stimulated CD14+ monocytes secreted tumor necrosis factor-α (TNF-α), interleukin (IL)-1β, IL-10, and type I interferon (IFN). Importantly, TNF-α and IL-1β regulated the secretion of IL-10, which then influenced the expression of signal transducer and activator of transcription 1 (STAT1) and interferon regulatory factor 1 (IRF1) and subsequently the production of type I IFN. Interestingly, type I IFN also regulated the production of IL-10, which in turn inhibited the nuclear factor (NF)-κB subunit, reducing TNF-α and IL-1β levels. Therefore, IL-10 appears to play a central role in regulating the production of cytokines induced by HCV core protein and polyI:C

Regulatory NK cells mediated between immunosuppressive monocytes and dysfunctional T cells in chronic HBV infection

Background and aims HBV infection represents a major health problem worldwide, but the immunological mechanisms by which HBV causes chronic persistent infection remain only partly understood. Recently, cell subsets with suppressive features have been recognised among monocytes and natural killer (NK) cells. Here we examine the effects of HBV on monocytes and NK cells. Methods Monocytes and NK cells derived from chronic HBV-infected patients and healthy controls were purified and characterised for phenotype, gene expression and cytokines secretion by flow cytometry, quantitative real-time (qRT)-PCR, ELISA and western blotting. Culture and coculture of monocytes and NK cells were used to determine NK cell activation, using intracellular cytokines staining. Results In chronic HBV infection, monocytes express higher levels of PD-L1, HLA-E, interleukin (IL)-10 and TGF-β, and NK cells express higher levels of PD-1, CD94 and IL-10, compared with healthy individuals. HBV employs hepatitis B surface antigen (HBsAg) to induce suppressive monocytes with HLA-E, PD-L1, IL-10 and TGF-β expression via the MyD88/NFκ B signalling pathway. HBV-treated monocytes induce NK cells to produce IL-10, via PD-L1 and HLA-E signals. Such NK cells inhibit autologous T cell activation. Conclusions Our findings reveal an immunosuppressive cascade, in which HBV generates suppressive monocytes, which initiate regulatory NK cells differentiation resulting in T cell inhibition

HCV core protein inhibits polarization and activity of both M1 and M2 macrophages through the TLR2 signaling pathway

Hepatitis C virus (HCV) establishes persistent infection in most infected patients, and eventually causes chronic hepatitis, cirrhosis, and hepatocellular carcinoma in some patients. Monocytes and macrophages provide the first line of defense against pathogens, but their roles in HCV infection remains unclear. We have reported that HCV core protein (HCVc) manipulates human blood-derived dendritic cell development. In the present study, we tested whether HCVc affects human blood-derived monocyte differentiating into macrophages. Results showed that HCVc inhibits monocyte differentiation to either M1 or M2 macrophages through TLR2, associated with impaired STATs signaling pathway. Moreover, HCVc inhibits phagocytosis activity of M1 and M2 macrophages, M1 macrophage-induced autologous and allogeneic CD4+ T cell activation, but promotes M2 macrophage-induced autologous and allogeneic CD4+ T cell activation. In conclusion, HCVc inhibits monocyte-derived macrophage polarization via TLR2 signaling, leading to dysfunctions of both M1 and M2 macrophages in chronic HCV infected patients. This may contribute to the mechanism of HCV persistent infection, and suggest that blockade of HCVc might be a novel therapeutic approach to treating HCV infection

One-dimensional fluids with second nearest-neighbor interactions

As is well known, one-dimensional systems with interactions restricted to first nearest neighbors admit a full analytically exact statistical-mechanical solution. This is essentially due to the fact that the knowledge of the first nearest-neighbor probability distribution function, $p_1(r)$, is enough to determine the structural and thermodynamic properties of the system. On the other hand, if the interaction between second nearest-neighbor particles is turned on, the analytically exact solution is lost. Not only the knowledge of $p_1(r)$ is not sufficient anymore, but even its determination becomes a complex many-body problem. In this work we systematically explore different approximate solutions for one-dimensional second nearest-neighbor fluid models. We apply those approximations to the square-well and the attractive two-step pair potentials and compare them with Monte Carlo simulations, finding an excellent agreement.Comment: 26 pages, 12 figures; v2: more references adde