Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising

Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only need to estimate the relevance between the ads and user's interests, but most importantly require a strategic response with respect to other advertisers bidding in the market. In this paper, we formulate bidding optimization with multi-agent reinforcement learning. To deal with a large number of advertisers, we propose a clustering method and assign each cluster with a strategic bidding agent. A practical Distributed Coordinated Multi-Agent Bidding (DCMAB) has been proposed and implemented to balance the tradeoff between the competition and cooperation among advertisers. The empirical study on our industry-scaled real-world data has demonstrated the effectiveness of our methods. Our results show cluster-based bidding would largely outperform single-agent and bandit approaches, and the coordinated bidding achieves better overall objectives than purely self-interested bidding agents

IL28B is associated with outcomes of chronic HBV infection

Purpose The role of IL28B gene variants and expression in hepatitis B virus (HBV) infections are not well understood. Here, we evaluated whether IL28B gene expression and rs12979860 variations are associated with HBV outcomes. Materials and Methods IL28B genetic variations (rs12979860) were genotyped by pyrosequencing of DNA samples from 137 individuals with chronic HBV infection [50 inactive carriers (IC), 34 chronic hepatitis B (CHB), 27 cirrhosis, 26 hepatocellular carcinoma (HCC)], and 19 healthy controls. IL28A/B mRNA expression in peripheral blood mononuclear cells was determined by qRT-PCR, and serum IL28B protein was measured by ELISA. Results Patients with IL28B C/C genotype had greater IL28A/B mRNA expression and higher IL28B protein levels than C/T patients. Within the various disease stages, compared to IC and healthy controls, IL28B expression was reduced in the CHB, cirrhosis, and HCC cohorts (CHB vs. IC, p=0.02; cirrhosis vs. IC, p=0.01; HCC vs. IC, p=0.001; CHB vs. controls, p&#60;0.01; cirrhosis vs. controls, p&#60;0.01; HCC vs. controls, p&#60;0.01). When stratified with respect to serum HBV markers in the IC and CHB cohorts, IL28B mRNA and protein levels were higher in HBeAg-positive than negative individuals (p=0.01). Logistic regression analysis revealed that factors associated with high IL28B protein levels were C/C versus C/T genotype [p=0.016, odds ratio (OR)=0.25, 95% confidence interval (CI)=0.08-0.78], high alanine aminotransferase values (p&#60;0.001, OR=8.02, 95% CI=2.64-24.4), and the IC stage of HBV infection (p&#60;0.001). Conclusion Our data suggest that IL28B genetic variations may play an important role in long-term development of disease in chronic HBV infections.</p

Cross-linking of CD81 by HCV-E2 protein inhibits human intrahepatic plasmacytoid dendritic cells response to CpG-ODN

Plasmacytoid dendritic cells (pDCs) are reported to be defective in HCV-infected patients, the mechanisms of which remain poorly understood. We isolated liver derived mononuclear cells (LMNCs) and pDCs from normal liver tissues of benign tumor dissections and liver transplant donors. Isolated pDCs and LMNCs were cultured with precoated HCV envelop protein E2 (HCV-E2) or anti-CD81 mAb in the presence of CpG-ODN. Our results show that cross-linking of CD81 by either HCV-E2 or anti-CD81 mAb inhibits IFN-α secretion in CpG-induced pDCs; down-regulates HLA-DR, CD80 and CD86 expression in pDCs; and suppresses CpG-ODN induced proliferation and survival of pDCs. The blockade of CD81 by soluble anti-CD81 antibody restores pDCs response to CpG-ODN. These results suggest that HCV E2 protein interacts with CD81 to inhibit pDC maturation, activation, and IFN-α production, and may thereby contribute to the impaired innate anti-viral immune response in HCV infection

A new contribution to the raptorial ciliate genus Lacrymaria (Protista: Ciliophora): a brief review and comprehensive descriptions of two new species from Changjiang Estuary

Ciliates serve as excellent indicators for water quality monitoring. However, their utilization is hindered by various taxonomic confusions. The ciliate genus Lacrymaria Bory de Saint-Vincent, 1824 is commonly found in different aquatic habitats, but its taxonomy has been sparsely investigated using state-of-the-art methods. This study investigated two new Lacrymaria species from Nanhui Wetland, Shanghai, China, using living observation, protargol staining, and molecular phylogeny methods. Lacrymaria songi sp. nov. is 180–340 × 20–25 μm in size and possesses 12–16 somatic kineties, 1 terminal contractile vacuole, 2 macronuclear nodules, and 2 types of rod-shaped extrusomes. Lacrymaria dragescoi sp. nov. is distinguished from its congeners by its cell size of 210–400 × 25–35 μm, 14–17 somatic kineties, 1 terminal contractile vacuole, 1 macronucleus, and 2 types of rod-shaped extrusomes. Phylogenetic analyses based on SSU rRNA gene sequences indicate that Lacrymariidae is monophyletic but Lacrymaria is not. Additionally, a brief review of the genus Lacrymaria is provided in this study. We suggest that L. bulbosa Alekperov, 1984, L. lanceolata Kahl, 1930, and L. ovata Burkovsky, 1970 be removed from the genus and propose Phialina lanceolata nov. comb. and Phialina ovata nov. comb. for the latter two.ZooBank registration: Present work: urn:lsid:zoobank.org:pub:CDFB1EBD-80BD-4533-B391-CEE89F62EDC4 Lacrymaria songi sp. nov.: urn:lsid:zoobank.org:act:417E7C2D-DAEC-4711-90BB-64AB3CD2F7D5 Lacrymaria dragescoi sp. nov.: urn:lsid:zoobank.org:act:8778D6B0-1F2E-473C-BE19-3F685391A40D

Learning to Infer User Hidden States for Online Sequential Advertising

To drive purchase in online advertising, it is of the advertiser's great interest to optimize the sequential advertising strategy whose performance and interpretability are both important. The lack of interpretability in existing deep reinforcement learning methods makes it not easy to understand, diagnose and further optimize the strategy. In this paper, we propose our Deep Intents Sequential Advertising (DISA) method to address these issues. The key part of interpretability is to understand a consumer's purchase intent which is, however, unobservable (called hidden states). In this paper, we model this intention as a latent variable and formulate the problem as a Partially Observable Markov Decision Process (POMDP) where the underlying intents are inferred based on the observable behaviors. Large-scale industrial offline and online experiments demonstrate our method's superior performance over several baselines. The inferred hidden states are analyzed, and the results prove the rationality of our inference.Comment: to be published in CIKM 202