Real-Time Bidding by Reinforcement Learning in Display Advertising

The majority of online display ads are served through real-time bidding (RTB) --- each ad display impression is auctioned off in real-time when it is just being generated from a user visit. To place an ad automatically and optimally, it is critical for advertisers to devise a learning algorithm to cleverly bid an ad impression in real-time. Most previous works consider the bid decision as a static optimization problem of either treating the value of each impression independently or setting a bid price to each segment of ad volume. However, the bidding for a given ad campaign would repeatedly happen during its life span before the budget runs out. As such, each bid is strategically correlated by the constrained budget and the overall effectiveness of the campaign (e.g., the rewards from generated clicks), which is only observed after the campaign has completed. Thus, it is of great interest to devise an optimal bidding strategy sequentially so that the campaign budget can be dynamically allocated across all the available impressions on the basis of both the immediate and future rewards. In this paper, we formulate the bid decision process as a reinforcement learning problem, where the state space is represented by the auction information and the campaign's real-time parameters, while an action is the bid price to set. By modeling the state transition via auction competition, we build a Markov Decision Process framework for learning the optimal bidding policy to optimize the advertising performance in the dynamic real-time bidding environment. Furthermore, the scalability problem from the large real-world auction volume and campaign budget is well handled by state value approximation using neural networks.Comment: WSDM 201

Characterization of the Fiber Connectivity Profile of the Cerebral Cortex in Schizotypal Personality Disorder: A Pilot Study

Schizotypal personality disorder (SPD) is considered one of the classic disconnection syndromes. However, the specific cortical disconnectivity pattern has not been fully investigated. In this study, we aimed to explore significant alterations in whole-cortex structural connectivity in SPD individuals (SPDs) by combining the techniques of brain surface morphometry and white matter (WM) tractography. Diffusion and structural MR data were collected from twenty subjects with SPD (all males; age, 19.7 ± 0.9 yrs) and eighteen healthy controls (all males; age, 20.3 ± 1.0 yrs). To measure the structural connectivity for a given unit area of the cortex, the fiber connectivity density (FiCD) value was proposed and calculated as the sum of the fractional anisotropy of all the fibers connecting to that unit area in tractography. Then, the resultant whole-cortex FiCD maps were compared in a vertex-wise manner between SPDs and controls. Compared with normal controls, SPDs showed significantly decreased FiCD in the rostral middle frontal gyrus (crossing BA9 and BA10) and significantly increased FiCD in the anterior part of the fusiform/inferior temporal cortex (P < 0.05, Monte Carlo simulation corrected). Moreover, the gray matter volume extracted from the left rostral middle frontal cluster was observed to be significantly greater in the SPD group (P = 0.02). Overall, this study identifies a decrease in connectivity in the left middle frontal cortex as a key neural deficit at the whole-cortex level in SPD, thus providing insight into its neuropathological basis

Identifying the Alteration Patterns of Brain Functional Connectivity in Progressive Mild Cognitive Impairment Patients: A Longitudinal Whole-Brain Voxel-Wise Degree Analysis

Patients with mild cognitive impairment (MCI) are at high risk for developing Alzheimer’s disease (AD), while some of them may remain stable over decades. The underlying mechanism is still not fully understood. In this study, we aimed to explore the connectivity differences between progressive MCI (PMCI) and stable MCI (SMCI) individuals on a whole-brain scale and on a voxel-wise basis, and we also aimed to reveal the differential dynamic alternation patterns between these two disease subtypes. The resting-state functional magnetic resonance images of PMCI and SMCI patients at baseline and year-one were obtained from the Alzheimer’s Disease Neuroimaging Initiative dataset, and the progression was determined based on a three-year follow-up. A whole-brain voxel-wise degree map that was calculated based on graph-theory was constructed for each subject, and then the cross-sectional and longitudinal analyses on the degree maps were performed between PMCI and SMCI patients. In longitudinal analyses, compared with SMCI group, PMCI group showed decreased long-range degree in the left middle occipital/supramarginal gyrus, while the short-range degree was increased in the left supplementary motor area and middle frontal gyrus and decreased in the right middle temporal pole. A significant longitudinal alteration of decreased short-range degree in the right middle occipital was found in PMCI group. Taken together with previous evidence, our current findings may suggest that PMCI, compared with SMCI, might be a severe presentation of disease along the AD continuum, and the rapidly reduced degree in the right middle occipital gyrus may have indicative value for the disease progression. Moreover, the cross-sectional comparison results and corresponding receiver-operator characteristic-curves analyses may indicate that the baseline degree difference is not a good predictor of disease progression in MCI patients. Overall, these findings may provide objective evidence and an indicator to characterize the progression-related brain connectivity changes in MCI patients

Would transcranial direct current stimulation (tDCS) enhance the effects of working memory training in older adults with mild neurocognitive disorder due to Alzheimer’s disease: study protocol for a randomized controlled trial

© 2015 Cheng et al. Background: There has been longstanding interesting in cognitive training for older adults with cognitive impairment. In this study, we will investigate the effects of working memory training, and explore augmentation strategies that could possibly consolidate the effects in older adults with mild neurocognitive disorder. Transcranial direct current stimulation (tDCS) has been demonstrated to affect the neuronal excitability and reported to enhance memory performance. As tDCS may also modulate cognitive function through changes in neuroplastic response, it would be adopted as an augmentation strategy for working memory training in the present study. Methods/Design: This is a 4-week intervention double-blind randomized controlled trial (RCT) of tDCS. Chinese older adults (aged 60 to 90 years) with mild neurocognitive disorder due to Alzheimer 's disease (DSM-5 criteria) would be randomized into a 4-week intervention of either tDCS-working memory (DCS-WM), tDCS-control cognitive training (DCS-CC), and sham tDCS-working memory (WM-CD) groups. The primary outcome would be working memory test - the n-back task performance and the Chinese version of the Alzheimer's Disease Assessment Scale - Cognitive Subscale (ADAS-Cog). Secondary outcomes would be test performance of specific cognitive domains and mood. Intention-to-treat analysis would be carried out. Changes of efficacy indicators with time and intervention would be tested with mixed effect models. Discussion: This study adopts the theory of neuroplasticity to evaluate the potential cognitive benefits of non-invasive electrical brain stimulation, working memory training and dual stimulation in older adults at risk of cognitive decline. It would also examine the tolerability, program adherence and adverse effects of this novel intervention. Information would be helpful for further research of dementia prevention studies. Trial registration: ChiCTR-TRC- 14005036Date of registration: 31 July 2014.published_or_final_versio

Design and manipulation of high-performance photovoltaic systems based on two-dimensional novel KAgSe/KAgX(X=S,Te) van der Waals heterojunctions

The realization of high-performance two-dimensional (2D) solar photovoltaic systems are both fundamentally intriguing and practically appealing to meet the fast-growing energy requirements. Since the limited application of single 2D crystals in photovoltaic, here we propose a family of 2D KAgSe/KAgX(X=S,Te) van der Waals heterostructures (vdWHs), which are constructed by combining two different KAgX layers through interlayer vdW interaction. After a systematic study and further regulatory research of these vdWHs based on the first-principles, numerous fascinating characteristics and physical mechanisms are obtained. Firstly, favorable potential applications of these vdWHs in photovoltaics are confirmed in virtue of their desirable optoelectronic properties, such as the robust stabilitis, moderate direct band gaps, type-II band alignments together with superior carrier mobilities, visible optical absorptions, power conversion efficiencys (PCEs) and photocurrents in their based photovoltaic devices. More importantly, when under varying vertical electric field Ez, a phase transition of band alignment from type-II to type-I of these vdWHs can be induced by the opposite band shifts between layers, which may enrich their applications in light-emitting diodes and lasers. Meanwhile, the PCE can be expanded up to 23%, and an obvious red-shift peak of the photocurrent in the visible light range are also obtained at different Ez. These fascinating tunable properties of KAgSe/KAgX vdWHs under varying Ez not only promote the improvement of their photoelectric performances, but the underlying mechanisms can also be applied to next experimental design and practical application of other 2D photovoltaic systems. Especially for the red-shift peak of the photocurrent, which is rarely found but highly desirable in practical visible photoelectric conversion.Comment: 11 pages, 7figure

Physiological Characterization of Cut-to-Cut Yield Variations of Alfalfa Genotypes under Controlled Greenhouse Conditions

In a temperate region, alfalfa (Medicago sativa) crops are usually harvested 3-6 times per annum. The biomass yields of first and second cuts in the spring are generally the high-est. However, in subsequent cuts the biomass yields decline, with the final 1 or 2 cuts producing the lowest yields (Wang et al. 2009). This seasonal reduction in alfalfa biomass yields could be associated with prevailing changes in environmental factors such as rainfall and heat stress or due to biological characteristics of alfalfa crop itself. In this study, alfalfa was grown under controlled greenhouse conditions with suitable temperature, light, water and nutrient supply to determine the driving force in cut-to-cut biomass yield variations among alfalfa genotypes

CYP2E1 Sensitizes the Liver to LPS- and TNF α-Induced Toxicity via Elevated Oxidative and Nitrosative Stress and Activation of ASK-1 and JNK Mitogen-Activated Kinases

The mechanisms by which alcohol causes cell injury are not clear. A major mechanism is the role of lipid peroxidation and oxidative stress in alcohol toxicity. Many pathways have been suggested to play a role in how alcohol induces oxidative stress. Considerable attention has been given to alcohol elevated production of lipopolysaccharide (LPS) and TNFα and to alcohol induction of CYP2E1. These two pathways are not exclusive of each other; however, interactions between them, have not been extensively evaluated. Increased oxidative stress from induction of CYP2E1 sensitizes hepatocytes to LPS and TNFα toxicity and oxidants, activation of inducible nitric oxide synthase and p38 and JNK MAP kinases, and mitochondrial dysfunction are downstream mediators of this CYP2E1-LPS/TNFα-potentiated hepatotoxicity. This paper will summarize studies showing potentiated interactions between these two risk factors in promoting liver injury and the mechanisms involved including activation of the mitogen-activated kinase kinase kinase ASK-1. Decreasing either cytosolic or mitochondrial thioredoxin in HepG2 cells expressing CYP2E1 causes loss of cell viability and elevated oxidative stress via an ASK-1/JNK-dependent mechanism. We hypothesize that similar interactions occur as a result of ethanol induction of CYP2E1 and TNFα

Randomly Projected Convex Clustering Model: Motivation, Realization, and Cluster Recovery Guarantees

In this paper, we propose a randomly projected convex clustering model for clustering a collection of $n$ high dimensional data points in $\mathbb{R}^d$ with $K$ hidden clusters. Compared to the convex clustering model for clustering original data with dimension $d$, we prove that, under some mild conditions, the perfect recovery of the cluster membership assignments of the convex clustering model, if exists, can be preserved by the randomly projected convex clustering model with embedding dimension $m = O(\epsilon^{-2}\log(n))$, where $0 < \epsilon < 1$ is some given parameter. We further prove that the embedding dimension can be improved to be $O(\epsilon^{-2}\log(K))$, which is independent of the number of data points. Extensive numerical experiment results will be presented in this paper to demonstrate the robustness and superior performance of the randomly projected convex clustering model. The numerical results presented in this paper also demonstrate that the randomly projected convex clustering model can outperform the randomly projected K-means model in practice