Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising

Real-time bidding (RTB) is an important mechanism in online display advertising, where a proper bid for each page view plays an essential role for good marketing results. Budget constrained bidding is a typical scenario in RTB where the advertisers hope to maximize the total value of the winning impressions under a pre-set budget constraint. However, the optimal bidding strategy is hard to be derived due to the complexity and volatility of the auction environment. To address these challenges, in this paper, we formulate budget constrained bidding as a Markov Decision Process and propose a model-free reinforcement learning framework to resolve the optimization problem. Our analysis shows that the immediate reward from environment is misleading under a critical resource constraint. Therefore, we innovate a reward function design methodology for the reinforcement learning problems with constraints. Based on the new reward design, we employ a deep neural network to learn the appropriate reward so that the optimal policy can be learned effectively. Different from the prior model-based work, which suffers from the scalability problem, our framework is easy to be deployed in large-scale industrial applications. The experimental evaluations demonstrate the effectiveness of our framework on large-scale real datasets.Comment: In The 27th ACM International Conference on Information and Knowledge Management (CIKM 18), October 22-26, 2018, Torino, Italy. ACM, New York, NY, USA, 9 page

Atmospheric connections with the North Atlantic enhanced the deglacial warming in northeast China

Variations in atmospheric circulation across the last deglaciation in the northernmost monsoon-influenced regions of Asia are not well constrained, highlighting a fundamental gap in our understanding of Asian climate. Here we reconstruct continental air temperatures for northeast China across the last deglaciation (past 16 k. y.), based on the distribution of bacterial branched glycerol dialkyl glycerol tetraethers in a sequence of the Hani peat (Jilin Province, northeast China). Our results indicate large (as much as 10 degrees C) oscillations in temperature in northeast China across the deglaciation, oscillations significantly larger than observed in other temperature records from low-latitude or same-latitude East Asia, but consistent with climate model simulations. This enhanced magnitude, as well as the timing of temperature variations, provides evidence for atmospheric teleconnections with high latitudes; in particular, we suggest that highlatitude cooling associated with Arctic ice expansion and changes in Atlantic Meridional Overturning Circulation enhanced the intensity and lowered the temperature of Eurasian mid-latitude westerlies and northwesterly winds over East Asia during the last glacial, delivering cold air masses to northeast China. During the deglaciation the westerlies and therefore delivery of cold air masses weakened, amplifying the deglacial warming in this region. We conclude that changes in North Atlantic climate had a particularly strong impact on the northernmost parts of the East Asian monsoon-influenced area

Temporal Changes in Extracellular Vesicle Hemostatic Protein Composition Predict Favourable Left Ventricular Remodeling after Acute Myocardial Infarction

The subset of plasma extracellular vesicles (EVs) that coprecipitate with low-density lipoprotein (LDL-EVs) carry coagulation and fibrinolysis pathway proteins as cargo. We investigated the association between LDL-EV hemostatic/fibrinolysis protein ratios and post-acute myocardial infarction (post-AMI) left ventricular (LV) remodeling which precedes heart failure. Protein concentrations of von Willebrand factor (VWF), SerpinC1 and plasminogen were determined in LDL-EVs extracted from plasma samples obtained at baseline (within 72 h post-AMI), 1 month and 6 months post-AMI from 198 patients. Patients were categorized as exhibiting adverse (n = 98) or reverse (n = 100) LV remodeling based on changes in LV end-systolic volume (increased or decreased ≥15) over a 6-month period. Multiple level longitudinal data analysis with structural equation (ML-SEM) model was used to assess predictive value for LV remodeling independent of baseline differences. At baseline, protein levels of VWF, SerpinC1 and plasminogen in LDL-EVs did not differ between patients with adverse versus reverse LV remodeling. At 1 month post-AMI, protein levels of VWF and SerpinC1 decreased whilst plasminogen increased in patients with adverse LV remodeling. In contrast, VWF and plasminogen decreased whilst SerpinC1 remained unchanged in patients with reverse LV remodeling. Overall, compared with patients with adverse LV remodeling, higher levels of SerpinC1 and VWF but lower levels of plasminogen resulted in higher ratios of VWF:Plasminogen and SerpinC1:Plasminogen at both 1 month and 6 months post-AMI in patients with reverse LV remodeling. More importantly, ratios VWF:Plasminogen (AUC = 0.674) and SerpinC1:Plasminogen (AUC = 0.712) displayed markedly better prognostic power than NT-proBNP (AUC = 0.384), troponin-I (AUC = 0.467) or troponin-T (AUC = 0.389) (p \u3c 0.001) to predict reverse LV remodeling post-AMI. Temporal changes in the ratios of coagulation to fibrinolysis pathway proteins in LDL-EVs outperform current standard plasma biomarkers in predicting post-AMI reverse LV remodeling. Our findings may provide clinical cues to uncover the cellular mechanisms underpinning post-AMI reverse LV remodeling

Selective laser melting of 1.2738 mold steel: densification, microstructure and microhardness

In this study, the effects of laser power and scanning speed on the relative density, microstructure and microhardness of selectively laser melted (SLM) 1.2738 mold steel were systematically investigated. The deposited energy density, which can express the change in these process parameters and the relative density with one curve, were found more reliable than volume energy density (VED) on the design of process parameters. With the same VED value, samples formed by the combination of a high laser power and scanning speed possess a higher densification than that formed by low laser power and scanning speed. High power may lead to keyhole pore formation. In the top of the molten pool, ultrafine cellular structure was formed, causing that the microhardness value of the top section was always higher than that of the side section

Optimal Design and Mean Stress Estimation of Modular Metamaterials Inspired by Burr Puzzles

Modular impact-resistant metamaterials inspired by burr puzzles were recently proposed to combine flexibility, efficiency and adaptivity, which were also beneficial for sustainability in engineering protection. However, the optimal design remains to be explored and the mean stress cannot be effectively estimated. To break these limits, a stiffness-enhanced strategy is implemented to enhance the crashworthiness, and the relation between the mechanical behavior of metamaterials and locking points is revealed. The average thickness of all modules in the metamaterial is denoted by tave, and the thickness ratio of axially loaded to laterally loaded modules is denoted by y. From the experimental and simulation results, the mean stress of the metamaterials significantly increases with tave and y, while the deformation mode is gradually transformed into an inefficient global buckling mode and impairs the crashworthiness when ψ≥4. ψ=3 can be taken as the optimal design of metamaterials, which can increase the specific energy absorption SEA, energy absorption efficiency h and mean stress sm, respectively, by 62.4%, 44.2% and 57.6% compared to the regular design (ψ=1). On this basis, we develop a universe method to estimate the mean stress of the metamaterials with a relative error less than 9.6%, and a guideline for their design and application in engineering fields is summarized. This research opens a new avenue for broadening the design and applications of modular metamaterials in engineering applications

Protection of the Liuzheng Water Source: A Karst Water System in Dawu, Zibo, China

The Dawu water source is a rare, large-scale groundwater source located in northern China. The water supply function from this water source has, however, been lost due to anthropogenic pollution. In order to fully utilize valuable groundwater resources, a new water source of urban domestic water in Liu Zheng is planned. In this study, a tracer test and a numerical simulation method are used to examine the hydraulic connection between the Liuzheng water source and the Wangzhai industrial park; to optimize the exploitation layout of the Liuzheng water source and Dawu water source; and to propose the extent of the Liuzheng water source protection area. Results indicate that: (1) Karst development in the study area is uneven, and the Wangzhai area is a recharge area of the Liuzheng water source; (2) it is predicted that the groundwater flow field will not be significantly changed when a groundwater volume of 150,000 m3/day is exploited from the Liuzheng water source; (3) it is predicted that the proposed chemical park in Wangzhai will gradually pollute to the groundwater in the northern area of Liuzheng; and (4) results using the empirical formula method and the numerical simulation method indicate that the area of the primary protection area of the Liuzheng water source is about 0.59 km2, and the area of the secondary protection area is about 14.98 km2. Results from this study provide a certain technical basis for the exploitation and protection of groundwater in the Liuzheng water source

Sources of variability in quantifying circulating thymosin beta-4: literature review and recommendations

10.1080/14712598.2018.1448382EXPERT OPINION ON BIOLOGICAL THERAPY18sup1141-14