Marketing Budget Allocation with Offline Constrained Deep Reinforcement Learning

We study the budget allocation problem in online marketing campaigns that utilize previously collected offline data. We first discuss the long-term effect of optimizing marketing budget allocation decisions in the offline setting. To overcome the challenge, we propose a novel game-theoretic offline value-based reinforcement learning method using mixed policies. The proposed method reduces the need to store infinitely many policies in previous methods to only constantly many policies, which achieves nearly optimal policy efficiency, making it practical and favorable for industrial usage. We further show that this method is guaranteed to converge to the optimal policy, which cannot be achieved by previous value-based reinforcement learning methods for marketing budget allocation. Our experiments on a large-scale marketing campaign with tens-of-millions users and more than one billion budget verify the theoretical results and show that the proposed method outperforms various baseline methods. The proposed method has been successfully deployed to serve all the traffic of this marketing campaign.Comment: WSDM 23, Best Paper Candidat

Fatty infiltration in the musculoskeletal system: pathological mechanisms and clinical implications

Fatty infiltration denotes the anomalous accrual of adipocytes in non-adipose tissue, thereby generating toxic substances with the capacity to impede the ordinary physiological functions of various organs. With aging, the musculoskeletal system undergoes pronounced degenerative alterations, prompting heightened scrutiny regarding the contributory role of fatty infiltration in its pathophysiology. Several studies have demonstrated that fatty infiltration affects the normal metabolism of the musculoskeletal system, leading to substantial tissue damage. Nevertheless, a definitive and universally accepted generalization concerning the comprehensive effects of fatty infiltration on the musculoskeletal system remains elusive. As a result, this review summarizes the characteristics of different types of adipose tissue, the pathological mechanisms associated with fatty infiltration in bone, muscle, and the entirety of the musculoskeletal system, examines relevant clinical diseases, and explores potential therapeutic modalities. This review is intended to give researchers a better understanding of fatty infiltration and to contribute new ideas to the prevention and treatment of clinical musculoskeletal diseases

Potential of Core-Collapse Supernova Neutrino Detection at JUNO

JUNO is an underground neutrino observatory under construction in Jiangmen, China. It uses 20kton liquid scintillator as target, which enables it to detect supernova burst neutrinos of a large statistics for the next galactic core-collapse supernova (CCSN) and also pre-supernova neutrinos from the nearby CCSN progenitors. All flavors of supernova burst neutrinos can be detected by JUNO via several interaction channels, including inverse beta decay, elastic scattering on electron and proton, interactions on C12 nuclei, etc. This retains the possibility for JUNO to reconstruct the energy spectra of supernova burst neutrinos of all flavors. The real time monitoring systems based on FPGA and DAQ are under development in JUNO, which allow prompt alert and trigger-less data acquisition of CCSN events. The alert performances of both monitoring systems have been thoroughly studied using simulations. Moreover, once a CCSN is tagged, the system can give fast characterizations, such as directionality and light curve

Detection of the Diffuse Supernova Neutrino Background with JUNO

As an underground multi-purpose neutrino detector with 20 kton liquid scintillator, Jiangmen Underground Neutrino Observatory (JUNO) is competitive with and complementary to the water-Cherenkov detectors on the search for the diffuse supernova neutrino background (DSNB). Typical supernova models predict 2-4 events per year within the optimal observation window in the JUNO detector. The dominant background is from the neutral-current (NC) interaction of atmospheric neutrinos with 12C nuclei, which surpasses the DSNB by more than one order of magnitude. We evaluated the systematic uncertainty of NC background from the spread of a variety of data-driven models and further developed a method to determine NC background within 15\% with {\it{in}} {\it{situ}} measurements after ten years of running. Besides, the NC-like backgrounds can be effectively suppressed by the intrinsic pulse-shape discrimination (PSD) capabilities of liquid scintillators. In this talk, I will present in detail the improvements on NC background uncertainty evaluation, PSD discriminator development, and finally, the potential of DSNB sensitivity in JUNO

An Active Path-Associated Cache Scheme for Mobile Scenes

With the widespread growth of mass content, information-centric networks (ICN) have become one of the research hotspots of future network architecture. One of the important features of ICN is ubiquitous in-network caching. In recent years, the explosive growth of mobile devices has brought content dynamics, which poses a new challenge to the original ICN caching mechanism. This paper focuses on the WiFi mobile scenario of ICN. We design a new path-associated active caching scheme to shorten the time delay of users obtaining content to enhance the user experience. In this article, based on the WiFi scenario, we first propose a solution for neighbor access point selection from a theoretical perspective, considering the caching cost and transition probability. The cache content is then forwarded based on the selected neighbor set. For cached content, we propose content freshness according to mobile characteristics and consider content popularity at the same time. For cache nodes, we focus on the size of the remaining cache space and the number of hops from the cache to the user. We have implemented this strategy based on the value of caching on the forwarding path. The simulation results show that our caching strategy has a significant improvement in performance compared with active caching and other caching strategies

Study on Selection of Pressure Regulating Valve for Hydropower Station with Both Surge Tank and Pressure Regulating Valve

The transient process in the hydropower station with both surge tank and pressure regulating valve is quite complicated and also critical to operation safety. According to the pressure regulating valve working principle, the influence of the valve diameter on the unit speed and spiral case pressure was analyzed theoretically. Mathematical models of the surge tank and pressure regulating valve in the hydropower station were established based on the characteristic method. In a practical engineering, numerical simulation of large fluctuation and hydraulic disturbance transient process are conducted, verifying the correctness of the theoretical analysis. Based on the calculation results, three principles for selecting the valve diameter are concluded: first, making sure the unit speed meet the regulating guarantee requirements when guide vanes fast close; second, the maximum spiral case pressure of two times should be approximate to each other by controlling the superposition of surge wave and water hammer; third, the maximum flow of the valve should be as close to the rated flow of the turbine as possible. The principles are helpful for selecting the valve diameter in similar hydropower station

Field Measurements of Residential Energy Consumption and Indoor Thermal Environment in Six Chinese Cities

energie