Deep Task-specific Bottom Representation Network for Multi-Task Recommendation

Neural-based multi-task learning (MTL) has gained significant improvement, and it has been successfully applied to recommendation system (RS). Recent deep MTL methods for RS (e.g. MMoE, PLE) focus on designing soft gating-based parameter-sharing networks that implicitly learn a generalized representation for each task. However, MTL methods may suffer from performance degeneration when dealing with conflicting tasks, as negative transfer effects can occur on the task-shared bottom representation. This can result in a reduced capacity for MTL methods to capture task-specific characteristics, ultimately impeding their effectiveness and hindering the ability to generalize well on all tasks. In this paper, we focus on the bottom representation learning of MTL in RS and propose the Deep Task-specific Bottom Representation Network (DTRN) to alleviate the negative transfer problem. DTRN obtains task-specific bottom representation explicitly by making each task have its own representation learning network in the bottom representation modeling stage. Specifically, it extracts the user's interests from multiple types of behavior sequences for each task through the parameter-efficient hypernetwork. To further obtain the dedicated representation for each task, DTRN refines the representation of each feature by employing a SENet-like network for each task. The two proposed modules can achieve the purpose of getting task-specific bottom representation to relieve tasks' mutual interference. Moreover, the proposed DTRN is flexible to combine with existing MTL methods. Experiments on one public dataset and one industrial dataset demonstrate the effectiveness of the proposed DTRN.Comment: CIKM'2

Efficient Optimal Selection for Composited Advertising Creatives with Tree Structure

Ad creatives are one of the prominent mediums for online e-commerce advertisements. Ad creatives with enjoyable visual appearance may increase the click-through rate (CTR) of products. Ad creatives are typically handcrafted by advertisers and then delivered to the advertising platforms for advertisement. In recent years, advertising platforms are capable of instantly compositing ad creatives with arbitrarily designated elements of each ingredient, so advertisers are only required to provide basic materials. While facilitating the advertisers, a great number of potential ad creatives can be composited, making it difficult to accurately estimate CTR for them given limited real-time feedback. To this end, we propose an Adaptive and Efficient ad creative Selection (AES) framework based on a tree structure. The tree structure on compositing ingredients enables dynamic programming for efficient ad creative selection on the basis of CTR. Due to limited feedback, the CTR estimator is usually of high variance. Exploration techniques based on Thompson sampling are widely used for reducing variances of the CTR estimator, alleviating feedback sparsity. Based on the tree structure, Thompson sampling is adapted with dynamic programming, leading to efficient exploration for potential ad creatives with the largest CTR. We finally evaluate the proposed algorithm on the synthetic dataset and the real-world dataset. The results show that our approach can outperform competing baselines in terms of convergence rate and overall CTR

Continual Learning in Predictive Autoscaling

Predictive Autoscaling is used to forecast the workloads of servers and prepare the resources in advance to ensure service level objectives (SLOs) in dynamic cloud environments. However, in practice, its prediction task often suffers from performance degradation under abnormal traffics caused by external events (such as sales promotional activities and applications re-configurations), for which a common solution is to re-train the model with data of a long historical period, but at the expense of high computational and storage costs. To better address this problem, we propose a replay-based continual learning method, i.e., Density-based Memory Selection and Hint-based Network Learning Model (DMSHM), using only a small part of the historical log to achieve accurate predictions. First, we discover the phenomenon of sample overlap when applying replay-based continual learning in prediction tasks. In order to surmount this challenge and effectively integrate new sample distribution, we propose a density-based sample selection strategy that utilizes kernel density estimation to calculate sample density as a reference to compute sample weight, and employs weight sampling to construct a new memory set. Then we implement hint-based network learning based on hint representation to optimize the parameters. Finally, we conduct experiments on public and industrial datasets to demonstrate that our proposed method outperforms state-of-the-art continual learning methods in terms of memory capacity and prediction accuracy. Furthermore, we demonstrate remarkable practicability of DMSHM in real industrial applications

Prompt-augmented Temporal Point Process for Streaming Event Sequence

Neural Temporal Point Processes (TPPs) are the prevalent paradigm for modeling continuous-time event sequences, such as user activities on the web and financial transactions. In real-world applications, event data is typically received in a \emph{streaming} manner, where the distribution of patterns may shift over time. Additionally, \emph{privacy and memory constraints} are commonly observed in practical scenarios, further compounding the challenges. Therefore, the continuous monitoring of a TPP to learn the streaming event sequence is an important yet under-explored problem. Our work paper addresses this challenge by adopting Continual Learning (CL), which makes the model capable of continuously learning a sequence of tasks without catastrophic forgetting under realistic constraints. Correspondingly, we propose a simple yet effective framework, PromptTPP\footnote{Our code is available at {\small \url{ https://github.com/yanyanSann/PromptTPP}}}, by integrating the base TPP with a continuous-time retrieval prompt pool. The prompts, small learnable parameters, are stored in a memory space and jointly optimized with the base TPP, ensuring that the model learns event streams sequentially without buffering past examples or task-specific attributes. We present a novel and realistic experimental setup for modeling event streams, where PromptTPP consistently achieves state-of-the-art performance across three real user behavior datasets.Comment: NeurIPS 2023 camera ready versio

Glycyrrhizin Treatment Facilitates Extinction of Conditioned Fear Responses After a Single Prolonged Stress Exposure in Rats

Background/Aims: Impaired fear memory extinction is widely considered a key mechanism of post-traumatic stress disorder (PTSD). Recent studies have suggested that neuroinflammation after a single prolonged stress (SPS) exposure may play a critical role in the impaired fear memory extinction. Studies have shown that high mobility group box chromosomal protein 1 (HMGB-1) is critically involved in neuroinflammation. However, the role of HMGB-1 underlying the development of impairment of fear memory extinction is still not known. Methods: Thus, we examined the levels of HMGB-1 in the basolateral amygdala (BLA) following SPS using Western blot and evaluated the levels of microglia and astrocytes activation in the BLA after SPS using immunohistochemical staining. We then examined the effects of pre-SPS intra-BLA administration of glycyrrhizin, an HMGB1 inhibitor, or LPS-RS, a competitive TLR4 antagonist, on subsequent post-SPS fear extinction. Results: We found that SPS treatment prolonged the extinction of contextual fear memory after the SPS. The impairment of SPS-induced extinction of contextual fear memory was associated with increased HMGB1 and Toll-like receptor 4 (TLR4) levels in the BLA. Additionally, the impairment of SPS-induced extinction of contextual fear memory was associated with increased activation of microglia and astrocyte in the BLA. Intra-BLA administrations of glycyrrhizin (HMGB-1 inhibitor) or LPS-RS (TLR4 antagonist) can prevent the development of SPS-induced fear extinction impairment. Conclusion: Taken together, these results suggested that SPS treatment may not only produce short term effects on the HMGB1/TLR4-mediated pro-inflammation, but alter the response of microglia and astrocytes to the exposure to fear associated contextual stimuli

Mechanism of Secondary Breakage in the Overlying Strata during Repetitious Mining of an Ultrathick Coal Seam in Design Stage

When designing the mining of an ultrathick coal seam, the laws governing movement in the overlying strata during mining are a fundamental issue based on which several problems are addressed, including determining the mining method and the roadway arrangement, controlling the surrounding strata, and selecting the devices. The present paper considers possible problems related to strata overlying a large mining space subjected to repeated disturbances during the mining of an ultrathick coal seam, including repeatedly broken strata and the existence or inexistence of the structure. The BM coal seam in the No. 2 coal mine of the Dajing mining area in the East Junggar coalfield is studied. Physical simulations are performed on the movements of the overlying strata during slicing mining of the ultrathick coal seam, revealing the new feature of “break-joint stability-instability-secondary breakage” in the overlying strata. Mechanical models are constructed of the secondary breakage of the overlying strata blocks under both static and impact loading, and mechanical criteria are proposed for such breakage. Based on the research findings, methods for controlling the surrounding strata during slicing mining of an ultrathick coal seam are proposed, including increasing the mining rate and designing reasonable heights for the slicing mining

An Empirical Study of Carbon Emission Calculation in the Production and Construction Phase of A Prefabricated Office Building from Zhejiang, China

This study analyzes an office building located in Hangzhou, Zhejiang region, with a high assembly rate of 96.8%. Based on whole-process records and first-hand factory data, using an original method, we empirically investigate the carbon emissions associated to the assembly production and construction phase by comparing the results collected in the field with the calculation results for the simulated non-prefabricated building. The calculation results show that the production and construction stage of the prefabricated office building is characterized by a large reduction in carbon emissions, where the total measured carbon emissions of the subject building were 2265.73 tCO2e, which is 22 kgCO2e/m2 less than that under the non-prefabricated method. In the future development of China’s construction industry, taking Zhejiang Province as an example, the implementation of prefabricated office buildings with a PEC structure system can effectively reduce carbon emissions, which can help China to achieve the carbon peak as soon as possible