Automated Machine Learning for Deep Recommender Systems: A Survey

Deep recommender systems (DRS) are critical for current commercial online service providers, which address the issue of information overload by recommending items that are tailored to the user's interests and preferences. They have unprecedented feature representations effectiveness and the capacity of modeling the non-linear relationships between users and items. Despite their advancements, DRS models, like other deep learning models, employ sophisticated neural network architectures and other vital components that are typically designed and tuned by human experts. This article will give a comprehensive summary of automated machine learning (AutoML) for developing DRS models. We first provide an overview of AutoML for DRS models and the related techniques. Then we discuss the state-of-the-art AutoML approaches that automate the feature selection, feature embeddings, feature interactions, and system design in DRS. Finally, we discuss appealing research directions and summarize the survey

Comparative venom gland transcriptome analysis of the scorpion Lychas mucronatus reveals intraspecific toxic gene diversity and new venomous components

<p>Abstract</p> <p>Background</p> <p><it>Lychas mucronatus </it>is one scorpion species widely distributed in Southeast Asia and southern China. Anything is hardly known about its venom components, despite the fact that it can often cause human accidents. In this work, we performed a venomous gland transcriptome analysis by constructing and screening the venom gland cDNA library of the scorpion <it>Lychas mucronatus </it>from Yunnan province and compared it with the previous results of Hainan-sourced <it>Lychas mucronatus</it>.</p> <p>Results</p> <p>A total of sixteen known types of venom peptides and proteins are obtained from the venom gland cDNA library of Yunnan-sourced <it>Lychas mucronatus</it>, which greatly increase the number of currently reported scorpion venom peptides. Interestingly, we also identified nineteen atypical types of venom molecules seldom reported in scorpion species. Surprisingly, the comparative transcriptome analysis of Yunnan-sourced <it>Lychas mucronatus </it>and Hainan-sourced <it>Lychas mucronatus </it>indicated that enormous diversity and vastly abundant difference could be found in venom peptides and proteins between populations of the scorpion <it>Lychas mucronatus </it>from different geographical regions.</p> <p>Conclusions</p> <p>This work characterizes a large number of venom molecules never identified in scorpion species. This result provides a comparative analysis of venom transcriptomes of the scorpion <it>Lychas mucronatus </it>from different geographical regions, which thoroughly reveals the fact that the venom peptides and proteins of the same scorpion species from different geographical regions are highly diversified and scorpion evolves to adapt a new environment by altering the primary structure and abundance of venom peptides and proteins.</p

Diffusion Augmentation for Sequential Recommendation

Sequential recommendation (SRS) has become the technical foundation in many applications recently, which aims to recommend the next item based on the user's historical interactions. However, sequential recommendation often faces the problem of data sparsity, which widely exists in recommender systems. Besides, most users only interact with a few items, but existing SRS models often underperform these users. Such a problem, named the long-tail user problem, is still to be resolved. Data augmentation is a distinct way to alleviate these two problems, but they often need fabricated training strategies or are hindered by poor-quality generated interactions. To address these problems, we propose a Diffusion Augmentation for Sequential Recommendation (DiffuASR) for a higher quality generation. The augmented dataset by DiffuASR can be used to train the sequential recommendation models directly, free from complex training procedures. To make the best of the generation ability of the diffusion model, we first propose a diffusion-based pseudo sequence generation framework to fill the gap between image and sequence generation. Then, a sequential U-Net is designed to adapt the diffusion noise prediction model U-Net to the discrete sequence generation task. At last, we develop two guide strategies to assimilate the preference between generated and origin sequences. To validate the proposed DiffuASR, we conduct extensive experiments on three real-world datasets with three sequential recommendation models. The experimental results illustrate the effectiveness of DiffuASR. As far as we know, DiffuASR is one pioneer that introduce the diffusion model to the recommendation

AutoAssign+: Automatic Shared Embedding Assignment in Streaming Recommendation

In the domain of streaming recommender systems, conventional methods for addressing new user IDs or item IDs typically involve assigning initial ID embeddings randomly. However, this practice results in two practical challenges: (i) Items or users with limited interactive data may yield suboptimal prediction performance. (ii) Embedding new IDs or low-frequency IDs necessitates consistently expanding the embedding table, leading to unnecessary memory consumption. In light of these concerns, we introduce a reinforcement learning-driven framework, namely AutoAssign+, that facilitates Automatic Shared Embedding Assignment Plus. To be specific, AutoAssign+ utilizes an Identity Agent as an actor network, which plays a dual role: (i) Representing low-frequency IDs field-wise with a small set of shared embeddings to enhance the embedding initialization, and (ii) Dynamically determining which ID features should be retained or eliminated in the embedding table. The policy of the agent is optimized with the guidance of a critic network. To evaluate the effectiveness of our approach, we perform extensive experiments on three commonly used benchmark datasets. Our experiment results demonstrate that AutoAssign+ is capable of significantly enhancing recommendation performance by mitigating the cold-start problem. Furthermore, our framework yields a reduction in memory usage of approximately 20-30%, verifying its practical effectiveness and efficiency for streaming recommender systems

Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation

Two-tower models are a prevalent matching framework for recommendation, which have been widely deployed in industrial applications. The success of two-tower matching attributes to its efficiency in retrieval among a large number of items, since the item tower can be precomputed and used for fast Approximate Nearest Neighbor (ANN) search. However, it suffers two main challenges, including limited feature interaction capability and reduced accuracy in online serving. Existing approaches attempt to design novel late interactions instead of dot products, but they still fail to support complex feature interactions or lose retrieval efficiency. To address these challenges, we propose a new matching paradigm named SparCode, which supports not only sophisticated feature interactions but also efficient retrieval. Specifically, SparCode introduces an all-to-all interaction module to model fine-grained query-item interactions. Besides, we design a discrete code-based sparse inverted index jointly trained with the model to achieve effective and efficient model inference. Extensive experiments have been conducted on open benchmark datasets to demonstrate the superiority of our framework. The results show that SparCode significantly improves the accuracy of candidate item matching while retaining the same level of retrieval efficiency with two-tower models. Our source code will be available at MindSpore/models.Comment: Accepted by SIGIR 2023. Code will be available at https://reczoo.github.io/SparCod

Scenario-Aware Hierarchical Dynamic Network for Multi-Scenario Recommendation

Click-Through Rate (CTR) prediction is a fundamental technique in recommendation and advertising systems. Recent studies have shown that implementing multi-scenario recommendations contributes to strengthening information sharing and improving overall performance. However, existing multi-scenario models only consider coarse-grained explicit scenario modeling that depends on pre-defined scenario identification from manual prior rules, which is biased and sub-optimal. To address these limitations, we propose a Scenario-Aware Hierarchical Dynamic Network for Multi-Scenario Recommendations (HierRec), which perceives implicit patterns adaptively and conducts explicit and implicit scenario modeling jointly. In particular, HierRec designs a basic scenario-oriented module based on the dynamic weight to capture scenario-specific information. Then the hierarchical explicit and implicit scenario-aware modules are proposed to model hybrid-grained scenario information. The multi-head implicit modeling design contributes to perceiving distinctive patterns from different perspectives. Our experiments on two public datasets and real-world industrial applications on a mainstream online advertising platform demonstrate that our HierRec outperforms existing models significantly

Migration and transformation of dissolved carbon during accumulated cyanobacteria decomposition in shallow eutrophic lakes: a simulated microcosm study

The decomposition processes of accumulated cyanobacteria can release large amounts of organic carbon and affect the carbon cycling in shallow eutrophic lakes. However, the migration and transformation mechanisms of dissolved carbon (DC) require further study and discussion. In this study, a 73-day laboratory microcosm experiment using suction samplers (Rhizon and syringe) was conducted to understand the migration and transformation of DC during the cyanobacteria decomposition. The decomposition of cyanobacteria biomass caused anoxic and reduction conditions, and changed the acid-base environment in the water column. During the early incubation (days 0–18), a large amount of cyanobacteria-derived particulate organic matter (POM) was decomposed into dissolved organic carbon (DOC) in the overlying water, reaching the highest peak value of 1.82 g L−1 in the treatment added the high cyanobacteria biomass (470 g). After 18 days of incubation, the mineralization of increased DOC to dissolved inorganic carbon (DIC) maintained a high DIC level of overlying water in treatments added cyanobacteria biomass. The treatment added the medium cyanobacteria biomass (235 g) presented the lower DOC/total dissolved carbon ratio than the high cyanobacteria biomass associated with the lower mineralization from DOC to DIC. Due to the concentration differences of DIC at water-sediment interface, the main migration of DIC from pore water to overlying water occurred in the treatment without added cyanobacteria biomass. However, the treatments added the cyanobacteria biomass presented the obvious diffusion of DOC and the low migration of DIC at the water-sediment interface. The diffusive fluxes of DOC at the water-sediment interface increased with the cyanobacteria biomass added, reaching the maximum value of 411.01 mg/(m2·d) in the treatment added the high cyanobacteria biomass. In the overlying water, the group added the sediment and medium cyanobacteria biomass presented a faster degradation of cyanobacteria-derived POM to DOC and a higher mineralization level of DOC to DIC than added the medium cyanobacteria biomass without sediment. Therefore, during accumulated cyanobacteria decomposition, the biomass of accumulated cyanobacteria and sediment property can influence the migration and transformation of DC, playing an important role in carbon cycling in shallow eutrophic lakes

SdPI, The First Functionally Characterized Kunitz-Type Trypsin Inhibitor from Scorpion Venom

Background: Kunitz-type venom peptides have been isolated from a wide variety of venomous animals. They usually have protease inhibitory activity or potassium channel blocking activity, which by virtue of the effects on predator animals are essential for the survival of venomous animals. However, no Kunitz-type peptides from scorpion venom have been functionally characterized. Principal Findings: A new Kunitz-type venom peptide gene precursor, SdPI, was cloned and characterized from a venom gland cDNA library of the scorpion Lychas mucronatus. It codes for a signal peptide of 21 residues and a mature peptide of 59 residues. The mature SdPI peptide possesses a unique cysteine framework reticulated by three disulfide bridges, different from all reported Kunitz-type proteins. The recombinant SdPI peptide was functionally expressed. It showed trypsin inhibitory activity with high potency (Ki = 1.6610 27 M) and thermostability. Conclusions: The results illustrated that SdPI is a potent and stable serine protease inhibitor. Further mutagenesis and molecular dynamics simulation revealed that SdPI possesses a serine protease inhibitory active site similar to other Kunitztype venom peptides. To our knowledge, SdPI is the first functionally characterized Kunitz-type trypsin inhibitor derive