A Unified Framework for Multi-Domain CTR Prediction via Large Language Models

Click-Through Rate (CTR) prediction is a crucial task in online recommendation platforms as it involves estimating the probability of user engagement with advertisements or items by clicking on them. Given the availability of various services like online shopping, ride-sharing, food delivery, and professional services on commercial platforms, recommendation systems in these platforms are required to make CTR predictions across multiple domains rather than just a single domain. However, multi-domain click-through rate (MDCTR) prediction remains a challenging task in online recommendation due to the complex mutual influence between domains. Traditional MDCTR models typically encode domains as discrete identifiers, ignoring rich semantic information underlying. Consequently, they can hardly generalize to new domains. Besides, existing models can be easily dominated by some specific domains, which results in significant performance drops in the other domains (i.e. the "seesaw phenomenon"). In this paper, we propose a novel solution Uni-CTR to address the above challenges. Uni-CTR leverages a backbone Large Language Model (LLM) to learn layer-wise semantic representations that capture commonalities between domains. Uni-CTR also uses several domain-specific networks to capture the characteristics of each domain. Note that we design a masked loss strategy so that these domain-specific networks are decoupled from backbone LLM. This allows domain-specific networks to remain unchanged when incorporating new or removing domains, thereby enhancing the flexibility and scalability of the system significantly. Experimental results on three public datasets show that Uni-CTR outperforms the state-of-the-art (SOTA) MDCTR models significantly. Furthermore, Uni-CTR demonstrates remarkable effectiveness in zero-shot prediction. We have applied Uni-CTR in industrial scenarios, confirming its efficiency.Comment: submited to TOI

Proteomic analysis of elite soybean Jidou17 and its parents using iTRAQ-based quantitative approaches

BACKGROUND: Derived from Hobbit as the female parent and Zao5241 as the male parent, the elite soybean cultivar Jidou17 is significantly higher yielding and shows enhanced qualities and stronger resistance to non-biological stress than its parents. The purpose of this study is to understand the difference in protein expression patterns between Jidou17 and its parental strains and to evaluate the parental contributions to its elite traits. RESULTS: Leaves (14 days old) from Jidou17 and its parental cultivars were analysed for differential expressed proteins using an iTRAQ-based (isobaric tags for relative and absolute quantitation) method. A total of 1269 proteins was detected, with 141 and 181 proteins in Jidou17 differing from its female and male parent, respectively. Functional classification and an enrichment analysis based on biological functions, biological processes, and cellular components revealed that all the differential proteins fell into many functional categories but that the number of proteins varied greatly for the different categories, with enrichment in specific categories. A pathway analysis indicated that the differentiated proteins were mainly classified into the ribosome assembly pathway. Protein expression clustering results showed that the expression profiles between Jidou17 and its female parent Hobbit were more similar than those between Jidou17 and its male parent Zao5241 and between the two parental strains. Therefore, the female parent Hobbit contributed more to the Jidou17 genotype. CONCLUSIONS: This study applied a proven technique to study proteomics in 14-day-old soybean leaves and explored the depth and breadth of soybean protein research. The results provide new data for further understanding the mechanisms of elite cultivar development

Genetic variation and marker−trait association affect the genomic selection prediction accuracy of soybean protein and oil content

IntroductionGenomic selection (GS) is a potential breeding approach for soybean improvement.MethodsIn this study, GS was performed on soybean protein and oil content using the Ridge Regression Best Linear Unbiased Predictor (RR-BLUP) based on 1,007 soybean accessions. The SoySNP50K SNP dataset of the accessions was obtained from the USDA-ARS, Beltsville, MD lab, and the protein and oil content of the accessions were obtained from GRIN.ResultsOur results showed that the prediction accuracy of oil content was higher than that of protein content. When the training population size was 100, the prediction accuracies for protein content and oil content were 0.60 and 0.79, respectively. The prediction accuracy increased with the size of the training population. Training populations with similar phenotype or with close genetic relationships to the prediction population exhibited better prediction accuracy. A greatest prediction accuracy for both protein and oil content was observed when approximately 3,000 markers with -log10(P) greater than 1 were included.DiscussionThis information will help improve GS efficiency and facilitate the application of GS

Prediction of peak energy demand and timestamping in commercial supermarkets using deep learning

Peak demand consumption is an ongoing research topic due to current environmental concerns. Accurate prediction of peak demand leads to improved power storage scheduling and smart grid management. However, existing researches on peak demand in commercial buildings lack focus on the timestamps for the incident of peak consumption. For this reason, this research proposes to label three indexes per day and use the novel Energy Peaks and Timestamping Prediction (EPTP) framework to detect the energy peaks as well as the timestamps for the occurring indexes. The EPTP framework is proposed with three phases. In the first phase, data preprocessing cleans the raw data into the intended input for the deep learning model, and timestamp labelling creates the expected output for training and evaluation of the model. The second phase focuses on energy consumption prediction using Long Short-Term Memory (LSTM) network, which is dedicated to processing sequential data. The last phase uses Multilayer Perceptron (MLP) for the purpose of timestamp prediction. The EPTP framework is evaluated using various data resolutions and compared to the common label of using block maxima from extreme value theory. Specifically, the two-hour hit rate improves from 21\% using the block maxima approach to 52.6\% with the proposed EPTP framework, and from 65.3\% to 86\% for the 1-hour resolution and the 15-minute resolution, respectively. In addition, the average minute deviation decreases from 120 minutes using the block maxima approach to 62 minutes with the proposed EPTP framework for the high-resolution data. The framework shows adequate results from high-resolution data using real-world commercial supermarket energy consumption

Advanced attack and defense techniques in machine learning systems

The security of machine learning systems has become a great concern in many real-world applications involving adversaries, including spam filtering, malware detection and e-commerce. There is an increasing trend of study on the security of machine learning systems but the current research is still far from satisfactory. Towards building secure machine learning systems, the first step is to study their vulnerability, which turns out to be very challenging due to the variety and complexity of machine learning systems. Combating adversaries in machine learning systems is even more challenging due to the strategic behavior of the adversaries. This thesis studies both the adversarial threats and the defenses in real-world machine learning systems. Regarding the adversarial threats, we begin by studying label contamination attacks, which is an important type of data poisoning attacks. Then we generalize the conventional data poisoning attacks on single-task learning models to multi-task learning models. Regarding defending against real-world attacks, we first study the spear phishing attacks in email systems and propose a framework for optimizing the personalized email filtering thresholds to mitigate such attacks. Then, we study the fraud transactions in e-commerce systems and propose a deep reinforcement learning based impression allocation mechanism for combating fraudulent sellers. The specific contributions of this thesis are listed below. First, regarding the label contamination attacks, we develop a Projected Gradient Ascent (PGA) algorithm to compute attacks on a family of empirical risk minimizations and show that an attack on one victim model can also be effective on other victim models. This makes it possible that the attacker designs an attack against a substitute model and transfers it to a black-box victim model. Based on the observation of the transferability, we develop a defense algorithm to identify the data points that are most likely to be attacked. Empirical studies show that PGA significantly outperforms existing baselines and linear learning models are better substitute models than nonlinear ones. Second, in the study of data poisoning attacks on muti-task learning models, we formulate the problem of computing optimal poisoning attacks on Multi-Task Relationship Learning (MTRL) as a bilevel program that is adaptive to arbitrary choice of \emph{target} tasks and \emph{attacking} tasks. We propose an efficient algorithm called PATOM for computing optimal attack strategies. PATOM leverages the optimality conditions of the subproblem of MTRL to compute the implicit gradients of the upper level objective function. Experimental results on real-world datasets show that MTRL models are very sensitive to poisoning attacks and the attacker can significantly degrade the performance of target tasks, by either directly poisoning the target tasks or indirectly poisoning the related tasks exploiting the task relatedness. We also found that the tasks being attacked are always strongly correlated, which provides a clue for defending against such attacks. Third, on defending against spear phishing email attacks, we consider two important extensions of the previous threat models. First, we consider the cases where multiple users provide access to the same information or credential. Second, we consider attackers who make sequential attack plans based on the outcome of previous attacks. Our analysis starts from scenarios where there is only one credential and then extends to more general scenarios with multiple credentials. For single-credential scenarios, we demonstrate that the optimal defense strategy can be found by solving a binary combinatorial optimization problem called PEDS. For multiple-credential scenarios, we formulate it as a bilevel optimization problem for finding the optimal defense strategy and then reduce it to a single level optimization problem called PEMS using complementary slackness conditions. Experimental results show that both PEDS and PEMS lead to significant higher defender utilities than two existing benchmarks in different parameter settings. Also, both PEDS and PEMS are more robust than the existing benchmarks considering uncertainties. Fourth, on combating fraudulent sellers in e-commerce platforms, we focus on improving the platform's impression allocation mechanism to maximize its profit and reduce the sellers' fraudulent behaviors simultaneously. First, we learn a seller behavior model to predict the sellers' fraudulent behaviors from the real-world data provided by one of the largest e-commerce company in the world. Then, we formulate the platform's impression allocation problem as a continuous Markov Decision Process (MDP) with unbounded action space. In order to make the action executable in practice and facilitate learning, we propose a novel deep reinforcement learning algorithm DDPG-ANP that introduces an action norm penalty to the reward function. Experimental results show that our algorithm significantly outperforms existing baselines in terms of scalability and solution quality.Doctor of Philosoph

Single-Channel Blind Source Separation of Spatial Aliasing Signal Based on Stacked-LSTM

Aiming at the problem of insufficient separation accuracy of aliased signals in space Internet satellite-ground communication scenarios, a stacked long short-term memory network (Stacked-LSTM) separation method based on deep learning is proposed. First, the coding feature representation of the mixed signal is extracted. Then, the long sequence input is divided into smaller blocks through the Stacked-LSTM network with the attention mechanism of the SE module, and the deep feature mask of the source signal is trained to obtain the Hadamard product of the mask of each source and the coding feature of the mixed signal, which is the encoding feature representation of the source signal. Finally, characteristics of the source signal is decoded by 1-D convolution to to obtain the original waveform. The negative scale-invariant source-to-noise ratio (SISNR) is used as the loss function of network training, that is, the evaluation index of single-channel blind source separation performance. The results show that in the single-channel separation of spatially aliased signals, the Stacked-LSTM method improves SISNR by 10.09∼38.17 dB compared with the two classic separation algorithms of ICA and NMF and the three deep learning separation methods of TasNet, Conv-TasNet and Wave-U-Net. The Stacked-LSTM method has better separation accuracy and noise robustness

Optimizing Personalized Email Filtering Thresholds to Mitigate Sequential Spear Phishing Attacks

Highly targeted spear phishing attacks are increasingly common, and have been implicated in many major security breeches. Email filtering systems are the first line of defense against such attacks. These filters are typically configured with uniform thresholds for deciding whether or not to allow a message to be delivered to a user. However, users have very significant differences in both their susceptibility to phishing attacks as well as their access to critical information and credentials that can cause damage. Recent work has considered setting personalized thresholds for individual users based on a Stackelberg game model. We consider two important extensions of the previous model. First, in our model user values can be substitutable, modeling cases where multiple users provide access to the same information or credential. Second, we consider attackers who make sequential attack plans based on the outcome of previous attacks. Our analysis starts from scenarios where there is only one credential and then extends to more general scenarios with multiple credentials. For single-credential scenarios, we demonstrate that the optimal defense strategy can be found by solving a binary combinatorial optimization problem called PEDS. For multiple-credential scenarios, we formulate it as a bilevel optimization problem for finding the optimal defense strategy and then reduce it to a single level optimization problem called PEMS using complementary slackness conditions. Experimental results show that both PEDS and PEMS lead to significant higher defender utilities than two existing benchmarks in different parameter settings. Also, both PEDS and PEMS are more robust than the existing benchmarks considering uncertainties