In modern commercial search engines and recommendation systems, data from
multiple domains is available to jointly train the multi-domain model.
Traditional methods train multi-domain models in the multi-task setting, with
shared parameters to learn the similarity of multiple tasks, and task-specific
parameters to learn the divergence of features, labels, and sample
distributions of individual tasks. With the development of large language
models, LLM can extract global domain-invariant text features that serve both
search and recommendation tasks. We propose a novel framework called S\&R
Multi-Domain Foundation, which uses LLM to extract domain invariant features,
and Aspect Gating Fusion to merge the ID feature, domain invariant text
features and task-specific heterogeneous sparse features to obtain the
representations of query and item. Additionally, samples from multiple search
and recommendation scenarios are trained jointly with Domain Adaptive
Multi-Task module to obtain the multi-domain foundation model. We apply the
S\&R Multi-Domain foundation model to cold start scenarios in the
pretrain-finetune manner, which achieves better performance than other SOTA
transfer learning methods. The S\&R Multi-Domain Foundation model has been
successfully deployed in Alipay Mobile Application's online services, such as
content query recommendation and service card recommendation, etc.Comment: CIKM 2023,6 page