2,578 research outputs found
Transfer Learning and Deep Domain Adaptation
Transfer learning is an emerging technique in machine learning, by which we can solve a new task with the knowledge obtained from an old task in order to address the lack of labeled data. In particular deep domain adaptation (a branch of transfer learning) gets the most attention in recently published articles. The intuition behind this is that deep neural networks usually have a large capacity to learn representation from one dataset and part of the information can be further used for a new task. In this research, we firstly present the complete scenarios of transfer learning according to the domains and tasks. Secondly, we conduct a comprehensive survey related to deep domain adaptation and categorize the recent advances into three types based on implementing approaches: fine-tuning networks, adversarial domain adaptation, and sample-reconstruction approaches. Thirdly, we discuss the details of these methods and introduce some typical real-world applications. Finally, we conclude our work and explore some potential issues to be further addressed
Advances in Convolutional Neural Networks
Deep Learning, also known as deep representation learning, has dramatically improved the performances on a variety of learning tasks and achieved tremendous successes in the past few years. Specifically, artificial neural networks are mainly studied, which mainly include Multilayer Perceptrons (MLPs), Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). Among these networks, CNNs got the most attention due to the kernel methods with the weight sharing mechanism, and achieved state-of-the-art in many domains, especially computer vision. In this research, we conduct a comprehensive survey related to the recent improvements in CNNs, and we demonstrate these advances from the low level to the high level, including the convolution operations, convolutional layers, architecture design, loss functions, and advanced applications
G2PTL: A Pre-trained Model for Delivery Address and its Applications in Logistics System
Text-based delivery addresses, as the data foundation for logistics systems,
contain abundant and crucial location information. How to effectively encode
the delivery address is a core task to boost the performance of downstream
tasks in the logistics system. Pre-trained Models (PTMs) designed for Natural
Language Process (NLP) have emerged as the dominant tools for encoding semantic
information in text. Though promising, those NLP-based PTMs fall short of
encoding geographic knowledge in the delivery address, which considerably trims
down the performance of delivery-related tasks in logistic systems such as
Cainiao. To tackle the above problem, we propose a domain-specific pre-trained
model, named G2PTL, a Geography-Graph Pre-trained model for delivery address in
Logistics field. G2PTL combines the semantic learning capabilities of text
pre-training with the geographical-relationship encoding abilities of graph
modeling. Specifically, we first utilize real-world logistics delivery data to
construct a large-scale heterogeneous graph of delivery addresses, which
contains abundant geographic knowledge and delivery information. Then, G2PTL is
pre-trained with subgraphs sampled from the heterogeneous graph. Comprehensive
experiments are conducted to demonstrate the effectiveness of G2PTL through
four downstream tasks in logistics systems on real-world datasets. G2PTL has
been deployed in production in Cainiao's logistics system, which significantly
improves the performance of delivery-related tasks
- …