5,527 research outputs found
Which Channel to Ask My Question? Personalized Customer Service Request Stream Routing using Deep Reinforcement Learning
Customer services are critical to all companies, as they may directly connect
to the brand reputation. Due to a great number of customers, e-commerce
companies often employ multiple communication channels to answer customers'
questions, for example, chatbot and hotline. On one hand, each channel has
limited capacity to respond to customers' requests, on the other hand,
customers have different preferences over these channels. The current
production systems are mainly built based on business rules, which merely
considers tradeoffs between resources and customers' satisfaction. To achieve
the optimal tradeoff between resources and customers' satisfaction, we propose
a new framework based on deep reinforcement learning, which directly takes both
resources and user model into account. In addition to the framework, we also
propose a new deep-reinforcement-learning based routing method-double dueling
deep Q-learning with prioritized experience replay (PER-DoDDQN). We evaluate
our proposed framework and method using both synthetic and a real customer
service log data from a large financial technology company. We show that our
proposed deep-reinforcement-learning based framework is superior to the
existing production system. Moreover, we also show our proposed PER-DoDDQN is
better than all other deep Q-learning variants in practice, which provides a
more optimal routing plan. These observations suggest that our proposed method
can seek the trade-off where both channel resources and customers' satisfaction
are optimal.Comment: 13 pages, 7 figure
User-oriented recommender systems in retail
User satisfaction is considered a key objective for all service provider platforms, regardless of the nature of the service, encompassing domains such as media, entertainment, retail, and information. While the goal of satisfying users is the same across different domains and services, considering domain-specific characteristics is of paramount importance to ensure users have a positive experience with a given system. User interaction data with a system is one of the main sources of data that facilitates achieving this goal. In this thesis, we investigate how to learn from domain-specific user interactions. We focus on recommendation as our main task, and retail as our main domain. We further explore the finance domain and the demand forecasting task as additional directions to understand whether our methodology and findings generalize to other tasks and domains. The research in this thesis is organized around the following dimensions: 1) Characteristics of multi-channel retail: we consider a retail setting where interaction data comes from both digital (i.e., online) and in-store (i.e., offline) shopping; 2) From user behavior to recommendation: we conduct extensive descriptive studies on user interaction log datasets that inform the design of recommender systems in two domains, retail and finance. Our key contributions in characterizing multi-channel retail are two-fold. First, we propose a neural model that makes use of sales in multiple shopping channels in order to improve the performance of demand forecasting in a target channel. Second, we provide the first study of user behavior in a multi-channel retail setting, which results in insights about the channel-specific properties of user behavior, and their effects on the performance of recommender systems. We make three main contributions in designing user-oriented recommender systems. First, we provide a large-scale user behavior study in the finance domain, targeted at understanding financial information seeking behavior in user interactions with company filings. We then propose domain-specific user-oriented filing recommender systems that are informed by the findings of the user behavior analysis. Second, we analyze repurchasing behavior in retail, specifically in the grocery shopping domain. We then propose a repeat consumption-aware neural recommender for this domain. Third, we focus on scalable recommendation in retail and propose an efficient recommender system that explicitly models users' personal preferences that are reflected in their purchasing history
User-oriented recommender systems in retail
User satisfaction is considered a key objective for all service provider platforms, regardless of the nature of the service, encompassing domains such as media, entertainment, retail, and information. While the goal of satisfying users is the same across different domains and services, considering domain-specific characteristics is of paramount importance to ensure users have a positive experience with a given system. User interaction data with a system is one of the main sources of data that facilitates achieving this goal. In this thesis, we investigate how to learn from domain-specific user interactions. We focus on recommendation as our main task, and retail as our main domain. We further explore the finance domain and the demand forecasting task as additional directions to understand whether our methodology and findings generalize to other tasks and domains. The research in this thesis is organized around the following dimensions: 1) Characteristics of multi-channel retail: we consider a retail setting where interaction data comes from both digital (i.e., online) and in-store (i.e., offline) shopping; 2) From user behavior to recommendation: we conduct extensive descriptive studies on user interaction log datasets that inform the design of recommender systems in two domains, retail and finance. Our key contributions in characterizing multi-channel retail are two-fold. First, we propose a neural model that makes use of sales in multiple shopping channels in order to improve the performance of demand forecasting in a target channel. Second, we provide the first study of user behavior in a multi-channel retail setting, which results in insights about the channel-specific properties of user behavior, and their effects on the performance of recommender systems. We make three main contributions in designing user-oriented recommender systems. First, we provide a large-scale user behavior study in the finance domain, targeted at understanding financial information seeking behavior in user interactions with company filings. We then propose domain-specific user-oriented filing recommender systems that are informed by the findings of the user behavior analysis. Second, we analyze repurchasing behavior in retail, specifically in the grocery shopping domain. We then propose a repeat consumption-aware neural recommender for this domain. Third, we focus on scalable recommendation in retail and propose an efficient recommender system that explicitly models users' personal preferences that are reflected in their purchasing history
Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition
Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters’ influence on performance to provide insights about their optimisation
Hierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systems
Recommending novel content, which expands user horizons by introducing them
to new interests, has been shown to improve users' long-term experience on
recommendation platforms \cite{chen2021values}. Users however are not
constantly looking to explore novel content. It is therefore crucial to
understand their novelty-seeking intent and adjust the recommendation policy
accordingly. Most existing literature models a user's propensity to choose
novel content or to prefer a more diverse set of recommendations at individual
interactions. Hierarchical structure, on the other hand, exists in a user's
novelty-seeking intent, which is manifested as a static and intrinsic user
preference for seeking novelty along with a dynamic session-based propensity.
To this end, we propose a novel hierarchical reinforcement learning-based
method to model the hierarchical user novelty-seeking intent, and to adapt the
recommendation policy accordingly based on the extracted user novelty-seeking
propensity. We further incorporate diversity and novelty-related measurement in
the reward function of the hierarchical RL (HRL) agent to encourage user
exploration \cite{chen2021values}. We demonstrate the benefits of explicitly
modeling hierarchical user novelty-seeking intent in recommendations through
extensive experiments on simulated and real-world datasets. In particular, we
demonstrate that the effectiveness of our proposed hierarchical RL-based method
lies in its ability to capture such hierarchically-structured intent. As a
result, the proposed HRL model achieves superior performance on several public
datasets, compared with state-of-art baselines
Mixed Information Flow for Cross-domain Sequential Recommendations
Cross-domain sequential recommendation is the task of predict the next item
that the user is most likely to interact with based on past sequential behavior
from multiple domains. One of the key challenges in cross-domain sequential
recommendation is to grasp and transfer the flow of information from multiple
domains so as to promote recommendations in all domains. Previous studies have
investigated the flow of behavioral information by exploring the connection
between items from different domains. The flow of knowledge (i.e., the
connection between knowledge from different domains) has so far been neglected.
In this paper, we propose a mixed information flow network for cross-domain
sequential recommendation to consider both the flow of behavioral information
and the flow of knowledge by incorporating a behavior transfer unit and a
knowledge transfer unit. The proposed mixed information flow network is able to
decide when cross-domain information should be used and, if so, which
cross-domain information should be used to enrich the sequence representation
according to users' current preferences. Extensive experiments conducted on
four e-commerce datasets demonstrate that mixed information flow network is
able to further improve recommendation performance in different domains by
modeling mixed information flow.Comment: 26 pages, 6 figures, TKDD journal, 7 co-author
- …