Customer purchase behavior prediction in E-commerce: a conceptual framework and research agenda

Digital retailers are experiencing an increasing number of transactions coming from their consumers online, a consequence of the convenience in buying goods via E-commerce platforms. Such interactions compose complex behavioral patterns which can be analyzed through predictive analytics to enable businesses to understand consumer needs. In this abundance of big data and possible tools to analyze them, a systematic review of the literature is missing. Therefore, this paper presents a systematic literature review of recent research dealing with customer purchase prediction in the E-commerce context. The main contributions are a novel analytical framework and a research agenda in the field. The framework reveals three main tasks in this review, namely, the prediction of customer intents, buying sessions, and purchase decisions. Those are followed by their employed predictive methodologies and are analyzed from three perspectives. Finally, the research agenda provides major existing issues for further research in the field of purchase behavior prediction online

Improving E-Commerce Recommendations using High Utility Sequential Patterns of Historical Purchase and Click Stream Data

Recommendation systems not only aim to recommend products that suit the taste of consumers but also generate higher revenue and increase customer loyalty for e-commerce companies (such as Amazon, Netflix). Recommendation systems can be improved if user purchase behaviour are used to improve the user-item matrix input to Collaborative Filtering (CF). This matrix is mostly sparse as in real-life, a customer would have bought only very few products from the hundreds of thousands of products in the e-commerce shelf. Thus, existing systems like Kim11Rec, HPCRec18 and HSPRec19 systems use the customer behavior information to improve the accuracy of recommendations. Kim11Rec system used behavior and navigations patterns which were not used earlier. HPCRec18 system used purchase frequency and consequential bond between click and purchased data to improve the user-item frequency matrix. The HSPRec19 system converts historic click and purchase data to sequential data and enhances the user-item frequency matrix with the sequential pattern rules mined from the sequential data for input to the CF. HSPRec19 system generates recommendations based on frequent sequential purchase patterns and does not capture whether the recommended items are also of high utility to the seller (e.g., are more profitable?).The thesis proposes a system called High Utility Sequential Pattern Recommendation System (HUSRec System), which is an extension to the HSPRec19 system that replaces frequent sequential patterns with use of high utility sequential patterns. The proposed HUSRec generates a high utility sequential database from ACM RecSys Challenge dataset using the HUSDBG (High Utility Sequential Database Generator) and HUSPM (High Utility Sequential Pattern Miner) mines the high utility sequential pattern rules which can yield high sales profits for the seller based on quantity and price of items on daily basis, as they have at least the minimum sequence utility. This improves the accuracy of the recommendations. The proposed HUSRec mines clicks sequential data using PrefixSpan algorithm to give frequent sequential rules to suggest items where no purchase has happened, decreasing the sparsity of user-item matrix, improving the user-item matrix for input to the collaborative filtering. Experimental results with mean absolute error, precision and graphs show that the proposed HUSRec system provides more accurate recommendations and higher revenue than the tested existing systems. Keywords: Data mining, Sequential pattern mining, Collaborative filtering, High utility pattern mining, E-commerce recommendation systems

News Session-Based Recommendations using Deep Neural Networks

News recommender systems are aimed to personalize users experiences and help them to discover relevant articles from a large and dynamic search space. Therefore, news domain is a challenging scenario for recommendations, due to its sparse user profiling, fast growing number of items, accelerated item's value decay, and users preferences dynamic shift. Some promising results have been recently achieved by the usage of Deep Learning techniques on Recommender Systems, specially for item's feature extraction and for session-based recommendations with Recurrent Neural Networks. In this paper, it is proposed an instantiation of the CHAMELEON -- a Deep Learning Meta-Architecture for News Recommender Systems. This architecture is composed of two modules, the first responsible to learn news articles representations, based on their text and metadata, and the second module aimed to provide session-based recommendations using Recurrent Neural Networks. The recommendation task addressed in this work is next-item prediction for users sessions: "what is the next most likely article a user might read in a session?" Users sessions context is leveraged by the architecture to provide additional information in such extreme cold-start scenario of news recommendation. Users' behavior and item features are both merged in an hybrid recommendation approach. A temporal offline evaluation method is also proposed as a complementary contribution, for a more realistic evaluation of such task, considering dynamic factors that affect global readership interests like popularity, recency, and seasonality. Experiments with an extensive number of session-based recommendation methods were performed and the proposed instantiation of CHAMELEON meta-architecture obtained a significant relative improvement in top-n accuracy and ranking metrics (10% on Hit Rate and 13% on MRR) over the best benchmark methods.Comment: Accepted for the Third Workshop on Deep Learning for Recommender Systems - DLRS 2018, October 02-07, 2018, Vancouver, Canada. https://recsys.acm.org/recsys18/dlrs

A Survey of Sequential Pattern Based E-Commerce Recommendation Systems

E-commerce recommendation systems usually deal with massive customer sequential databases, such as historical purchase or click stream sequences. Recommendation systems’ accuracy can be improved if complex sequential patterns of user purchase behavior are learned by integrating sequential patterns of customer clicks and/or purchases into the user–item rating matrix input of collaborative filtering. This review focuses on algorithms of existing E-commerce recommendation systems that are sequential pattern-based. It provides a comprehensive and comparative performance analysis of these systems, exposing their methodologies, achievements, limitations, and potential for solving more important problems in this domain. The review shows that integrating sequential pattern mining of historical purchase and/or click sequences into a user–item matrix for collaborative filtering can (i) improve recommendation accuracy, (ii) reduce user–item rating data sparsity, (iii) increase the novelty rate of recommendations, and (iv) improve the scalability of recommendation systems

Automating Software Customization via Crowdsourcing using Association Rule Mining and Markov Decision Processes

As systems grow in size and complexity so do their configuration possibilities. Users of modern systems are easy to be confused and overwhelmed by the amount of choices they need to make in order to fit their systems to their exact needs. In this thesis, we propose a technique to select what information to elicit from the user so that the system can recommend the maximum number of personalized configuration items. Our method is based on constructing configuration elicitation dialogs through utilizing crowd wisdom. A set of configuration preferences in form of association rules is first mined from a crowd configuration data set. Possible configuration elicitation dialogs are then modeled through a Markov Decision Processes (MDPs). Within the model, association rules are used to automatically infer configuration decisions based on knowledge already elicited earlier in the dialog. This way, an MDP solver can search for elicitation strategies which maximize the expected amount of automated decisions, reducing thereby elicitation effort and increasing user confidence of the result. We conclude by reporting results of a case study in which this method is applied to the privacy configuration of Facebook

Skipping-Based Collaborative Recommendations inspired from Statistical Language Modeling

Due to the almost unlimited resource space on the Web, efficient search engines and recommender systems have become a key element for users to find resources corresponding to their needs. Recommender systems aims at helping users in this task by providing them some pertinent resources according to their context and their profiles, by applying various techniques such as statistical and knowledge discovery algorithms. One of the most successful approaches is Collaborative Filtering, which consists in considering user ratings to provide recommendations, without considering the content of the resources; however the ratings are the only criterion taken into account to provide the recommendations, although including some other criterion should enhance their accuracy. One such criterion is the context, which can be geographical, meteorological, social, etc. In this chapter we focus on the temporal context, more specifically on the order in which the resources were consulted. The appropriateness of considering the order is domain dependent: for instance, it seems of little help in domains such as online moviestores, in which user transactions are barely sequential; however it is especially appropriate for domains such as Web navigation, which has a sequential structure. We propose to follow this direction for this domain, the challenge being to find a low enough complexity sequential model while providing a better accuracy. We first put forward similarities between Web navigation and natural language, and propose to adapt statistical language models to Web navigation to compute recommendations. Second, we propose a new model inspired from the n-gram skipping model. This model has several advantages: (1) It has both a low time and a low space complexity while providing a full coverage, (2) it is able to handle parallel navigations and noise, (3) it is able to perform recommendations in an anytime framework, (4) weighting schemes are used to alleviate the importance of distant resources. Third, we provide a comparison of this SLM inspired model to the state of the art in terms of features, complexity, accuracy and robustness and present experimental results. Tests are performed on a browsing dataset extracted from Intranet logs provided by a French bank. Results show that the use of exponential decay weighting schemes when taking into account non contiguous resources highly improves the accuracy, and that the anytime configuration is able to provide a satisfying trade-off between an even lower computation time and a good accuracy while conserving a good coverage

New accurate, explainable, and unbiased machine learning models for recommendation with implicit feedback.

Recommender systems have become ubiquitous Artificial Intelligence (AI) tools that play an important role in filtering online information in our daily lives. Whether we are shopping, browsing movies, or listening to music online, AI recommender systems are working behind the scene to provide us with curated and personalized content, that has been predicted to be relevant to our interest. The increasing prevalence of recommender systems has challenged researchers to develop powerful algorithms that can deliver recommendations with increasing accuracy. In addition to the predictive accuracy of recommender systems, recent research has also started paying attention to their fairness, in particular with regard to the bias and transparency of their predictions. This dissertation contributes to advancing the state of the art in fairness in AI by proposing new Machine Learning models and algorithms that aim to improve the user\u27s experience when receiving recommendations, with a focus that is positioned at the nexus of three objectives, namely accuracy, transparency, and unbiasedness of the predictions. In our research, we focus on state-of-the-art Collaborative Filtering (CF) recommendation approaches trained on implicit feedback data. More specifically, we address the limitations of two established deep learning approaches in two distinct recommendation settings, namely recommendation with user profiles and sequential recommendation. First, we focus on a state of the art pairwise ranking model, namely Bayesian Personalized Ranking (BPR), which has been found to outperform pointwise models in predictive accuracy in the recommendation with the user profiles setting. Specifically, we address two limitations of BPR: (1) BPR is a black box model that does not explain its outputs, thus limiting the user\u27s trust in the recommendations, and the analyst\u27s ability to scrutinize a model\u27s outputs; and (2) BPR is vulnerable to exposure bias due to the data being Missing Not At Random (MNAR). This exposure bias usually translates into an unfairness against the least popular items because they risk being under-exposed by the recommender system. We propose a novel explainable loss function and a corresponding model called Explainable Bayesian Personalized Ranking (EBPR) that generates recommendations along with item-based explanations. Then, we theoretically quantify the additional exposure bias resulting from the explainability, and use it as a basis to propose an unbiased estimator for the ideal EBPR loss. This being done, we perform an empirical study on three real-world benchmarking datasets that demonstrate the advantages of our proposed models, compared to existing state of the art techniques. Next, we shift our attention to sequential recommendation systems and focus on modeling and mitigating exposure bias in BERT4Rec, which is a state-of-the-art recommendation approach based on bidirectional transformers. The bi-directional representation capacity in BERT4Rec is based on the Cloze task, a.k.a. Masked Language Model, which consists of predicting randomly masked items within the sequence, assuming that the true interacted item is the most relevant one. This results in an exposure bias, where non-interacted items with low exposure propensities are assumed to be irrelevant. Thus far, the most common approach to mitigating exposure bias in recommendation has been Inverse Propensity Scoring (IPS), which consists of down-weighting the interacted predictions in the loss function in proportion to their propensities of exposure, yielding a theoretically unbiased learning. We first argue and prove that IPS does not extend to sequential recommendation because it fails to account for the sequential nature of the problem. We then propose a novel propensity scoring mechanism, that we name Inverse Temporal Propensity Scoring (ITPS), which is used to theoretically debias the Cloze task in sequential recommendation. We also rely on the ITPS framework to propose a bidirectional transformer-based model called ITPS-BERT4Rec. Finally, we empirically demonstrate the debiasing capabilities of our proposed approach and its robustness to the severity of exposure bias. Our proposed explainable approach in recommendation with user profiles, EBPR, showed an increase in ranking accuracy of about 4% and an increase in explainability of about 7% over the baseline BPR model when performing experiments on real-world recommendation datasets. Moreover, experiments on a real-world unbiased dataset demonstrated the importance of coupling explainability and exposure debiasing in capturing the true preferences of the user with a significant improvement of 1% over the baseline unbiased model UBPR. Furthermore, coupling explainability with exposure debiasing was also empirically proven to mitigate popularity bias with an improvement in popularity debiasing metrics of over 10% on three real-world recommendation tasks over the unbiased UBPR model. These results demonstrate the viability of our proposed approaches in recommendation with user profiles and their capacity to improve the user\u27s experience in recommendation by better capturing and modeling their true preferences, improving the explainability of the recommendations, and presenting them with more diverse recommendations that span a larger portion of the item catalog. On the other hand, our proposed approach in sequential recommendation ITPS-BERT4Rec has demonstrated a significant increase of 1% in terms of modeling the true preferences of the user in a semi-synthetic setting over the state-of-the-art sequential recommendation model BERT4Rec while also being unbiased in terms of exposure. Similarly, ITPS-BERT4Rec showed an average increase of 8.7% over BERT4Rec in three real-world recommendation settings. Moreover, empirical experiments demonstrated the robustness of our proposed ITPS-BERT4Rec model to increasing levels of exposure bias and its stability in terms of variance. Furthermore, experiments on popularity debiasing showed a significant advantage of our proposed ITPS-BERT4Rec model for both the short and long term sequences. Finally, ITPS-BERT4Rec showed respective improvements of around 60%, 470%, and 150% over vanilla BERT4Rec in capturing the temporal dependencies between the items within the sequences of interactions for three different evaluation metrics. These results demonstrate the potential of our proposed unbiased estimator to improve the user experience in the context of sequential recommendation by presenting them with more accurate and diverse recommendations that better match their true preferences and the sequential dependencies between the recommended items