Automated Recommender Systems

Recommender systems have been existing accompanying by web development, driving personalized experience for billions of users. They play a vital role in the information retrieval process, overcome the information overload by facilitating the communication between business people and the public, and boost the business world. Powered by the advances of machine learning techniques, modern recommender systems enable tremendous automation on the data preprocessing, information distillations, and contextual inferences. It allows us to mine patterns and relationships from massive datasets and various data resources to make inferences. Moreover, the fast evolvement of deep learning techniques brings vast vitality and improvements dived in both academic research and industry applications. Despite the prominence achieved in the recent recommender systems, the automation they have been achieved is still limited in a narrow scope. On the one hand, beyond the static setting, real-world recommendation tasks are often imbued with high-velocity streaming data. On the other hand, with the increasing complexity of model structure and system architecture, the handcrafted design and tuning process is becoming increasingly complicated and time-consuming. With these challenges in mind, this dissertation aims to enable advanced automation in recommender systems. In particular, we discuss how to update factorization-based recommendation models adaptively and how to automatically design and tune recommendation models with automated machine learning techniques. Four main contributions are made via tackling the challenges: (1) The first contribution of this research dissertation is the development of a tensor-based algorithm for streaming recommendation tasks. (2) As deep learning techniques have shown their superiority in recommendation tasks and become dominant in both academia and industry applications, the second contribution is exploring and developing advanced deep learning algorithms to tackle the recommendation problem with the streaming dataset. (3) To alleviate the burden of human efforts, we explore adopting automated machine learning in designing and tuning recommender systems. The third contribution of this dissertation is the development of a novel neural architecture search approaches for discovering useful features interactions and designing better models for the click-through rate prediction problem. (4) Considering a large number of recommendation tasks in industrial applications and their similarities, in the last piece of work work, we focus on the hyperparameter tuning problem in the transfer-learning setting and develop a transferable framework for meta-level tuning of machine learning models

Modeling Events and Interactions through Temporal Processes -- A Survey

In real-world scenario, many phenomena produce a collection of events that occur in continuous time. Point Processes provide a natural mathematical framework for modeling these sequences of events. In this survey, we investigate probabilistic models for modeling event sequences through temporal processes. We revise the notion of event modeling and provide the mathematical foundations that characterize the literature on the topic. We define an ontology to categorize the existing approaches in terms of three families: simple, marked, and spatio-temporal point processes. For each family, we systematically review the existing approaches based based on deep learning. Finally, we analyze the scenarios where the proposed techniques can be used for addressing prediction and modeling aspects.Comment: Image replacement

A novel Auto-ML Framework for Sarcasm Detection

Many domains have sarcasm or verbal irony presented in the text of reviews, tweets, comments, and dialog discussions. The purpose of this research is to classify sarcasm for multiple domains using the deep learning based AutoML framework. The proposed AutoML framework has five models in the model search pipeline, these five models are the combination of convolutional neural network (CNN), Long Short-Term Memory (LSTM), deep neural network (DNN), and Bidirectional Long Short-Term Memory (BiLSTM). The hybrid combination of CNN, LSTM, and DNN models are presented as CNN-LSTM-DNN, LSTM-DNN, BiLSTM-DNN, and CNN-BiLSTM-DNN. This work has proposed the algorithms that contrast polarities between terms and phrases, which are categorized into implicit and explicit incongruity categories. The incongruity and pragmatic features like punctuation, exclamation marks, and others integrated into the AutoML DeepConcat framework models. That integration was possible when the DeepConcat AutoML framework initiate a model search pipeline for five models to achieve better performance. Conceptually, DeepConcat means that model will integrate with generalized features. It was evident that the pretrain model BiLSTM achieved a better performance of 0.98 F1 when compared with the other five model performances. Similarly, the AutoML based BiLSTM-DNN model achieved the best performance of 0.98 F1, which is better than core approaches and existing state-of-the-art Tweeter tweet dataset, Amazon reviews, and dialog discussion comments. The proposed AutoML framework has compared performance metrics F1 and AUC and discovered that F1 is better than AUC. The integration of all feature categories achieved a better performance than the individual category of pragmatic and incongruity features. This research also evaluated the performance of the dropout layer hyperparameter and it achieved better performance than the fixed percentage like 10% of dropout parameter of the AutoML based Bayesian optimization. Proposed AutoML framework DeepConcat evaluated best pretrain models BiLSTM-DNN and CNN-CNN-DNN to transfer knowledge across domains like Amazon reviews and Dialog discussion comments (text) using the last strategy, full layer, and our fade-out freezing strategies. In the transfer learning fade-out strategy outperformed the existing state-of-the-art model BiLSTM-DNN, the performance is 0.98 F1 on tweets, 0.85 F1 on Amazon reviews, and 0.87 F1 on the dialog discussion SCV2-Gen dataset. Further, all strategies with various domains can be compared for the best model selection

Uncertainty-guided Boundary Learning for Imbalanced Social Event Detection

Real-world social events typically exhibit a severe class-imbalance distribution, which makes the trained detection model encounter a serious generalization challenge. Most studies solve this problem from the frequency perspective and emphasize the representation or classifier learning for tail classes. While in our observation, compared to the rarity of classes, the calibrated uncertainty estimated from well-trained evidential deep learning networks better reflects model performance. To this end, we propose a novel uncertainty-guided class imbalance learning framework - UCL$_{SED}$, and its variant - UCL-EC$_{SED}$, for imbalanced social event detection tasks. We aim to improve the overall model performance by enhancing model generalization to those uncertain classes. Considering performance degradation usually comes from misclassifying samples as their confusing neighboring classes, we focus on boundary learning in latent space and classifier learning with high-quality uncertainty estimation. First, we design a novel uncertainty-guided contrastive learning loss, namely UCL and its variant - UCL-EC, to manipulate distinguishable representation distribution for imbalanced data. During training, they force all classes, especially uncertain ones, to adaptively adjust a clear separable boundary in the feature space. Second, to obtain more robust and accurate class uncertainty, we combine the results of multi-view evidential classifiers via the Dempster-Shafer theory under the supervision of an additional calibration method. We conduct experiments on three severely imbalanced social event datasets including Events2012\_100, Events2018\_100, and CrisisLexT\_7. Our model significantly improves social event representation and classification tasks in almost all classes, especially those uncertain ones.Comment: Accepted by TKDE 202