442 research outputs found
SALSA-TEXT : self attentive latent space based adversarial text generation
Inspired by the success of self attention mechanism and Transformer
architecture in sequence transduction and image generation applications, we
propose novel self attention-based architectures to improve the performance of
adversarial latent code- based schemes in text generation. Adversarial latent
code-based text generation has recently gained a lot of attention due to their
promising results. In this paper, we take a step to fortify the architectures
used in these setups, specifically AAE and ARAE. We benchmark two latent
code-based methods (AAE and ARAE) designed based on adversarial setups. In our
experiments, the Google sentence compression dataset is utilized to compare our
method with these methods using various objective and subjective measures. The
experiments demonstrate the proposed (self) attention-based models outperform
the state-of-the-art in adversarial code-based text generation.Comment: 10 pages, 3 figures, under review at ICLR 201
Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
Sentence matching is widely used in various natural language tasks such as
natural language inference, paraphrase identification, and question answering.
For these tasks, understanding logical and semantic relationship between two
sentences is required but it is yet challenging. Although attention mechanism
is useful to capture the semantic relationship and to properly align the
elements of two sentences, previous methods of attention mechanism simply use a
summation operation which does not retain original features enough. Inspired by
DenseNet, a densely connected convolutional network, we propose a
densely-connected co-attentive recurrent neural network, each layer of which
uses concatenated information of attentive features as well as hidden features
of all the preceding recurrent layers. It enables preserving the original and
the co-attentive feature information from the bottommost word embedding layer
to the uppermost recurrent layer. To alleviate the problem of an
ever-increasing size of feature vectors due to dense concatenation operations,
we also propose to use an autoencoder after dense concatenation. We evaluate
our proposed architecture on highly competitive benchmark datasets related to
sentence matching. Experimental results show that our architecture, which
retains recurrent and attentive features, achieves state-of-the-art
performances for most of the tasks.Comment: Accepted at AAAI 201
Deep Learning for Recommender Systems
The widespread adoption of the Internet has led to an explosion in the number of choices available to consumers. Users begin to expect personalized content in modern E-commerce, entertainment and social media platforms. Recommender Systems (RS) provide a critical solution to this problem by maintaining user engagement and satisfaction with personalized content.
Traditional RS techniques are often linear limiting the expressivity required to model complex user-item interactions and require extensive handcrafted features from domain experts. Deep learning demonstrated significant breakthroughs in solving problems that have alluded the artificial intelligence community for many years advancing state-of-the-art results in domains such as computer vision and natural language processing.
The recommender domain consists of heterogeneous and semantically rich data such as unstructured text (e.g. product descriptions), categorical attributes (e.g. genre of a movie), and user-item feedback (e.g. purchases). Deep learning can automatically capture the intricate structure of user preferences by encoding learned feature representations from high dimensional data.
In this thesis, we explore five novel applications of deep learning-based techniques to address top-n recommendation. First, we propose Collaborative Memory Network, which unifies the strengths of the latent factor model and neighborhood-based methods inspired by Memory Networks to address collaborative filtering with implicit feedback. Second, we propose Neural Semantic Personalized Ranking, a novel probabilistic generative modeling approach to integrate deep neural network with pairwise ranking for the item cold-start problem. Third, we propose Attentive Contextual Denoising Autoencoder augmented with a context-driven attention mechanism to integrate arbitrary user and item attributes. Fourth, we propose a flexible encoder-decoder architecture called Neural Citation Network, embodying a powerful max time delay neural network encoder augmented with an attention mechanism and author networks to address context-aware citation recommendation. Finally, we propose a generic framework to perform conversational movie recommendations which leverages transfer learning to infer user preferences from natural language. Comprehensive experiments validate the effectiveness of all five proposed models against competitive baseline methods and demonstrate the successful adaptation of deep learning-based techniques to the recommendation domain
- …