End-to-end Learning for Short Text Expansion

Effectively making sense of short texts is a critical task for many real world applications such as search engines, social media services, and recommender systems. The task is particularly challenging as a short text contains very sparse information, often too sparse for a machine learning algorithm to pick up useful signals. A common practice for analyzing short text is to first expand it with external information, which is usually harvested from a large collection of longer texts. In literature, short text expansion has been done with all kinds of heuristics. We propose an end-to-end solution that automatically learns how to expand short text to optimize a given learning task. A novel deep memory network is proposed to automatically find relevant information from a collection of longer documents and reformulate the short text through a gating mechanism. Using short text classification as a demonstrating task, we show that the deep memory network significantly outperforms classical text expansion methods with comprehensive experiments on real world data sets.Comment: KDD'201

Neural Document Expansion with User Feedback

This paper presents a neural document expansion approach (NeuDEF) that enriches document representations for neural ranking models. NeuDEF harvests expansion terms from queries which lead to clicks on the document and weights these expansion terms with learned attention. It is plugged into a standard neural ranker and learned end-to-end. Experiments on a commercial search log demonstrate that NeuDEF significantly improves the accuracy of state-of-the-art neural rankers and expansion methods on queries with different frequencies. Further studies show the contribution of click queries and learned expansion weights, as well as the influence of document popularity of NeuDEF's effectiveness.Comment: The 2019 ACM SIGIR International Conference on the Theory of Information Retrieva

Joint Modeling of Local and Global Temporal Dynamics for Multivariate Time Series Forecasting with Missing Values

Multivariate time series (MTS) forecasting is widely used in various domains, such as meteorology and traffic. Due to limitations on data collection, transmission, and storage, real-world MTS data usually contains missing values, making it infeasible to apply existing MTS forecasting models such as linear regression and recurrent neural networks. Though many efforts have been devoted to this problem, most of them solely rely on local dependencies for imputing missing values, which ignores global temporal dynamics. Local dependencies/patterns would become less useful when the missing ratio is high, or the data have consecutive missing values; while exploring global patterns can alleviate such problems. Thus, jointly modeling local and global temporal dynamics is very promising for MTS forecasting with missing values. However, work in this direction is rather limited. Therefore, we study a novel problem of MTS forecasting with missing values by jointly exploring local and global temporal dynamics. We propose a new framework LGnet, which leverages memory network to explore global patterns given estimations from local perspectives. We further introduce adversarial training to enhance the modeling of global temporal distribution. Experimental results on real-world datasets show the effectiveness of LGnet for MTS forecasting with missing values and its robustness under various missing ratios.Comment: Accepted by AAAI 202

A Survey of Natural Language Generation

This paper offers a comprehensive review of the research on Natural Language Generation (NLG) over the past two decades, especially in relation to data-to-text generation and text-to-text generation deep learning methods, as well as new applications of NLG technology. This survey aims to (a) give the latest synthesis of deep learning research on the NLG core tasks, as well as the architectures adopted in the field; (b) detail meticulously and comprehensively various NLG tasks and datasets, and draw attention to the challenges in NLG evaluation, focusing on different evaluation methods and their relationships; (c) highlight some future emphasis and relatively recent research issues that arise due to the increasing synergy between NLG and other artificial intelligence areas, such as computer vision, text and computational creativity.Comment: Accepted by ACM Computing Survey (CSUR) 202

Klasifikasi Berita Pada Twitter Dengan Menggunakan Metode Naϊve Bayes Dan Feature Expansion Berbasis Cosine Similarity

Informasi telah menjadi hal yang sangat dibutuhkan di era modern ini, terlebih dengan adanya berbagai media sosial yang mendukung perbaruan informasi. Twitter sebagai salah satu media sosial yang aktif digunakan untuk memperbarui informasi yang tergolong dalam short text atau berita pendek yang memiliki beberapa kesulitan ketika dilakukan klasifikasi, seperti kata yang ambigu, kata yang terdapat dalam data uji tidak pernah muncul dalam data latih dan sebagainya. Penelitian ini dilakukan untuk mengetahui pengaruh penggunaan feature expansion atau penambahan kata pada short text dalam hasil klasifikasi. Sebelum dilakukan klasifikasi, terlebih dahulu data yang akan diujikan ditambahkan dengan daftar kata yang telah dibuat sebelumnya sebagai sumber eksternal atau kamus dengan batasan tertentu yang telah ditetapkan. Batasan ini bertujuan untuk mengetahui nilai batasan minimal yang paling optimal dalam menghasilkan akurasi tertinggi dalam proses klasifikasi. Dalam proses pembuatan sumber eksternal dilakukan proses cosine similarity untuk mencari kedekatan antar kata. Hasil penelitian berupa akurasi yang menunjukkan adanya pengaruh penambahan feature expansion dalam hasil klasifikasi, hasil akurasi sebesar 83% pada klasifikasi tanpa penggunaan feature expansion dan meningkat menjadi 87% pada penggunaan feature expansion dengan nilai threshold 0,9