Better, Faster, Stronger Sequence Tagging Constituent Parsers

Sequence tagging models for constituent parsing are faster, but less accurate than other types of parsers. In this work, we address the following weaknesses of such constituent parsers: (a) high error rates around closing brackets of long constituents, (b) large label sets, leading to sparsity, and (c) error propagation arising from greedy decoding. To effectively close brackets, we train a model that learns to switch between tagging schemes. To reduce sparsity, we decompose the label set and use multi-task learning to jointly learn to predict sublabels. Finally, we mitigate issues from greedy decoding through auxiliary losses and sentence-level fine-tuning with policy gradient. Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebanks, and reduce their parsing time even further. On the SPMRL datasets, we observe even greater improvements across the board, including a new state of the art on Basque, Hebrew, Polish and Swedish.Comment: NAACL 2019 (long papers). Contains corrigendu

Towards Knowledge-Based Personalized Product Description Generation in E-commerce

Quality product descriptions are critical for providing competitive customer experience in an e-commerce platform. An accurate and attractive description not only helps customers make an informed decision but also improves the likelihood of purchase. However, crafting a successful product description is tedious and highly time-consuming. Due to its importance, automating the product description generation has attracted considerable interests from both research and industrial communities. Existing methods mainly use templates or statistical methods, and their performance could be rather limited. In this paper, we explore a new way to generate the personalized product description by combining the power of neural networks and knowledge base. Specifically, we propose a KnOwledge Based pErsonalized (or KOBE) product description generation model in the context of e-commerce. In KOBE, we extend the encoder-decoder framework, the Transformer, to a sequence modeling formulation using self-attention. In order to make the description both informative and personalized, KOBE considers a variety of important factors during text generation, including product aspects, user categories, and knowledge base, etc. Experiments on real-world datasets demonstrate that the proposed method out-performs the baseline on various metrics. KOBE can achieve an improvement of 9.7% over state-of-the-arts in terms of BLEU. We also present several case studies as the anecdotal evidence to further prove the effectiveness of the proposed approach. The framework has been deployed in Taobao, the largest online e-commerce platform in China.Comment: KDD 2019 Camera-ready. Website: https://sites.google.com/view/kobe201

Data-driven Methods for Course Selection and Sequencing

University of Minnesota Ph.D. dissertation.May 2019. Major: Computer Science. Advisor: George Karypis. 1 computer file (PDF); xiii, 115 pages.Learning analytics in higher education is an emerging research field that combines data mining, machine learning, statistics, and education on learning-related data, in order to develop methods that can improve the learning environment for learners and allow educators and administrators to be more effective. The vast amount of data available about students' interactions and their performance in classrooms has motivated researchers to analyze this data in order to gain insights about the learning environment for the ultimate goal of improving undergraduate education and student retention rates. In this thesis, we focus on the problem of course selection and sequencing, where we would like to help students make informed decisions about which courses to register for in their following terms. By analyzing the historical enrollment and grades data, this thesis studies the two main problems of course selection and sequencing, namely grade prediction and course recommendation. In addition, it analyzes the relationship between degree planning in terms of course timing and ordering and the students' GPA and time to degree. First, we focus on predicting the grades that students will obtain on future courses so that they can make informed decisions about which courses to register for in their following terms. We model the grade prediction problem as cumulative knowledge-based linear regression models that learn the courses' required and provided knowledge components and use them to estimate a student's knowledge state at each term and predict the grades that he/she can obtain on future courses. Second, we focus on improving the knowledge-based regression models we previously developed by modeling the complex interactions among prior courses using non-linear and neural attentive models, in order to have more accurate estimation of a student's knowledge state. In addition, we model the interactions between a target course, which we would like to predict its grade, and the other courses taken concurrently with it. We hypothesize that concurrently-taken courses can affect a student's performance in a target course, and thus modeling their interactions with that course should lead to better predictions. Third, we focus on analyzing the degree plans of students to gain more insights about how course timing and sequencing relate to their GPAs and time to degree. Toward this end, we define several course timing and course sequencing metrics and compare different sub-groups of students who have achieved high vs low GPA as well as sub-groups of students who have graduated on time vs over time. Fourth, we focus on improving course recommendation by recommending to each student a set of courses which he/she is prepared for and expected to perform well in. We model this problem as a grade-aware course recommendation problem, where we propose two different approaches. The first approach ranks the courses by using an objective function that differentiates between courses that are expected to increase or decrease a student's GPA. The second approach combines the grades predicted by grade prediction methods with the rankings produced by course recommendation methods to improve the final course rankings. To obtain the course rankings in both approaches, we adapted two widely-used representation learning techniques to learn the optimal temporal ordering between courses. In summary, this thesis addresses two closely related problems by: (1) developing cumulative knowledge-based regression models for grade prediction; % (2) developing context-aware non-linear and neural attentive knowledge-based models for grade prediction; % (3) analyzing degree planning and how the time when students take courses and how they sequence them relate to their GPAs and time to degree; and % (4) developing novel approaches for grade-aware course recommendation.

Improving Patch-Based Convolutional Neural Networks for MRI Brain Tumor Segmentation by Leveraging Location Information.

The manual brain tumor annotation process is time consuming and resource consuming, therefore, an automated and accurate brain tumor segmentation tool is greatly in demand. In this paper, we introduce a novel method to integrate location information with the state-of-the-art patch-based neural networks for brain tumor segmentation. This is motivated by the observation that lesions are not uniformly distributed across different brain parcellation regions and that a locality-sensitive segmentation is likely to obtain better segmentation accuracy. Toward this, we use an existing brain parcellation atlas in the Montreal Neurological Institute (MNI) space and map this atlas to the individual subject data. This mapped atlas in the subject data space is integrated with structural Magnetic Resonance (MR) imaging data, and patch-based neural networks, including 3D U-Net and DeepMedic, are trained to classify the different brain lesions. Multiple state-of-the-art neural networks are trained and integrated with XGBoost fusion in the proposed two-level ensemble method. The first level reduces the uncertainty of the same type of models with different seed initializations, and the second level leverages the advantages of different types of neural network models. The proposed location information fusion method improves the segmentation performance of state-of-the-art networks including 3D U-Net and DeepMedic. Our proposed ensemble also achieves better segmentation performance compared to the state-of-the-art networks in BraTS 2017 and rivals state-of-the-art networks in BraTS 2018. Detailed results are provided on the public multimodal brain tumor segmentation (BraTS) benchmarks

AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling

In Click-through rate (CTR) prediction models, a user's interest is usually represented as a fixed-length vector based on her history behaviors. Recently, several methods are proposed to learn an attentive weight for each user behavior and conduct weighted sum pooling. However, these methods only manually select several fields from the target item side as the query to interact with the behaviors, neglecting the other target item fields, as well as user and context fields. Directly including all these fields in the attention may introduce noise and deteriorate the performance. In this paper, we propose a novel model named AutoAttention, which includes all item/user/context side fields as the query, and assigns a learnable weight for each field pair between behavior fields and query fields. Pruning on these field pairs via these learnable weights lead to automatic field pair selection, so as to identify and remove noisy field pairs. Though including more fields, the computation cost of AutoAttention is still low due to using a simple attention function and field pair selection. Extensive experiments on the public dataset and Tencent's production dataset demonstrate the effectiveness of the proposed approach.Comment: Accepted by ICDM 202