Towards More Efficient Insertion Transformer with Fractional Positional Encoding

Auto-regressive neural sequence models have been shown to be effective across text generation tasks. However, their left-to-right decoding order prevents generation from being parallelized. Insertion Transformer (Stern et al., 2019) is an attractive alternative that allows outputting multiple tokens in a single generation step. Nevertheless, due to the incompatibility between absolute positional encoding and insertion-based generation schemes, it needs to refresh the encoding of every token in the generated partial hypothesis at each step, which could be costly. We design a novel reusable positional encoding scheme for insertion transformers called Fractional Positional Encoding (FPE), which allows reusing representations calculated in previous steps. Empirical studies on various text generation tasks demonstrate the effectiveness of FPE, which leads to floating-point operation reduction and latency improvements on batched decoding

Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification

Implicit discourse relation classification is of great challenge due to the lack of connectives as strong linguistic cues, which motivates the use of annotated implicit connectives to improve the recognition. We propose a feature imitation framework in which an implicit relation network is driven to learn from another neural network with access to connectives, and thus encouraged to extract similarly salient features for accurate classification. We develop an adversarial model to enable an adaptive imitation scheme through competition between the implicit network and a rival feature discriminator. Our method effectively transfers discriminability of connectives to the implicit features, and achieves state-of-the-art performance on the PDTB benchmark.Comment: To appear in ACL201

Fast and Accurate Neural Word Segmentation for Chinese

Neural models with minimal feature engineering have achieved competitive performance against traditional methods for the task of Chinese word segmentation. However, both training and working procedures of the current neural models are computationally inefficient. This paper presents a greedy neural word segmenter with balanced word and character embedding inputs to alleviate the existing drawbacks. Our segmenter is truly end-to-end, capable of performing segmentation much faster and even more accurate than state-of-the-art neural models on Chinese benchmark datasets.Comment: To appear in ACL201

Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training

In this work we propose a pragmatic method that reduces the annotation cost for structured label spaces using active learning. Our approach leverages partial annotation, which reduces labeling costs for structured outputs by selecting only the most informative sub-structures for annotation. We also utilize self-training to incorporate the current model's automatic predictions as pseudo-labels for un-annotated sub-structures. A key challenge in effectively combining partial annotation with self-training to reduce annotation cost is determining which sub-structures to select to label. To address this challenge, we adopt an error estimator to adaptively decide the partial selection ratio according to the current model's capability. In evaluations spanning four structured prediction tasks, we show that our combination of partial annotation and self-training using an adaptive selection ratio reduces annotation cost over strong full annotation baselines under a fair comparison scheme that takes reading time into consideration.Comment: Findings of EMNLP 202

Prognostic Outcomes and Risk Factors for Patients with Renal Cell Carcinoma and Venous Tumor Thrombus after Radical Nephrectomy and Thrombectomy: The Prognostic Significance of Venous Tumor Thrombus Level.

IntroductionTo evaluate the prognostic outcomes and risk factors for renal cell carcinoma (RCC) patients with venous tumor thrombus in China.Materials and methodsWe reviewed the clinical information of 169 patients who underwent radical nephrectomy and thrombectomy. Overall and cancer-specific survival rates were analyzed. Univariate and multivariate analyses were used to investigate the potential prognostic factors.ResultsThe median survival time was 63 months. The five-year overall survival and cancer-specific survival rate were 53.6% and 54.4% for all patients. For all patients, significant survival difference was only observed between early (below hepatic vein) and advanced (above hepatic vein) tumor thrombus. However, significant differences existed between both RV/IVC and early/advanced tumor thrombus groups in N0M0 patients. Multivariate analysis demonstrated that higher tumor thrombus level (p = 0.016, RR = 1.58), N (p = 0.013, RR = 2.60), and M (p < 0.001, RR = 4.14) stages and adrenal gland invasion (p = 0.001, RR = 4.91) were the most significant negative prognostic predictors.ConclusionsIn this study, we reported most cases of RCC patients with venous extension in China. We proved that patients with RCC and venous tumor thrombus may have relative promising long-term survival rate, especially those with early tumor thrombus