Paraphrase Generation with Deep Reinforcement Learning

Automatic generation of paraphrases from a given sentence is an important yet challenging task in natural language processing (NLP), and plays a key role in a number of applications such as question answering, search, and dialogue. In this paper, we present a deep reinforcement learning approach to paraphrase generation. Specifically, we propose a new framework for the task, which consists of a \textit{generator} and an \textit{evaluator}, both of which are learned from data. The generator, built as a sequence-to-sequence learning model, can produce paraphrases given a sentence. The evaluator, constructed as a deep matching model, can judge whether two sentences are paraphrases of each other. The generator is first trained by deep learning and then further fine-tuned by reinforcement learning in which the reward is given by the evaluator. For the learning of the evaluator, we propose two methods based on supervised learning and inverse reinforcement learning respectively, depending on the type of available training data. Empirical study shows that the learned evaluator can guide the generator to produce more accurate paraphrases. Experimental results demonstrate the proposed models (the generators) outperform the state-of-the-art methods in paraphrase generation in both automatic evaluation and human evaluation.Comment: EMNLP 201

Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands

To understand diverse natural language commands, virtual assistants today are trained with numerous labor-intensive, manually annotated sentences. This paper presents a methodology and the Genie toolkit that can handle new compound commands with significantly less manual effort. We advocate formalizing the capability of virtual assistants with a Virtual Assistant Programming Language (VAPL) and using a neural semantic parser to translate natural language into VAPL code. Genie needs only a small realistic set of input sentences for validating the neural model. Developers write templates to synthesize data; Genie uses crowdsourced paraphrases and data augmentation, along with the synthesized data, to train a semantic parser. We also propose design principles that make VAPL languages amenable to natural language translation. We apply these principles to revise ThingTalk, the language used by the Almond virtual assistant. We use Genie to build the first semantic parser that can support compound virtual assistants commands with unquoted free-form parameters. Genie achieves a 62% accuracy on realistic user inputs. We demonstrate Genie's generality by showing a 19% and 31% improvement over the previous state of the art on a music skill, aggregate functions, and access control.Comment: To appear in PLDI 201

Multilingual Unsupervised Sentence Simplification

Progress in Sentence Simplification has been hindered by the lack of supervised data, particularly in languages other than English. Previous work has aligned sentences from original and simplified corpora such as English Wikipedia and Simple English Wikipedia, but this limits corpus size, domain, and language. In this work, we propose using unsupervised mining techniques to automatically create training corpora for simplification in multiple languages from raw Common Crawl web data. When coupled with a controllable generation mechanism that can flexibly adjust attributes such as length and lexical complexity, these mined paraphrase corpora can be used to train simplification systems in any language. We further incorporate multilingual unsupervised pretraining methods to create even stronger models and show that by training on mined data rather than supervised corpora, we outperform the previous best results. We evaluate our approach on English, French, and Spanish simplification benchmarks and reach state-of-the-art performance with a totally unsupervised approach. We will release our models and code to mine the data in any language included in Common Crawl

The Steep Road to Happily Ever After: An Analysis of Current Visual Storytelling Models

Visual storytelling is an intriguing and complex task that only recently entered the research arena. In this work, we survey relevant work to date, and conduct a thorough error analysis of three very recent approaches to visual storytelling. We categorize and provide examples of common types of errors, and identify key shortcomings in current work. Finally, we make recommendations for addressing these limitations in the future.Comment: Accepted to the NAACL 2019 Workshop on Shortcomings in Vision and Language (SiVL

The effect of editing techniques on machine translation-informed academic foreing language writing

[EN] Although the field of machine translation has witnessed huge improvements in recent years, its potentials have not been fully exploited in other interdisciplinary areas such as foreign language teaching. The aim of this paper, therefore, is to report an experiment in which this technology was employed to teach a foreign language to a group of students. This mixed-method study explores the effect of teaching editing techniques in machine translation to a group of Persian EFL university students in an academic writing course. Twenty students took part in a 4-day workshop in which one session was devoted to teaching editing techniques and three remaining sessions to the use of editing techniques, namely, correcting mistakes, removing ambiguities, simplifying structures and combining structures. Each session consisted of a pre-test, a training and a post-test. In addition, in each session, one key writing point, namely, determiners, paraphrasing and collocations were discussed. A questionnaire for candidates’ demographic information and another for learning experiences were administered. The results indicated a statistically significant improvement in the overall gain score. Further analysis showed a significant improvement in the use of determiners in contrast to paraphrasing and collocations. Lack of improvement in data driven learning in paraphrasing and collocation seemed to stem from weakness in vocabulary and grammatical knowledge in both the mother tongue and the target language. Analysis of questionnaire data revealed that the instruction proved to be beneficial since it could be easily implemented in correction and confirmation.  On the whole, it can be concluded that providing the correct type of guidance and feedback on how to use machine translation will indeed have a profound effect on foreign language writing skill.Mirzaeian, VR. (2021). The effect of editing techniques on machine translation-informed academic foreing language writing. The EuroCALL Review. 29(2):33-43. https://doi.org/10.4995/eurocall.2021.12930OJS3343292Acar, A., Geluso, J., and Shiki, T. (2011). How can search engines improve your writing? CALL-EJ, 12(1), 1−10. Retrieved from http://callej.org/journal/12-1/Acar_2011.pdfAnderson, D. D. (1995). Machine translation as a tool in second language learning. CALICO Journal, 13(1), 68−97.BNC Consortium. (2007). The British national corpus, version 3 (BNC XML Edition). Distributed by Oxford University Computing Services on behalf of the BNC Consortium. Retrieved from http://www.natcorp.ox.ac.uk/Boulton, A. (2010). Data‐driven learning: Taking the computer out of the equation. Language Learning, 60(3), 534−572. https://doi.org/10.1111/j.1467-9922.2010.00566.xBueno, J. L. (1992). Traducción automática mediante el ordenador (Automatic Translation Using a Computer). Retrieved from http://eric.ed.gov/?id=ED354715Chambers, A., and O'Sullivan, Í. (2004). Corpus consultation and advanced learners' writing skills in French. ReCALL, 16(1), 158-172. https://doi.org/10.1017/S0958344004001211Chapelle, C. A. (2003). English language learning and technology: Lectures on teaching and research in the age of information and communication. Amsterdam: John Benjamins. https://doi.org/10.1075/lllt.7Chujo, K., Oghigian, K., and Akasegawa, S. (2015). A corpus and grammatical browsing system for remedial EFL learners. In A. Leńko-Szymańska and A. Boulton (Eds.), Multiple affordances of language corpora for data-driven learning (pp. 109-128). Amsterdam: John Benjamins. https://doi.org/10.1075/scl.69.06chuConroy, M. (2010). Internet tools for language learning: University students taking control of their writing. Australasian Journal of Educational Technology, 26(6), 861−882. https://doi.org/10.14742/ajet.1047Crosthwaite, P. (2017) Retesting the limits of data-driven learning: feedback and error correction, Computer Assisted Language Learning, 30(6), 447−473. https://doi.org/10.1080/09588221.2017.1312462Davies, M. (2008). The corpus of contemporary American English (COCA): 520 million words, 1990-present. Retrieved from http://corpus.byu.edu/coca/Egbert, J. (2005). CALL essentials: Principles and practice in CALL classrooms. Alexandria, VA: TESOL Publications.Eubanks, J., Yeh, H., and Tseng, H. (2018) Learning Chinese through a twenty-first century writing workshop with the integration of mobile technology in a language immersion elementary school, Computer Assisted Language Learning, 31(4), 346−366. https://doi.org/10.1080/09588221.2017.1399911Flowerdew, L. (2015). Data-driven learning and language learning theories. In A. Leńko-Szymańska and A. Boulton (Eds.), Multiple affordances of language corpora for data-driven learning (pp. 15−36). Amsterdam: John Benjamins. https://doi.org/10.1075/scl.69.02floFranken, M. (2014). The nature and scope of student search strategies in using a web derived corpus for writing. Language Learning Journal, 42(1), 85−102. https://doi.org/10.1080/09571736.2012.678013Garcia, I., and Pena, M. I. (2011). Machine translation-assisted language learning: Writing for beginners. Computer Assisted Language Learning, 24(5), 471−487. https://doi.org/10.1080/09588221.2011.582687Geiller, L. (2014). How EFL students can use Google to correct their 'untreatable' written errors. The EuroCALL Review, 22(2), 26−45. https://doi.org/10.4995/eurocall.2014.3633Geluso, J. (2013). Phraseology and frequency of occurrence on the web: Native speakers' perceptions of Google-informed second language writing. Computer Assisted Language Learning, 26(2), 144−157. https://doi.org/10.1080/09588221.2011.639786Granger, S., Hung, J., and Petch-Tyson, S. (Eds.). (2002). Computer learner corpora, second language acquisition, and foreign language teaching. Amsterdam: John Benjamins. https://doi.org/10.1075/lllt.6Johns, T. (1991). Should you be persuaded: Two samples of data-driven learning materials. English Language Research Journal, 4, 1−16.Kern, R. (2006). Perspectives on technology in learning and teaching languages. TESOL Quarterly, 40(1), 183−210. https://doi.org/10.2307/40264516Kilgarriff, A., and Grefenstette, G. (2003). Introduction to the special issue on the web as corpus. Computational Linguistics, 29(3), 333−347. https://doi.org/10.1162/089120103322711569Lewis, D. (1997) Machine translation in a modern languages curriculum. Computer Assisted Language Learning, 10(3), pp. 255−271. https://doi.org/10.1080/0958822970100305Liu, D., and Jiang, P. (2009). Using a corpus-based lexicogrammatical approach to grammar instruction in EFL and ESL contexts. Modern Language Journal, 93(1), 61−78. https://doi.org/10.1111/j.1540-4781.2009.00828.xMilton, J. (2006). Resource-rich web-based feedback: Helping learners become independent writers. In K. Hyland and F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 123−137). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139524742.009Nesselhauf, N. (2005). Collocations in a learner corpus. Amsterdam: John Benjamins. https://doi.org/10.1075/scl.14Nino, A. (2008). Evaluating the use of machine translation post-editing in the foreign language class. Computer Assisted Language Learning, 21(1), 29−49. https://doi.org/10.1080/09588220701865482O'Neill, E. (2012). The effect of online translators on L2 writing in French (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign.O'Sullivan, Í., and Chambers, A. (2006). Learners' writing skills in French: Corpus consultation and learner evaluation. Journal of Second Language Writing, 15(1), 49-68. https://doi.org/10.1016/j.jslw.2006.01.002Park, K. (2010). Using a specialized corpus and Google custom search engine (CSE) for enhancing L2 teaching and learning. Retrieved from http://www.personal.psu.edu/xxl13/teaching/au10/588/park_2010.pdfPlonsky, L., and Oswald, F. L. (2014). How big is 'big'? Interpreting effect sizes in L2 research. Language Learning, 64, 878-912. https://doi.org/10.1111/lang.12079Reppen, R. (2010). Using corpora in the language classroom. Cambridge. UK: Cambridge University Press.Richmond, I. M. (1994) Doing it backwards: Using translation software to teach target language grammaticality. Computer Assisted Language Learning, 7(1), pp. 65−78. https://doi.org/10.1080/0958822940070106Shei, C. C. (2002) Teaching MT through pre-editing: Three case studies. Proceedings of the 6th EAMT Workshop on Teaching Machine Translation. Manchester, pp. 89−98.Shei, C.-C. (2008). Discovering the hidden treasure on the Internet: Using Google to uncover the veil of phraseology? Computer Assisted Language Learning, 21(1), 67−85. https://doi.org/10.1080/09588220701865516Smart, J. (2014). The role of guided induction in paper-based data-driven learning. ReCALL, 26(2), 184-201. https://doi.org/10.1017/S0958344014000081Somers, H., Gaspari, F., and Nino, A. (2006). Detecting inappropriate use of free online machine-translation by language students − A special case of plagiarism detection. In 11th Annual Conference of the European Association for Machine Translation-Proceedings (pp. 41−48).Todd, R. W. (2001). Induction from self-selected concordances and self-correction. System, 29, 91-102. https://doi.org/10.1016/S0346-251X(00)00047-6Yang, Y. (2018). New language knowledge construction through indirect feedback in web-based collaborative writing, Computer Assisted Language Learning, 31(4), 459−480. https://doi.org/10.1080/09588221.2017.1414852Yoon, C. (2016). Concordancers and dictionaries as problem-solving tools for ESL academic writing. Language Learning & Technology, 20(1), 209-229. Retrieved from http://llt.msu.edu/issues/february2016/yoon.pdfYoon, H., and Hirvela, A. (2004). ESL student attitudes toward corpus use in L2. Journal of Second Language Writing, 13(4), 257-283. https://doi.org/10.1016/j.jslw.2004.06.002Yoon, H., and Jo, J. W. (2014). Direct and indirect access to corpora: An exploratory case study comparing students' error correction and learning strategy use in L2 writing. Language Learning & Technology, 18(1), 96-117. Retrieved from http://llt.msu.edu/issues/february2014/yoonjo.pd

Fine-Grained Linguistic Soft Constraints on Statistical Natural Language Processing Models

This dissertation focuses on effective combination of data-driven natural language processing (NLP) approaches with linguistic knowledge sources that are based on manual text annotation or word grouping according to semantic commonalities. I gainfully apply fine-grained linguistic soft constraints -- of syntactic or semantic nature -- on statistical NLP models, evaluated in end-to-end state-of-the-art statistical machine translation (SMT) systems. The introduction of semantic soft constraints involves intrinsic evaluation on word-pair similarity ranking tasks, extension from words to phrases, application in a novel distributional paraphrase generation technique, and an introduction of a generalized framework of which these soft semantic and syntactic constraints can be viewed as instances, and in which they can be potentially combined. Fine granularity is key in the successful combination of these soft constraints, in many cases. I show how to softly constrain SMT models by adding fine-grained weighted features, each preferring translation of only a specific syntactic constituent. Previous attempts using coarse-grained features yielded negative results. I also show how to softly constrain corpus-based semantic models of words (“distributional profiles”) to effectively create word-sense-aware models, by using semantic word grouping information found in a manually compiled thesaurus. Previous attempts, using hard constraints and resulting in aggregated, coarse-grained models, yielded lower gains. A novel paraphrase generation technique incorporating these soft semantic constraints is then also evaluated in a SMT system. This paraphrasing technique is based on the Distributional Hypothesis. The main advantage of this novel technique over current “pivoting” techniques for paraphrasing is the independence from parallel texts, which are a limited resource. The evaluation is done by augmenting translation models with paraphrase-based translation rules, where fine-grained scoring of paraphrase-based rules yields significantly higher gains. The model augmentation includes a novel semantic reinforcement component: In many cases there are alternative paths of generating a paraphrase-based translation rule. Each of these paths reinforces a dedicated score for the “goodness” of the new translation rule. This augmented score is then used as a soft constraint, in a weighted log-linear feature, letting the translation model learn how much to “trust” the paraphrase-based translation rules. The work reported here is the first to use distributional semantic similarity measures to improve performance of an end-to-end phrase-based SMT system. The unified framework for statistical NLP models with soft linguistic constraints enables, in principle, the combination of both semantic and syntactic constraints -- and potentially other constraints, too -- in a single SMT model