Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe

Privacy concerns have attracted increasing attention in data-driven products due to the tendency of machine learning models to memorize sensitive training data. Generating synthetic versions of such data with a formal privacy guarantee, such as differential privacy (DP), provides a promising path to mitigating these privacy concerns, but previous approaches in this direction have typically failed to produce synthetic data of high quality. In this work, we show that a simple and practical recipe in the text domain is effective: simply fine-tuning a pretrained generative language model with DP enables the model to generate useful synthetic text with strong privacy protection. Through extensive empirical analyses on both benchmark and private customer data, we demonstrate that our method produces synthetic text that is competitive in terms of utility with its non-private counterpart, meanwhile providing strong protection against potential privacy leakages.Comment: ACL 2023 Main Conference (Honorable Mention

Predicative possession in Medieval Slavic Bible translations Predicative Possession in Early Biblical Slavic

Late Proto-Slavic (LPS) had an inventory of three constructions for expressing predicative possession. Using the earliest Slavic Bible translations from Old Church Slavic (OCS), and to a lesser degree Old Czech, a number of conclusions can be drawn about the status of predicative possession for LPS. The verb iměti ‘have’ was the most frequent and least syntactically and semantically restricted predicative possessive construction (PPC). Existential PPCs with a dative possessor appear primarily with kinship relations, abstract possessums, and in a number of other fixed construction types; existential PPCs with the possessor in an u + genitive prepositional phrase primarily appear with concrete and countable possessums. Both existential PPCs call for an animate, most often pronominal, possessor. The u + genitive was the rarest type of PPC in LPS, though it had undoubtedly grammaticalized as a PPC

The competing roles of SV(O) and VS(O) word orders in Xoždenie igumena Daniila

Though both Early East Slavic (EES) and Modern Russian have a relatively free word order, the distribution and function of word order in EES is quite distinct from Modern Russian. This paper is a study of word order within a single EES text, Xoždenie igumena Daniila, which is split into two major subdivisions: travel guide and narrative. In the travel guide, existential, stance, and motion verbs occur more frequently in VS order, and VS(O) order is more frequent overall; copular and transitive verbs occur more frequently in SV(O) order. Instances of the less frequent word order for the clause type occur as a result of specific conditioning contexts. The narrative, in contrast, has proportionally more SV(O) clauses and transitive verbs than the travel guide

The History of Predicative Possession in Slavic: Internal Development vs. Language Contact

The languages of the world encode possession in a variety of ways. In Slavic languages, possession on the level of the clause, or predicative possession, is represented by two main encoding strategies. Most Slavic languages, including those in the West and South Slavic sub-groupings, use a `have' verb comparable to English have and German haben. But Russian, an East Slavic language, encodes predicative possession only infrequently with its `have' verb imet'; instead, Russian uses a construction for predicative possession originating in a locative phrase, e.g. u menja est' kniga, which literally means `at me is a book' for `I have a book'. This locative construction for predicative possession in Russian is often singled out as an aberrant construction in Slavic and attributed to contact-induced influence from Finnic languages. The opposite point of view is also put forth: that the locative construction for predicative possession in Russian is the original construction inherited from Late Proto-Slavic and the `have' verb used in other Slavic languages is merely a calque from Greek. Neither explanation is entirely satisfactory. As a matter of fact, early Slavic textual traditions, based on a comparison of textual examples from Old Church Slavic, Old Serbian and Croatian, Old Czech, and Early East Slavic, reveal that both a `have' verb and a locative construction for predicative possession were used in Late Proto-Slavic, alongside a third construction with the possessor encoded in the dative case.The present-day distribution of encoding strategies in the Slavic languages is explained by tracing the different textual and population histories for multiple areas of Slavdom. Contacts with neighboring languages, especially neighboring non-Slavic languages, over the course of history influenced predicative possessive constructions (PPCs) in all areas of Slavdom. Because the Slavic languages spread rather rapidly over a vast geographic expanse, covering most of Eastern Europe in a matter of a few centuries in the latter half of the first millennium CE, the languages that different Slavic populations came into contact with were often quite different. In particular, languages in the western end of Slavdom were in contact with German-speaking populations to varying degrees of intensity; in the northeastern end of Slavdom, Early East Slavic assimilated and lived alongside large numbers of originally Finnic-speaking populations, who spoke languages closely related to Modern Finnish and Estonian. In short, each Slavic language expanded usage of one of the three original encoding strategies for predicative possession already attested in Late Proto-Slavic and the encoding strategy that expanded brought it closer to usage in neighboring and historically substrate non-Slavic languages. Not only the form of the PPCs themselves came to parallel usage in neighboring non-Slavic languages, but the morphosyntactic and semantic properties of the Slavic constructions also converged with the PPCs used in areal languages. Additional support for the scenarios put forth in this dissertation comes from examination of factors outside the domain of predicative possession, including linguistic features other than predicative possession, textual histories and considerations of language standardization in different areas, socio-historical factors, and demographic factors. While this dissertation traces the development of one grammatical category - predicative possession - in the history of Slavic, the scenarios outlined are meant to contribute more generally to an understanding of linguistic change in the history of Slavic and how those changes reflect the influence of population processes on shaping the path of historical linguistic change

Synchrony and diachrony of the Bulgarian predicative possessive constructions

The paper investigates the system of predicative possession in Bulgarian from a Slavic and Balkan perspective. The constructions are described in terms of their semantic and syntactic properties and several generalizations are made about the distribution of possessive features such as alienable vs inalienable and permanent vs temporary. In the second part of paper, I bring forward some observations about the diachrony of the Bulgarian predicative possessive constructions and their potential (Slavic or Balkan) source