Improving Zero-Shot Cross-lingual Transfer Between Closely Related Languages by Injecting Character-Level Noise

Cross-lingual transfer between a high-resource language and its dialects or closely related language varieties should be facilitated by their similarity. However, current approaches that operate in the embedding space do not take surface similarity into account. This work presents a simple yet effective strategy to improve cross-lingual transfer between closely related varieties. We propose to augment the data of the high-resource source language with character-level noise to make the model more robust towards spelling variations. Our strategy shows consistent improvements over several languages and tasks: Zero-shot transfer of POS tagging and topic identification between language varieties from the Finnic, West and North Germanic, and Western Romance language branches. Our work provides evidence for the usefulness of simple surface-level noise in improving transfer between language varieties

Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by Injecting Character-level Noise

Cross-lingual transfer between a high-resource language and its dialects or closely related language varieties should be facilitated by their similarity. However, current approaches that operate in the embedding space do not take surface similarity into account. This work presents a simple yet effective strategy to imrove cross-lingual transfer between closely related varieties. We propose to augment the data of the high-resource source language with character-level noise to make the model more robust towards spelling variations. Our strategy shows consistent improvements over several languages and tasks: Zero-shot transfer of POS tagging and topic identification between language varieties from the Finnic, West and North Germanic, and Western Romance language branches. Our work provides evidence for the usefulness of simple surface-level noise in improving transfer between language varieties.Comment: ACL 202

Essays in Applied Microeconomics with a Focus on Vocational Education and Training

This thesis consists of three chapters, each derived from an individual paper. Although each of these chapters deals with a unique research question, there are similarities that are strong between Chapters 1 and 2 and weak regarding the third. First, all chapters raise questions in the field of microeconomics and touch on the Swiss system of Vocational Education and Training (VET). Second, all chapters apply quantitative methods widely used in the field. And finally, all chapters are embedded in the Swiss context. Whereas this Swissness merely concerns the data in chapter three, it additionally includes the outcome of interest and the fundamentals of the empirical methodology in chapter one and two. Both Chapters 1 and 2 investigate firms' engagement in the Swiss VET system. Within this system firms' voluntary participation is crucial – they help design curricula, hire apprentices, pay their wages, and are responsible for most of their training – and remarkable: Setting a world record, Switzerland's dual VET system accepts around 60% of all pupils after compulsory schooling each year. It thus seems fair to say that Swiss firms bear a large proportion of the investments needed to secure their own future skill demand. Chapter 1, which is joint work with Andreas Kuhn, investigates what happens to this voluntary engagement in the skill formation process if firms are permitted to secure their skill demand from another source: we focus on immigration. In recent years, this channel has changed substantially, both quantitatively and qualitatively. Through mutual agreements, foremost the Agreement on the Free Movement of Persons implemented in 2002, the labor markets of Switzerland and the European Union have increasingly integrated, and the non-Swiss workforce has grown by roughly 50% or 600,000 workers since 1995. Meanwhile, most immigrant workers today hold a vocational or tertiary degree, whereas in the 1990s fewer reached educational attainments higher than compulsory schooling. Overall, we hypothesize that this enlargement of skills provided by the non-native to the Swiss labor market may incentivizes firms to substitute their investments in VET by hiring immigrant workers. To examine this claim empirically, Chapter 1 focuses on crossborder workers, who work in but live outside Switzerland. Today, crossborder workers account for 6% of the total Swiss workforce and 20% of the immigrant workforce. Moreover, their numbers doubled between 1995 and 2018, an increase twice as large in relative terms as the simultaneous increase in resident immigrant workers. To understand Switzerland's particular attraction for crossborder workers, it is first worth picturing a map of Switzerland's distinct language regions. The triangle on the left pointing towards France forms the French-speaking part. The much smaller triangle hanging upside down into Italy covers the Italian-speaking, and the rest, mostly bordering Germany and Austria, the German-speaking part. One notices that all three main Swiss language regions border countries in which they are the sole official language. Evidently, language is not a major barrier at the Swiss border within the institutionally relatively well integrated European labor market, which together with the comparatively high Swiss wages makes Switzerland attractive for workers from neighboring countries. Second, Switzerland's comparatively high cost of living together with assumed personal preferences for residing in one's country of origin makes it relatively unattractive to work and live in Switzerland. Quite obviously, the resulting high numbers of crossborder workers are not evenly distributed across Switzerland. Swiss firms' opportunities to employ them is substantially constrained by their distance from the border. The empirical approach presented in Chapter 1 exploits this setting by comparing firms close to the border with large and firms far from the border with limited access to crossborder workers that are otherwise similar. Overall, we find that the increase observed in crossborder workers between 1995 and 2008 led to a decrease of about 3,500 apprenticeship positions, corresponding to roughly 2% of the total number of apprentice positions. Although the exact channel through which this substitution works remains ambiguous, policy makers designing institutions in either immigration or VET might want to pay attention to this trade-off, especially because it involves two goods that are in general positively valued by many employers. Chapter 2, which is joint work with Andreas Kuhn and Jürg Schweri, sticks with firms' investments in VET and, moreover, also investigates the spatial distribution of it across Switzerland. The focus lies on the varying proportions of firms providing apprenticeship positions across Switzerland's language regions (that you remember when recalling the map pictured above). Note first that we do not claim there is any direct link between different languages and different levels of firm engagement in VET. However, the distinct languages in Switzerland also maintain cultural differences within a small country, despite the fact that national institutions are well accepted and people increasingly mobile. It may be due to exclusive communication with same-language peers, selective media consumption, or varying exposure to Switzerland's neighboring countries with whom each Swiss language region forms a distinct supranational linguistic region: French speakers are most open to immigration and international cooperation and eat more meat than their German-speaking counterparts. German speakers have a more traditional understanding of gender roles than their French-speaking counterparts and donate the most for charity of all linguistic groups. Italian speakers use public transport the least and value leisure more than their German-speaking counterparts. In Chapter 2, we focus on a clear discontinuity that analysis of voting results reveals among the language-cultural regions constituting Switzerland: Whereas French and Italian speakers approve of strong state involvement, for example in the health insurance sector, for pension schemes, and in the VET system, German speakers prefer private engagement over the states in the same domains. From this starting point, we ask whether a favorable attitude towards private engagement expressed at the ballot box is actually accompanied by higher levels of the privately provided good apprenticeship. Chapter 2 reveals the answer to be yes: firms located in German-speaking municipalities are about 10% more likely to train apprentices than firms in very nearby French- and Italian-speaking municipalities. Altogether, we argue that norm-guided behavior is a complementary explanation for why some firms train apprentices and others do not. One can draw two policy implications from this finding. First, persistent norms might strengthen the sustainability of the Swiss VET system against potential shocks to firms' cost-benefit ratio. Second, behavior bound by norms might hinder the export of a Swiss-style VET system with its strong focus on firm engagement to other countries even if they set up the institutional framework to foster it. Chapter 3 focuses on the adult labor market, where skills acquired, e.g. in the VET system, are applied. Many economists claim that recent rapid technological change penetrating the labor market has shifted firms' skill demand and altered the nature of jobs. Given individuals' skills, these demand shifts potentially foster horizontal skill mismatches, such that someone's acquired skills do not match the skills needed in their current occupation. In line with previous mismatch literature, Chapter 3 shows that such horizontal skill mismatches are a multi-faceted phenomenon. First, whereas only about half of all individuals work in exactly the occupations that they learned formally, the degree of mismatch among the other half varies widely. Second, many individuals actually realize wage gains when becoming mismatched; this suggests that objectively identified mismatches are not bad per se. Based on these general findings and on the task-based approach, I hypothesize that horizontal skill mismatches are harmful to the wages of individuals who mostly hold skills substitutable by new technology, whereas they are not harmful in general. I account for this heterogeneity of mismatches in the empirical analysis of Chapter 3 by exploiting detailed occupational task data to measure the strength of mismatches and to focus on mismatches presumably caused by skill demand shifts due to new technology. The main result yields a wage penalty of roughly 12% for mismatched individuals with high shares of substitutable skills. Applying other methods to the same dataset suggests that objectively identified horizontal mismatches have zero wage implications on average, even after accounting for unobservable individual characteristics. From a policy perspective, it thus seems important to bear the heterogeneity of the mismatch phenomena in mind; otherwise, revealed average effects might mask negative effects on certain subgroups. In this spirit, I estimate mismatch wage penalties for different educational subgroups including VET diploma holders. And in this sense, all chapters of this thesis deal with the Swiss system of vocational education and training, a system that is recognized by many as one of the key contributors to the country's economic success. However, as this thesis shows, it is also a system that contains frictions, even contradictions at first sight, a system regularly challenged by pupils entering it and labor markets demanding its outcomes, and therefore a system that must remain agile. Thus, the recognition that the Swiss VET system receives from inside and outside the country must motivate constant reflection, adjustment, and amelioration, and should never tempt to rest on its laurels. I hope this thesis plays its modest role in fulfilling this purpose

On Biasing Transformer Attention Towards Monotonicity

Many sequence-to-sequence tasks in natural language processing are roughly monotonic in the alignment between source and target sequence, and previous work has facilitated or enforced learning of monotonic attention behavior via specialized attention functions or pretraining. In this work, we introduce a monotonicity loss function that is compatible with standard attention mechanisms and test it on several sequence-to-sequence tasks: grapheme-to-phoneme conversion, morphological inflection, transliteration, and dialect normalization. Experiments show that we can achieve largely monotonic behavior. Performance is mixed, with larger gains on top of RNN baselines. General monotonicity does not benefit transformer multihead attention, however, we see isolated improvements when only a subset of heads is biased towards monotonic behavior.Comment: To be published in: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2021

Reducing Gender Bias in NMT with FUDGE

Gender bias appears in many neural machine translation (NMT) models and commercial translation software. Research has become more aware of this problem in recent years and there has been work on mitigating gender bias. However, the challenge of addressing gender bias in NMT persists. This work utilizes a controlled text generation method, Future Discriminators for Generation (FUDGE), to reduce the so-called Speaking As gender bias. This bias emerges when translating from English to a language that openly marks the gender of the speaker. We evaluate the model on MuST-SHE, a challenge set to specifically evaluate gender translation. The results demonstrate improvements in the translation accuracy of the feminine terms

A Benchmark for Evaluating Machine Translation Metrics on Dialects without Standard Orthography

For sensible progress in natural language processing, it is important that we are aware of the limitations of the evaluation metrics we use. In this work, we evaluate how robust metrics are to non-standardized dialects, i.e. spelling differences in language varieties that do not have a standard orthography. To investigate this, we collect a dataset of human translations and human judgments for automatic machine translations from English to two Swiss German dialects. We further create a challenge set for dialect variation and benchmark existing metrics' performances. Our results show that existing metrics cannot reliably evaluate Swiss German text generation outputs, especially on segment level. We propose initial design adaptations that increase robustness in the face of non-standardized dialects, although there remains much room for further improvement. The dataset, code, and models are available here: https://github.com/textshuttle/dialect_eva

A Benchmark for Evaluating Machine Translation Metrics on Dialects Without Standard Orthography

For sensible progress in natural language processing, it is important that we are aware of the limitations of the evaluation metrics we use. In this work, we evaluate how robust metrics are to non-standardized dialects, i.e. spelling differences in language varieties that do not have a standard orthography. To investigate this, we collect a dataset of human translations and human judgments for automatic machine translations from English to two Swiss German dialects. We further create a challenge set for dialect variation and benchmark existing metrics' performances. Our results show that existing metrics cannot reliably evaluate Swiss German text generation outputs, especially on segment level. We propose initial design adaptations that increase robustness in the face of non-standardized dialects, although there remains much room for further improvement. The dataset, code, and models are available here: https://github.com/textshuttle/dialect_evalComment: WMT 2023 Research Pape

Buntbrachen und extensiv genutzte Wiesen mit QII-Status : wie unterscheidet sich die Tagfalterfauna dieser BFF-Typen?

In der Schweiz wird mit Biodiversitätsförderflächen (BFF) versucht, dem Rückgang der Biodiversität auf den landwirtschaftlichen Nutzflächen entgegenzuwirken. Damit die Biodiversität gezielt gefördert werden kann, ist es wichtig zu wissen, welche Effekte die unterschiedlichen BFF-Typen auf verschiedene Artengruppen haben. Diese Arbeit untersucht die Tagfalter- und Heuschreckenfauna auf den zwei BFF-Typen Buntbrachen und extensiv genutzte Wiesen mit Qualitätsstufe II. Sie analysiert die Unterschiede in der Artenvielfalt sowie der Individuenzahl und versucht zu ermitteln, welche Vegetations- und Umgebungsparameter einen Einfluss auf Tagfalter und Heuschrecken haben. Dazu wurden Kartierungen von 8 Buntbrachen und 8 extensiv genutzten Wiesen (QII) im Kanton Zürich durchgeführt. Zusätzlich wurden Vegetationsparameter erhoben und Umgebungsparameter mit einer GIS-Analyse berechnet. Es wurden signifikante Unterschiede in der Anzahl Arten und Individuen von Tagfalter zwischen den beiden BFF-Typen festgestellt. Sowohl auf Buntbrachen wie auch auf extensiv genutzten Wiesen (QII) wurden Arten gefunden, die auf dem anderen Typ fehlen. Die Kombination der zwei BFF-Typen erhöht damit die Vielfalt an Arten in einer Region. Die Artenvielfalt und die Individuenzahl nimmt mit steigendem Anteil BFF in der Umgebung ab. Dies widerspricht vorgängigen Studien. Für die Heuschrecken waren die Unterschiede nicht oder nur marginal signifikant. Es konnten jedoch mehr Individuen gefährdeter Arten auf Buntbrachen nachgewiesen werden als auf extensiv genutzten Wiesen QII. Die Bewirtschaftungsweise von Buntbrachen erklärt zudem das Vorkommen von zwei Arten, welche auf höhere Vegetation angewiesen sind. Für die Heuschrecken wurde der Anteil der Blüten als signifikanter Vegetationsparameter ermittelt, der einen Teil der aufgenommenen Arten- und Individuenzahl erklärt. Ein signifikanter Umgebungsparameter für die Anzahl Heuschreckenindividuen war der Anteil der landwirtschaftlichen Nutzfläche in einem Umkreis von 500 Metern. Der positive Zusammenhang entspricht dem häufigen Vorkommen von Heuschrecken im offenen Grünland. Auf Buntbrachen wurden Arten gefunden, deren Vorkommen von ausreichend hoher Vegetation abhängt. Diese Funde unterstreichen die Wichtigkeit von Flächen im Kulturland ohne Schnittregime, auf denen die Vegetation eine gewisse Höhe erreicht.In Switzerland ecological compensations areas (ECA) aim to counteract the decline in biodiversity on agricultural land. To systematically promote biodiversity, it is important to understand the effects of different ECA types on various groups of species. This study examines the populations of butterflies and grasshoppers on two types of ECA: wildflower strips and extensively used meadows of Quality Level II. It analyzes differences in species diversity and abundance and addition aims to determine which vegetation and environmental parameters may have an influence. For this purpose, surveys were conducted on 8 wildflower strips and 8 extensively used meadows (QII) in the Canton of Zurich. Additionally, vegetation parameters were recorded, and environmental parameters were calculated using GIS analysis. Significant differences in the number of butterfly species and individuals were found between the two types of ECA. On both types of ECA species were identified that were absent on the other type. The combination of these two ECA types thereby increases the diversity of species in a region. Species diversity and the number of individuals decrease as the proportion of BFF in the surrounding area increases. This contradicts previous studies. For grasshoppers, the differences were either non-significant or only marginally significant. However, more individuals of endangered species were detected on wildflower strips than on extensively used meadows. Furthermore, the management practices of wildflower strips explain the occurrence of two species that rely on taller vegetation. For grasshoppers, the proportion of flowers emerged as a significant vegetation parameter explaining part of the recorded species and individual counts. Addition-ally, a significant environmental parameter for the count of grasshopper individuals was the proportion of agricultural land within a 500-meter radius. This positive correlation aligns with the frequent presence of grasshoppers in open grassland habitats. Species whose occurrence depends on sufficiently high vegetation were found on wildflower strips. These findings underline the importance of areas in cultivated land without a cutting regime where the vegetation reaches a certain height