ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning

We propose a new paradigm to continually evolve pretrained models, denoted ColD Fusion. It provides the benefits of multitask learning but leverages distributed computation with limited communication and eliminates the need for shared data. Consequentially, ColD Fusion can give rise to a synergistic loop, where finetuned models can be recycled to continually improve the pretrained model they are based upon. We show that ColD Fusion yields comparable benefits to multitask training by producing a model that (a) attains strong performance on all of the datasets it was trained on; and (b) is a better starting point for finetuning on unseen datasets. We show that ColD Fusion outperforms RoBERTa and even previous multitask models. Specifically, when training and testing on 35 diverse datasets, ColD Fusion-based model outperforms RoBERTa by 2.33 points on average without any changes to the architecture.Comment: ACL 2

PreQuEL: Quality Estimation of Machine Translation Outputs in Advance

We present the task of PreQuEL, Pre-(Quality-Estimation) Learning. A PreQuEL system predicts how well a given sentence will be translated, without recourse to the actual translation, thus eschewing unnecessary resource allocation when translation quality is bound to be low. PreQuEL can be defined relative to a given MT system (e.g., some industry service) or generally relative to the state-of-the-art. From a theoretical perspective, PreQuEL places the focus on the source text, tracing properties, possibly linguistic features, that make a sentence harder to machine translate. We develop a baseline model for the task and analyze its performance. We also develop a data augmentation method (from parallel corpora), that improves results substantially. We show that this augmentation method can improve the performance of the Quality-Estimation task as well. We investigate the properties of the input text that our model is sensitive to, by testing it on challenge sets and different languages. We conclude that it is aware of syntactic and semantic distinctions, and correlates and even over-emphasizes the importance of standard NLP features

The ‘symbolic homeland’ in the Jewish Italian Diaspora: the celebration of civil religion in Italy

In this article, the attitudes of the Italian Jewish diaspora towards Israel, seen as a symbolic homeland, are analyzed. This analysis is based on the theoretical discourse on diaspora. Attitudes are explored through the participation of Italian Jews in the celebration of Yom ha atzma’ut. Given the significance of Israel’s Independence Day in the sequence of the three-day civil religious commemoration in Israel – which constitutes a sort of national narrative – the entire sequence is also considered in the analysis of the Italian diaspor