Limitations of Cross-Lingual Learning from Image Search

Cross-lingual representation learning is an important step in making NLP scale to all the world's languages. Recent work on bilingual lexicon induction suggests that it is possible to learn cross-lingual representations of words based on similarities between images associated with these words. However, that work focused on the translation of selected nouns only. In our work, we investigate whether the meaning of other parts-of-speech, in particular adjectives and verbs, can be learned in the same way. We also experiment with combining the representations learned from visual data with embeddings learned from textual data. Our experiments across five language pairs indicate that previous work does not scale to the problem of learning cross-lingual representations beyond simple nouns

Why is unsupervised alignment of English embeddings from different algorithms so hard?

This paper presents a challenge to the community: Generative adversarial networks (GANs) can perfectly align independent English word embeddings induced using the same algorithm, based on distributional information alone; but fails to do so, for two different embeddings algorithms. Why is that? We believe understanding why, is key to understand both modern word embedding algorithms and the limitations and instability dynamics of GANs. This paper shows that (a) in all these cases, where alignment fails, there exists a linear transform between the two embeddings (so algorithm biases do not lead to non-linear differences), and (b) similar effects can not easily be obtained by varying hyper-parameters. One plausible suggestion based on our initial experiments is that the differences in the inductive biases of the embedding algorithms lead to an optimization landscape that is riddled with local optima, leading to a very small basin of convergence, but we present this more as a challenge paper than a technical contribution.Comment: Accepted at EMNLP 201

Issue Framing in Online Discussion Fora

In online discussion fora, speakers often make arguments for or against something, say birth control, by highlighting certain aspects of the topic. In social science, this is referred to as issue framing. In this paper, we introduce a new issue frame annotated corpus of online discussions. We explore to what extent models trained to detect issue frames in newswire and social media can be transferred to the domain of discussion fora, using a combination of multi-task and adversarial training, assuming only unlabeled training data in the target domain.Comment: To appear in NAACL-HLT 201

Putting Humans in the Image Captioning Loop

Image Captioning (IC) models can highly benefit from human feedback in the training process, especially in cases where data is limited. We present work-in-progress on adapting an IC system to integrate human feedback, with the goal to make it easily adaptable to user-specific data. Our approach builds on a base IC model pre-trained on the MS COCO dataset, which generates captions for unseen images. The user will then be able to offer feedback on the image and the generated/predicted caption, which will be augmented to create additional training instances for the adaptation of the model. The additional instances are integrated into the model using step-wise updates, and a sparse memory replay component is used to avoid catastrophic forgetting. We hope that this approach, while leading to improved results, will also result in customizable IC models

Mapping (Dis-)Information Flow about the MH17 Plane Crash

Digital media enables not only fast sharing of information, but also disinformation. One prominent case of an event leading to circulation of disinformation on social media is the MH17 plane crash. Studies analysing the spread of information about this event on Twitter have focused on small, manually annotated datasets, or used proxys for data annotation. In this work, we examine to what extent text classifiers can be used to label data for subsequent content analysis, in particular we focus on predicting pro-Russian and pro-Ukrainian Twitter content related to the MH17 plane crash. Even though we find that a neural classifier improves over a hashtag based baseline, labeling pro-Russian and pro-Ukrainian content with high precision remains a challenging problem. We provide an error analysis underlining the difficulty of the task and identify factors that might help improve classification in future work. Finally, we show how the classifier can facilitate the annotation task for human annotators

Bariatrische Operation: verschiedene präoperative Behandlungsstrategien und deren Ergebnisse

Bedingt durch eine zunehmend ungesunde Lebensweise steigt die Prävalenz der Adipositas besonders in den industrialisierten Ländern stetig. Neben der konservativen Therapie tritt vor allem bei extremer Adipositas die bariatrische Operation als neuere Therapiemethode immer weiter in den Vordergrund. Für ein gutes postoperatives Ergebnis erscheint eine präoperative Therapie sinnvoll, allerdings fehlt in Deutschland bisher ein Standard für die Durchführung einer solchen Maßnahme. Im Rahmen des Obesity Balance Programms, welches an zwei Standorten der Charité Berlin (Stoff- wechselcentrum des Campus Virchow-Klinikum (CVK) und Adipositaszentrum des Campus Charité Mitte (CCM)) in modifizierten Varianten des gleichen multimodalen Konzepts mit Bewegungs-, Verhal- tens und Ernährungstherapie durchgeführt wurde, wurden laborchemische sowie psychosoziale Daten von 68 Patienten (CVK: 35 Patienten; CCM: 33 Patienten) erhoben. BMI-Veränderungen im Verlauf des sechsmonatigen Programms, die Anzahl an Patienten, die sich im Anschluss an das Programm operieren ließen, sowie deren postoperativer BMI-Verlauf wurden dokumentiert. Anschließend wurden die beiden Konzepte sowie ihre Ergebnisse im Bezug auf BMI-Veränderung und Operationsbereitschaft verglichen. Am CVK wurde das Programm teilstationär bei Diabetikern, am CCM ambulant bei nicht-Diabetikern durchgeführt. Die Patienten des CVK erhielten durch eine fünftägige theoretische und eine fünftägige praktische Schulung sowie den alle vier Wochen stattfindenden halbtägigen Aufenthalt in der Klinik, eine intensivere Betreuung als die Patienten des CCM. Die Bewegungstherapie fand am CVK 24 Mal, am CCM 12 Mal in den sechs Monate statt. Die beiden Gruppen unterschieden sich zudem deutlich hinsichtlich des medianen Alters (CVK: 55,7 Jahre; CCM: 37,7 Jahre), der Anzahl an Komorbiditäten (CVK: 5,0; CCM: 2,0), dem erzielten BMI- Verlust während des Obesity Balance Programms (CVK: 3,2 kg/m2; CCM: 0,8 kg/m2) sowie der Anzahl an operierten Patienten (CVK: 25,0 % der Patienten; CCM: 59,3 % der Patienten). Patienten, die wäh- rend des Programms erfolgreich Gewicht verloren hatten, ließen sich tendenziell seltener operieren. Ebenso waren ältere Patienten sowie Patienten mit vielen Komorbiditäten eher zurückhaltend bezüglich einer Operation. Frauen ließen sich häufiger operieren als Männer (Frauen: 51,5 %; Männer: 26,9 %). Mithilfe eines multimodalen Therapiekonzepts ist es möglich, erfolgreich Gewicht abzunehmen. Die Intensität der Betreuung sowie die Häufigkeit physischer Aktivität zeigen sich dabei als wichtige Fak- toren. Die Gefahr einer Gewichtswiederzunahme ist ohne eine kontinuierliche Betreuung jedoch groß. Nach einer bariatrischen Operation verlieren die Patienten zunächst deutlich an Gewicht, doch auch bei ihnen ist eine lebenslange Begleitung notwendig, um Rückfälle zu verhindern und die durch die Operation entstandenen Probleme, wie den Nährstoffmangel, kontrollieren zu können.Due to an increasingly unhealthy lifestile, the prevalence of obesity is rising steadily worldwide, especially in the industrialised countries. Apart from non-surgical treatment, bariatric surgery is emerging as a new treatment, particularly for morbid obesity. For a positive postoperative result, a preoperative intervention seems to be reasonable. However, in Germany, standards are lacking for conducing such treatment. The Obesity Balance Programm was conducted at two sites of the Charité Berlin (Stoffwechselcentrum of Campus Virchow-Klinikum (CVK) and Adipositaszentrum of Campus Charité Mitte (CCM)) for six months, with each site conducting the same programm with slight modifications, including physical activity, behavioural counseling, and dietary advice. Laboratory values as well as psychosocial data of 68 patients (CVK: 35 patients; CCM: 33 patients) were compiled. Changes in BMI during the programm, the number of patients who underwent bariatric surgery following the programm, as well as their postoperativ changes in BMI were recorded. Finally, the two concepts, the respective results with regard to changes in BMI and the patients’ readiness to be operated, were compared. At the CVK, the programm was conducted in a semi-residential care setting with patients diagnosed with diabetes, whereas the programm at the CCM was carried out in an outpatient setting with patients not diagnosed with diabetes. The patients of the CVK additionally took part in a five-day theoretical and a five-day practical course. These patients also underwent greater supervision through participation in half-day meetings that took place every four weeks. Furthermore, physical training was conducted 24 times at the CVK and 12 times at the CCM during the six month programm. Both groups differed significantly regarding median age (CVK: 55,7 years; CCM: 37,7 years), number of comorbidities (CVK: 5,0; CCM: 2,0), BMI-reduction during the Obesity Balance Programm (CVK: 3,2 kg/m2; CCM: 0,8 kg/m2), and the number of patients who underwent surgery (CVK: 25,0 %; CCM: 59,3 %). Patients who successfully lost weight during the programm were less likely to undergo surgery. Similarly, older patients and patients with several comorbidities were more reluctant with respect to the surgery. Women were more likely to have surgery compared to men (women: 51,5 %; men: 26,9 %). It is possible to loose weight successfully by means of a multimodal treatment concept . The frequency of care as well as physical activity appear to be important factors. Nevertheless, without continuous supervision, the danger of regaining weight is high. Patients loose weight quickly immediately following bariatric surgery. However as after non-surgical therapy, a lifelong support is necessary to avoid relapses and to address problems created by the surgery, such as nutrient deficiency

Towards Adaptable and Interactive Image Captioning with Data Augmentation and Episodic Memory

Interactive machine learning (IML) is a beneficial learning paradigm in cases of limited data availability, as human feedback is incrementally integrated into the training process. In this paper, we present an IML pipeline for image captioning which allows us to incrementally adapt a pre-trained image captioning model to a new data distribution based on user input. In order to incorporate user input into the model, we explore the use of a combination of simple data augmentation methods to obtain larger data batches for each newly annotated data instance and implement continual learning methods to prevent catastrophic forgetting from repeated updates. For our experiments, we split a domain-specific image captioning dataset, namely VizWiz, into non-overlapping parts to simulate an incremental input flow for continually adapting the model to new data. We find that, while data augmentation worsens results, even when relatively small amounts of data are available, episodic memory is an effective strategy to retain knowledge from previously seen clusters