Compositional Generalization in Image Captioning

Image captioning models are usually evaluated on their ability to describe a held-out set of images, not on their ability to generalize to unseen concepts. We study the problem of compositional generalization, which measures how well a model composes unseen combinations of concepts when describing images. State-of-the-art image captioning models show poor generalization performance on this task. We propose a multi-task model to address the poor performance, that combines caption generation and image--sentence ranking, and uses a decoding mechanism that re-ranks the captions according their similarity to the image. This model is substantially better at generalizing to unseen combinations of concepts compared to state-of-the-art captioning models.Comment: To appear at CoNLL 2019, EMNL

Robust Multiple Stopping -- A Pathwise Duality Approach

In this paper we develop a solution method for general optimal stopping problems. Our general setting allows for multiple exercise rights, i.e., optimal multiple stopping, for a robust evaluation that accounts for model uncertainty, and for general reward processes driven by multi-dimensional jump-diffusions. Our approach relies on first establishing robust martingale dual representation results for the multiple stopping problem which satisfy appealing pathwise optimality (almost sure) properties. Next, we exploit these theoretical results to develop upper and lower bounds which, as we formally show, not only converge to the true solution asymptotically, but also constitute genuine upper and lower bounds. We illustrate the applicability of our general approach in a few examples and analyze the impact of model uncertainty on optimal multiple stopping strategies

Robust multiple stopping -- A path-wise duality approach

In this paper we develop a solution method for general optimal stopping problems. Our general setting allows for multiple exercise rights, i.e., optimal multiple stopping, for a robust evaluation that accounts for model uncertainty, and for general reward processes driven by multi-dimensional jump-diffusions. Our approach relies on first establishing robust martingale dual representation results for the multiple stopping problem which satisfy appealing path-wise optimality (almost sure) properties. Next, we exploit these theoretical results to develop upper and lower bounds which, as we formally show, not only converge to the true solution asymptotically, but also constitute genuine upper and lower bounds. We illustrate the applicability of our general approach in a few examples and analyze the impact of model uncertainty on optimal multiple stopping strategies

Robust multiple stopping -- A path-wise duality approach

In this paper we develop a solution method for general optimal stopping problems. Our general setting allows for multiple exercise rights, i.e., optimal multiple stopping, for a robust evaluation that accounts for model uncertainty, and for general reward processes driven by multi-dimensional jump-diffusions. Our approach relies on first establishing robust martingale dual representation results for the multiple stopping problem which satisfy appealing path-wise optimality (almost sure) properties. Next, we exploit these theoretical results to develop upper and lower bounds which, as we formally show, not only converge to the true solution asymptotically, but also constitute genuine upper and lower bounds. We illustrate the applicability of our general approach in a few examples and analyze the impact of model uncertainty on optimal multiple stopping strategies

On the Realization of Compositionality in Neural Networks

We present a detailed comparison of two types of sequence to sequence models trained to conduct a compositional task. The models are architecturally identical at inference time, but differ in the way that they are trained: our baseline model is trained with a task-success signal only, while the other model receives additional supervision on its attention mechanism (Attentive Guidance), which has shown to be an effective method for encouraging more compositional solutions (Hupkes et al.,2019). We first confirm that the models with attentive guidance indeed infer more compositional solutions than the baseline, by training them on the lookup table task presented by Li\v{s}ka et al. (2019). We then do an in-depth analysis of the structural differences between the two model types, focusing in particular on the organisation of the parameter space and the hidden layer activations and find noticeable differences in both these aspects. Guided networks focus more on the components of the input rather than the sequence as a whole and develop small functional groups of neurons with specific purposes that use their gates more selectively. Results from parameter heat maps, component swapping and graph analysis also indicate that guided networks exhibit a more modular structure with a small number of specialized, strongly connected neurons.Comment: To appear at BlackboxNLP 2019, AC

Timely and individualized heart failure management: need for implementation into the new guidelines

Due to remarkable improvements in heart failure (HF) management over the last 30 years, a significant reduction in mortality and hospitalization rates in HF patients with reduced ejection fraction (HFrEF) has been observed. Currently, the optimization of guideline-directed chronic HF therapy remains the mainstay to further improve outcomes for patients with HFrEF to reduce mortality and HF hospitalization. This includes established device therapies, such as implantable defibrillators and cardiac resynchronization therapies, which improved patients' symptoms and prognosis. Over the last 10 years, new HF drugs have merged targeting various pathways, such as those that simultaneously suppress the renin–angiotensin–aldosterone system and the breakdown of endogenous natriuretic peptides (e.g., sacubitril/valsartan), and those that inhibit the If channel and, thus, reduce heart rate (e.g., ivabradine). Furthermore, the treatment of patient comorbidities (e.g., iron deficiency) has shown to improve functional capacity and to reduce hospitalization rates, when added to standard therapy. More recently, other potential treatment mechanisms have been explored, such as the sodium/glucose co-transporter inhibitors, the guanylate cyclase stimulators and the cardiac myosin activators. In this review, we summarize the novel developments in HFrEF pharmacological and device therapy and discuss their implementation strategies into practice to further improve outcomes

Feedback communicatif dans l'acquisition du langage

Children start to communicate and use language in social interactions from a very young age. This allows them to experiment with their developing linguistic knowledge and receive valuable feedback from their interlocutors. While research in language acquisition has focused a great deal on children's ability to learn from the linguistic input or social cues, little work, in comparison, has investigated the nature and role of Communicative Feedback, a process that results from children and caregivers trying to coordinate mutual understanding. By drawing on insights from theories of communicative coordination we can formalize a new mechanism for language acquisition: We argue that children can improve their linguistic knowledge in conversation by leveraging explicit or implicit signals of communication success or failure. Based on this hypothesis, we conducted two corpus studies that highlight the role of Communicative Feedback as a mechanism supporting the production of intelligible speech, as well as the acquisition of the grammar of one's native language. Finally, we design and evaluate computational models that instantiate a feedback-based learning mechanism in addition to statistical learning and demonstrate that such feedback can improve the acquisition of semantics.Communicative Feedback provides a common framework for several lines of research in child development and will enable us to obtain a more complete understanding of language acquisition within and through social interaction.Bien avant que leurs compétences linguistiques ne soient pleinement développées, les enfants participent à des échanges conversationnels. Cela leur permet d'expérimenter avec leurs connaissances linguistiques émergentes et de recevoir un feedback communicatif de la part de leurs interlocuteurs. En s'inspirant des théories de la coordination communicative, nous pouvons formaliser un nouveau mécanisme d'acquisition des langages : Les enfants peuvent améliorer leurs connaissances linguistiques au cours d'une conversation en exploitant les signaux explicites ou implicites de réussite ou de rupture de la communication.À partir de cette hypothèse, nous menons deux études de corpus qui soulignent le rôle du feedback communicatif en tant que mécanisme soutenant la production d'un langage intelligible, ainsi que l'acquisition de la grammaire de la langue maternelle. Enfin, nous concevons et évaluons des modèles computationnels qui instancient un mécanisme d'apprentissage basé sur le feedback en plus de l'apprentissage statistique et nous démontrons que ce feedback peut améliorer l'acquisition de la sémantique.Le feedback communicatif fournit un cadre commun à plusieurs lignes de recherche sur le développement de l'enfant et nous permettra d'obtenir une compréhension plus complète de l'acquisition du langage au sein et à travers l'interaction sociale

Evaluating the Acquisition of Semantic Knowledge from Cross-situational Learning in Artificial Neural Networks

When learning their native language, children acquire the meanings of words and sentences from highly ambiguous input without much explicit supervision. One possible learning mechanism is cross-situational learning, which has been successfully tested in laboratory experiments with children. Here we use Artificial Neural Networks to test if this mechanism scales up to more natural language and visual scenes using a large dataset of crowd-sourced images with corresponding descriptions. We evaluate learning using a series of tasks inspired by methods commonly used in laboratory studies of language acquisition. We show that the model acquires rich semantic knowledge both at the word- and sentence-level, mirroring the patterns and trajectory of learning in early childhood. Our work highlights the usefulness of low-level co-occurrence statistics across modalities in facilitating the early acquisition of higher-level semantic knowledge

Evaluating the Acquisition of Semantic Knowledge from Cross-situational Learning in Artificial Neural Networks

International audienceWhen learning their native language, children acquire the meanings of words and sentences from highly ambiguous input without much explicit supervision. One possible learning mechanism is cross-situational learning, which has been successfully tested in laboratory experiments with children. Here we use Artificial Neural Networks to test if this mechanism scales up to more natural language and visual scenes using a large dataset of crowd-sourced images with corresponding descriptions. We evaluate learning using a series of tasks inspired by methods commonly used in laboratory studies of language acquisition. We show that the model acquires rich semantic knowledge both at the word-and sentence-level, mirroring the patterns and trajectory of learning in early childhood. Our work highlights the usefulness of low-level cooccurrence statistics across modalities in facilitating the early acquisition of higher-level semantic knowledge

Modeling the Interaction Between Perception-Based and Production-Based Learning in Children's Early Acquisition of Semantic Knowledge

Children learn the meaning of words and sentences in their native language at an impressive speed and from highly ambiguous input. To account for this learning, previous computational modeling has focused mainly on the study of perception-based mechanisms like cross-situational learning. However, children do not learn only by exposure to the input. As soon as they start to talk, they practice their knowledge in social interactions and they receive feedback from their caregivers. In this work, we propose a model integrating both perception- and production-based learning using artificial neural networks which we train on a large corpus of crowd-sourced images with corresponding descriptions. We found that production-based learning improves performance above and beyond perception-based learning across a wide range of semantic tasks including both word- and sentence-level semantics. In addition, we documented a synergy between these two mechanisms, where their alternation allows the model to converge on more balanced semantic knowledge. The broader impact of this work is to highlight the importance of modeling language learning in the context of social interactions where children are not only understood as passively absorbing the input, but also as actively participating in the construction of their linguistic knowledge