Knowledge-rich Image Gist Understanding Beyond Literal Meaning

We investigate the problem of understanding the message (gist) conveyed by images and their captions as found, for instance, on websites or news articles. To this end, we propose a methodology to capture the meaning of image-caption pairs on the basis of large amounts of machine-readable knowledge that has previously been shown to be highly effective for text understanding. Our method identifies the connotation of objects beyond their denotation: where most approaches to image understanding focus on the denotation of objects, i.e., their literal meaning, our work addresses the identification of connotations, i.e., iconic meanings of objects, to understand the message of images. We view image understanding as the task of representing an image-caption pair on the basis of a wide-coverage vocabulary of concepts such as the one provided by Wikipedia, and cast gist detection as a concept-ranking problem with image-caption pairs as queries. To enable a thorough investigation of the problem of gist understanding, we produce a gold standard of over 300 image-caption pairs and over 8,000 gist annotations covering a wide variety of topics at different levels of abstraction. We use this dataset to experimentally benchmark the contribution of signals from heterogeneous sources, namely image and text. The best result with a Mean Average Precision (MAP) of 0.69 indicate that by combining both dimensions we are able to better understand the meaning of our image-caption pairs than when using language or vision information alone. We test the robustness of our gist detection approach when receiving automatically generated input, i.e., using automatically generated image tags or generated captions, and prove the feasibility of an end-to-end automated process

Learning strategies in interpreting text: From comprehension to illustration

Learning strategies can be described as behaviours and thoughts a learner engages in during learning that are aimed at gaining knowledge. Learners are, to use Mayer’s (1996) constructivist definition, ‘sense makers’. We can therefore position this to mean that, if learners are sense makers, then learning strategies are essentially cognitive processes used when learners are striving to make sense out of newly presented material. This paper intends to demonstrate that such thoughts and behaviours can be made explicit and that students can co-ordinate the basic cognitive processes of selecting, organising and integrating. I will discuss two learning strategies which were developed during three cycles of an action research enquiry with a group of illustration students. While each cycle had its own particular structure and aims, the main task, that of illustrating a passage of expository text into an illustration was a constant factor. The first learning strategy involved assisting students develop ‘macropropositions’—personal understandings of the gist or essence of a text (Louwerse and Graesser, 2006; Armbruster, Anderson and Ostertag, 1987; Van Dijk & Kintsch, 1983). The second learning strategy used a form of induction categorised as analogical reasoning (Holyoak, 2005; Sloman and Lagnado, 2005). Both strategies were combined to illustrate the expository text extract. The data suggests that design students benefit from a structured approach to learning, where thinking processes and approaches can be identified and accessible for other learning situations. The research methodology is based on semi-structured interviews, questionnaires, developmental design (including student notes) and final design output. All student names used are pseudonyms. The text extract from ‘Through the Magic Door’ an essay Sir Arthur Conan Doyle, (1907) has been included as it provides context to analysis outcomes, student comments and design outputs. Keywords: Action Research; Illustration; Macrostructures; Analogical Reasoning; Learning Strategies</p

Metaphor in Analytic Philosophy and Cognitive Science

This article surveys theories of metaphor in analytic philosophy and cognitive science. In particular, it focuses on contemporary semantic, pragmatic and non-cognitivist theories of linguistic metaphor and on the Conceptual Metaphor Theory advanced by George Lakoff and his school. Special attention is given to the mechanisms that are shared by nearly all these approaches, i.e. mechanisms of interaction and mapping between conceptual domains. Finally, the article discusses several recent attempts to combine these theories of linguistic and conceptual metaphor into a unitary account

Understanding the message of images

We investigate the problem of understanding the message (gist) conveyed by images and their captions as found, for instance, on websites or news articles. To this end, we propose a methodology to capture the meaning of image-caption pairs on the basis of large amounts of machine-readable knowledge that have previously been shown to be highly effective for text understanding. Our method identifies the connotation of objects beyond their denotation: where most approaches to image or image-text understanding focus on the denotation of objects, i.e., their literal meaning, our work addresses the identification of connotations, i.e., iconic meanings of objects, to understand the message of images. We view image understanding as the task of representing an image-caption pair on the basis of a widecoverage vocabulary of concepts such as the one provided by Wikipedia, and cast gist detection as a concept-ranking problem with image-caption pairs as queries

Aesthetic Comprehension of Abstract and Emotion Concepts: Kant’s Aesthetics Renewed

In § 49 of the Critique of the Power of Judgment Kant puts forward a view that the feeling of pleasure in the experience of the beautiful can be stimulated not merely by perceptual properties, but by ideas and thoughts as well. The aim of this paper is to argue that aesthetic ideas fill in the emptiness that abstract and emotion concepts on their own would have without empirical intuitions. That is, aesthetic ideas make these concepts more accessible to us, by creating image schemas that allow us to think about these abstract concepts in a way linked to sensory experience, thereby imbuing them with a more substantive meaning and understanding

Translation as a Subjective and Creative Act: Choices and Constraints

This paper starts with the assumption that translation is neither a transference nor a replacement of one word or symbol in one language with a word or symbol in another It is a matter of choice selecting the most appropriate or equivalent form to the original This paper argues that translators subjectivity and creativity play a significant role in striking a balance between what is accurate and what is acceptable during the translation process Although translation is based on the source text its outcome depends essentially on the translator s own realization and perception of the text Therefore it may be difficult to approach translation from decoding and encoding perspectives because words and phrases change over the years However at the heart of the translation process there is the translator s free will which determines the outcome and quality of the text to be translate

Literalism and Contextualism: Some Varieties

Both Literalism and Contextualism come in many varieties. There are radical, and less radical, versions of both Literalism and Contextualism. Some intermediate positions are mixtures of Literalism and Contextualism. In this paper I describe several literalist positions, several contextualist positions, and a couple of intermediate positions. My aim is to convince the reader that the Literalism/Contextualism controversy is far from being settled. In the first section, I look at the historical development of Literalism. This development reveals a gradual weakening. The question that naturally arises is: How far can we go in this direction? Where will this tendency ultimately lead us? And the obvious answer is: to Contextualism. In the second section I describe the steps which, from a critique of the currently dominant literalist position (Minimalism), can lead to Contextualism. In the last three sections I describe various contextualist positions, and I discuss possible literalist replies to the contextualist challenge