Image Semantics in the Description and Categorization of Journalistic Photographs

This paper reports a study on the description and categorization of images. The aim of the study was to evaluate existing indexing frameworks in the context of reportage photographs and to find out how the use of this particular image genre influences the results. The effect of different tasks on image description and categorization was also studied. Subjects performed keywording and free description tasks and the elicited terms were classified using the most extensive one of the reviewed frameworks. Differences were found in the terms used in constrained and unconstrained descriptions. Summarizing terms such as abstract concepts, themes, settings and emotions were used more frequently in keywording than in free description. Free descriptions included more terms referring to locations within the images, people and descriptive terms due to the narrative form the subjects used without prompting. The evaluated framework was found to lack some syntactic and semantic classes present in the data and modifications were suggested. According to the results of this study image categorization is based on high-level interpretive concepts, including affective and abstract themes. The results indicate that image genre influences categorization and keywording modifies and truncates natural image description

Can Representationism Explain how Attention Affects Appearances?

Recent psychological research shows that attention affects appearances. An “attended item looks bigger, faster, earlier, more saturated, stripier.” (Block 2010, p. 41). What is the significance of these findings? Ned Block has argued that they undermine representationism, roughly the view that the phenomenal character of perception is determined by its representational content. My first goal in this paper is to show that Block’s argument has the structure of a Problem of Arbitrary Phenomenal Variation and that it improves on other instances of arguments of the same form along several dimensions (most prominently, these are arguments based on the possibility of spectral inversion). My second goal is to consider responses to Block’s version of the arbitrariness problem. I will show that most of them have serious drawbacks. Overall, the best view is to accept that attention may distort perception, sacrificing veridicality for usability. I end my discussion by showing how to develop that view

Image featuring for retrieval of multimedia documents

The utilization of massive multimedia documents collections, such as multimedia documents in the global Internet, needs search engines which can rank using both text and image evidence. Massive size and (dynamic) nature of collection can make manual indexing prohibitively expensive in such situations. Traditional search engines utilize only text components of multimedia documents. But there are information needs, which require the utilization of image evidence. In this paper, we investigate image-feature for large and heterogeneous collections. Both the nature and complexities of information needs are key elements for an effective retrieval. Retrieval needs that depend on perceptual similarities (as found in art galleries, building architecture) require the utilization of visual cues. In such situations, the retrieval of multimedia document based on image ranking can provide higher effectiveness. Experimental results show that effectiveness of ranking based on image feature can be higher where perceptual similarities are key elements for retrieval than the retrieval effectiveness of algorithms based on text ranking algorithms <br /

Towards an Indexical Model of Situated Language Comprehension for Cognitive Agents in Physical Worlds

We propose a computational model of situated language comprehension based on the Indexical Hypothesis that generates meaning representations by translating amodal linguistic symbols to modal representations of beliefs, knowledge, and experience external to the linguistic system. This Indexical Model incorporates multiple information sources, including perceptions, domain knowledge, and short-term and long-term experiences during comprehension. We show that exploiting diverse information sources can alleviate ambiguities that arise from contextual use of underspecific referring expressions and unexpressed argument alternations of verbs. The model is being used to support linguistic interactions in Rosie, an agent implemented in Soar that learns from instruction.Comment: Advances in Cognitive Systems 3 (2014

Learning perceptual schemas to avoid the utility problem

This paper describes principles for representing and organising planning knowledge in a machine learning architecture. One of the difficulties with learning about tasks requiring planning is the utility problem: as more knowledge is acquired by the learner, the utilisation of that knowledge takes on a complexity which overwhelms the mechanisms of the original task. This problem does not, however, occur with human learners: on the contrary, it is usually the case that, the more knowledgeable the learner, the greater the efficiency and accuracy in locating a solution. The reason for this lies in the types of knowledge acquired by the human learner and its organisation. We describe the basic representations which underlie the superior abilities of human experts, and describe algorithms for using equivalent representations in a machine learning architecture

Exploring cognitive issues in visual information retrieval

A study was conducted that compared user performance across a range of search tasks supported by both a textual and a visual information retrieval interface (VIRI). Test scores representing seven distinct cognitive abilities were examined in relation to user performance. Results indicate that, when using VIRIs, visual-perceptual abilities account for significant amounts of within-subjects variance, particularly when the relevance criteria were highly specific. Visualisation ability also seemed to be a critical factor when users were required to change topical perspective within the visualisation. Suggestions are made for navigational cues that may help to reduce the effects of these individual differences

Multi-level Semantic Analysis for Sports Video

There has been a huge increase in the utilization of video as one of the most preferred type of media due to its content richness for many significant applications including sports. To sustain an ongoing rapid growth of sports video, there is an emerging demand for a sophisticated content-based indexing system. Users recall video contents in a high-level abstraction while video is generally stored as an arbitrary sequence of audio-visual tracks. To bridge this gap, this paper will demonstrate the use of domain knowledge and characteristics to design the extraction of high-level concepts directly from audio-visual features. In particular, we propose a multi-level semantic analysis framework to optimize the sharing of domain characteristics

Dynamical patterns of human postural responses to emotional stimuli

Erotic scenes and images of mutilated bodies are emotional stimuli that have repeatedly shown to evoke specific neurophysiological responses associated with enhanced attention and perceptual processing. Remarkably however, only a handful of studies have investigated human motor reactions to emotional activation as a direct index of physical approximation or withdrawal. Given the inconclusive results of these studies, the approach-avoidance distinction, one of the most salient concepts in human motivational research, remains a broadly exploited hypothesis that has never been empirically demonstrated. Here, we investigate postural responses elicited by discrete emotional stimuli in healthy young adults. We discover that both positive and negative affective pictures induce a significant posterior deviation from postural baseline equilibrium. Further, we find that neutral pictures also evoke posterior deviation, although with a less pronounced amplitude. Exploring the dynamical evolution of postural responses to emotional pictures at high temporal resolution, we uncover a characteristic profile that remains stable for stimuli from all three affective categories. In contrast, the postural response amplitude is modulated by the emotional content of the stimulus. Our observations do not support the interpretation of postural responses to affective picture-viewing as approach-avoidance behavior. Instead, our findings indicate the involvement of a previously unrecognized motor component of the physiological mechanism underlying human orienting responses