Image Semantics in the Description and Categorization of Journalistic Photographs

This paper reports a study on the description and categorization of images. The aim of the study was to evaluate existing indexing frameworks in the context of reportage photographs and to find out how the use of this particular image genre influences the results. The effect of different tasks on image description and categorization was also studied. Subjects performed keywording and free description tasks and the elicited terms were classified using the most extensive one of the reviewed frameworks. Differences were found in the terms used in constrained and unconstrained descriptions. Summarizing terms such as abstract concepts, themes, settings and emotions were used more frequently in keywording than in free description. Free descriptions included more terms referring to locations within the images, people and descriptive terms due to the narrative form the subjects used without prompting. The evaluated framework was found to lack some syntactic and semantic classes present in the data and modifications were suggested. According to the results of this study image categorization is based on high-level interpretive concepts, including affective and abstract themes. The results indicate that image genre influences categorization and keywording modifies and truncates natural image description

Image Semantics in the Description and Categorization of Journalistic Photographs

This paper reports a study on the description and categorization of images. The aim of the study was to evaluate existing indexing frameworks in the context of reportage photographs and to find out how the use of this particular image genre influences the results. The effect of different tasks on image description and categorization was also studied. Subjects performed keywording and free description tasks and the elicited terms were classified using the most extensive one of the reviewed frameworks. Differences were found in the terms used in constrained and unconstrained descriptions. Summarizing terms such as abstract concepts, themes, settings and emotions were used more frequently in keywording than in free description. Free descriptions included more terms referring to locations within the images, people and descriptive terms due to the narrative form the subjects used without prompting. The evaluated framework was found to lack some syntactic and semantic classes present in the data and modifications were suggested. According to the results of this study image categorization is based on high-level interpretive concepts, including affective and abstract themes. The results indicate that image genre influences categorization and keywording modifies and truncates natural image description

Journalistic image access : description, categorization and searching

The quantity of digital imagery continues to grow, creating a pressing need to develop efficient methods for organizing and retrieving images. Knowledge on user behavior in image description and search is required for creating effective and satisfying searching experiences. The nature of visual information and journalistic images creates challenges in representing and matching images with user needs. The goal of this dissertation was to understand the processes in journalistic image access (description, categorization, and searching), and the effects of contextual factors on preferred access points. These were studied using multiple data collection and analysis methods across several studies. Image attributes used to describe journalistic imagery were analyzed based on description tasks and compared to a typology developed through a meta-analysis of literature on image attributes. Journalistic image search processes and query types were analyzed through a field study and multimodal image retrieval experiment. Image categorization was studied via sorting experiments leading to a categorization model. Advances to research methods concerning search tasks and categorization procedures were implemented. Contextual effects on image access were found related to organizational contexts, work, and search tasks, as well as publication context. Image retrieval in a journalistic work context was contextual at the level of image needs and search process. While text queries, together with browsing, remained the key access mode to journalistic imagery, participants also used visual access modes in the experiment, constructing multimodal queries. Assigned search task type and searcher expertise had an effect on query modes utilized. Journalistic images were mostly described and queried for on the semantic level but also syntactic attributes were used. Constraining the description led to more abstract descriptions. Image similarity was evaluated mainly based on generic semantics. However, functionally oriented categories were also constructed, especially by domain experts. Availability of page context promoted thematic rather than object-based categorization. The findings increase our understanding of user behavior in image description, categorization, and searching, as well as have implications for future solutions in journalistic image access. The contexts of image production, use, and search merit more interest in research as these could be leveraged for supporting annotation and retrieval. Multiple access points should be created for journalistic images based on image content and function. Support for multimodal query formulation should also be offered. The contributions of this dissertation may be used to create evaluation criteria for journalistic image access systems

Combining Text Semantics and Image Geometry to Improve Scene Interpretation

Inthispaper,wedescribeanovelsystemthatidentifiesrelationsbetweentheobjectsextractedfromanimage. We started from the idea that in addition to the geometric and visual properties of the image objects, we could exploit lexical and semantic information from the text accompanying the image. As experimental set up, we gathered a corpus of images from Wikipedia as well as their associated articles. We extracted two types of objects: human beings and horses and we considered three relations that could hold between them: Ride, Lead, or None. We used geometric features as a baseline to identify the relations between the entities and we describe the improvements brought by the addition of bag-of-wordf eatures and predicate–arguments tructures we derived from the text. The best semantic model resulted in a relative error reduction of more than 18% over the baseline

The attitudinal work of news journalism images – a search for visual and verbal analogues

This paper is concerned with the potential of journalistic still images (photographs, pictorial layouts, artwork and political cartooning) to position readers/viewers to take a positive or negative view of the people, events and situations which are the subject matter of news journalism coverage. Referencing prior work by Economou (2009) and Swain (2012), it offers an account of the mechanisms, the particular visual qualities and compositional arrangements, by which such attitudinal effects are achieved. Its primary concern is exploring the grounds by which the mechanisms through which journalistic images activate positive or negative attitudes might be treated as analogous to related verbal expressions of attitude. In developing this discussion, it references and revisits the account of the language of evaluation developed within the Appraisal literature (Martin and White 2005). It proposes that in order to identify analogues to verbal expressions which, on the one hand, explicitly assert attitudinal assessments and which, on the other hand, activate attitudinal positions through implication and association, it is useful to attend to the following issues: (1) the salience or detectability of the author’s subjective presence in the text as the communicative agent who puts an attitudinal meaning into play; (2) the stability of the attitudinal associations of a given expression across multiple contexts of use; (3) the role of the reader in supplying attitudinal interpretations or inferences; (4) the terms under which relations of author-reader solidarity are negotiated or put at risk but the expression currently under consideration

Valokuvaajasta katsojaan – mikä tekee toimituksellisista kuvista kiinnostavia?

The licentiate thesis focused on the editorial interestingness of photos. The term editorial interest was coined in the study; it refers to all factors that contribute to making a photograph editorially interesting in professional workflow. The thesis considered the whole process from taking, editing and selecting editorial photos to viewing the final, published photos. Three research questions were set for the study. The first concentrated on the factors that influ-ence the editorial interestingness of photos in the process of their creation and selection. The sec-ond question dealt with the factors, which affect the interestingness of editorial photos according to viewers. These two questions were approached through three case studies. The third research question analyzed what factors of editorial interest are overarching independent of the scenario. The two first cases examined the production perspective of news photos: the first with observa-tions and thematic interviews in a photo agency, the second through a survey integrated in the publishing system of four newspapers. The third case study was conducted as an interestingness evaluation with supporting interviews in a laboratory setting with amateur participants and maga-zine photos as material. All three cases had five factors of editorial interestingness in common; those factors were over-arching independent of test setting, photo genre, and role of evaluator. The factors were aesthetics, affect, novelty-complexity, semantics, and utility. The editorial interestingness of a photo was thus influenced by aesthetic criteria, the ability of the photo to evoke emotion, unexpectedness, con-tent-related criteria, and versatile possibilities of photo use. Viewers considered editorial interestingness to encompass two additional factors: the ability of the photo to attract and hold attention, and self-reference experienced by the viewer. These two were confirmed by literature, which suggested also one additional factor, so called coping potential, i.e. person’s ability to adapt and to cope with a novel, complex view.Lisensiaatintyön aiheena oli valokuvien toimituksellinen kiinnostavuus. Toimituksellinen kiinnostavuus asetettiin työssä kattotermiksi, jonka alle kuuluvat kaikki kuvan kiinnostavuuteen ammattilaistyönkulussa vaikuttavat tekijät. Työssä tarkasteltiin koko prosessia kuvan ottamisesta, käsittelystä ja valinnasta valmiin, julkaistun kuvan katseluun. Työlle asetettiin kolme tutkimuskysymystä. Ensimmäinen keskittyi tekijöihin, jotka vaikuttavat toimituksellisten kuvien kiinnostavuuteen niiden luonti- ja valintavaiheessa. Toinen kysymys selvitti, mitkä tekijät vaikuttavat toimituksellisten kuvien kiinnostavuuteen katsojan näkökulmasta. Näitä kahta tutkimuskysymystä lähestyttiin kolmen tapaustutkimuksen kautta. Kolmas kysymys analysoi, mitkä toimituksellisen kiinnostavuuden tekijät ovat yhdistäviä tilanteesta riippumatta. Kaksi ensimmäistä tutkimusta selvittelivät tuotantonäkökulmaa uutiskuvien puolella: ensimmäinen havainnointien ja teemahaastattelujen kautta kuvatoimistossa ja jälkimmäinen toimitusjärjestelmään integroidulla kyselyllä neljässä sanomalehdessä. Kolmas tapaustutkimus toteutettiin aikakauslehtikuvien kiinnostavuusarviointeina ja tarkentavina haastatteluina laboratoriossa, amatööreillä koehenkilöillä. Kaikissa kolmessa tutkimuksessa nousi esiin viisi toimituksellisten kuvien kiinnostavuuteen liittyvää osatekijää, jotka yhdistivät kaikkia tapauksia koeympäristöstä, kuvagenrestä, arvioijan ammattimaisuudesta riippumatta. Nämä tekijät olivat: affektiivisuus, estetiikka, käyttökelpoisuus, semantiikka ja uutuus-kompleksisuus. Kuvan toimitukselliseen kiinnostavuuteen siis vaikuttivat kuvan kyky herättää tunteita, esteettiset tekijät, monipuolinen käytettävyys, sisältöön liittyvät semanttiset tekijät, sekä ennennäkemättömyyteen ja monimutkaisuuteen liittyvät attribuutit. Katsojien näkökulmasta kuvan kiinnostavuudella oli kaksi muutakin tekijää: kuvan kyky kiinnittää ja ylläpitää katsojan huomio, sekä katsojan kokema omakohtaisuus. Nämä vahvistuivat kirjallisuuskatsauksessa, kirjallisuudesta löytyi lisäksi yksi tekijä, nk. selviytymispotentiaali, eli katsojan kyky sopeutua ja selvitä katselemastaan uudesta ja monimutkaisesta asiasta

Finding emotional-laden resources on the World Wide Web

Some content in multimedia resources can depict or evoke certain emotions in users. The aim of Emotional Information Retrieval (EmIR) and of our research is to identify knowledge about emotional-laden documents and to use these findings in a new kind of World Wide Web information service that allows users to search and browse by emotion. Our prototype, called Media EMOtion SEarch (MEMOSE), is largely based on the results of research regarding emotive music pieces, images and videos. In order to index both evoked and depicted emotions in these three media types and to make them searchable, we work with a controlled vocabulary, slide controls to adjust the emotions’ intensities, and broad folksonomies to identify and separate the correct resource-specific emotions. This separation of so-called power tags is based on a tag distribution which follows either an inverse power law (only one emotion was recognized) or an inverse-logistical shape (two or three emotions were recognized). Both distributions are well known in information science. MEMOSE consists of a tool for tagging basic emotions with the help of slide controls, a processing device to separate power tags, a retrieval component consisting of a search interface (for any topic in combination with one or more emotions) and a results screen. The latter shows two separately ranked lists of items for each media type (depicted and felt emotions), displaying thumbnails of resources, ranked by the mean values of intensity. In the evaluation of the MEMOSE prototype, study participants described our EmIR system as an enjoyable Web 2.0 service

From Text to Image - Shaping a Visual Grounded Theory Methodology

Qualitative social and cultural research is increasingly engaging with visual data. Starting from the premise "all is data" in grounded theory methodology (GTM), we propose a general framework to realize a visual grounded theory methodology (VGTM). Referring to exploratory visual methods based on objective hermeneutics, the documentary method, and segment analysis, as well as existing GTM discourses, we discuss how this text-centered procedure can be applied to visual data. We focus on the (re)formulation of procedural steps (such as making an inventory, segmentation and coding, memo writing, and sampling strategies), and the examination of images in relation to GTM logic

User-generated descriptions of individual images versus labels of groups 3 of images: A comparison using basic level theory

Although images are visual information sources with little or no text associated with them, users still tend to use text to describe images and formulate queries. This is because digital libraries and search engines provide mostly text query options and rely on text annotations for representation and retrieval of the semantic content of images. While the main focus of image research is on indexing and retrieval of individual images, the general topic of image browsing and indexing, and retrieval of groups of images has not been adequately investigated. Comparisons of descriptions of individual images as well as labels of groups of images supplied by users using cognitive models are scarce. This work fills this gap. Using the basic level theory as a framework, a comparison of the descriptions of individual images and labels assigned to groups of images by 180 participants in three studies found a marked difference in their level of abstraction. Results confirm assertions by previous researchers in LIS and other fields that groups of images are labeled using more superordinate level terms while individual image descriptions are mainly at the basic level. Implications for design of image browsing interfaces, taxonomies, thesauri, and similar tools are discussed