Learning Visual Features from Snapshots for Web Search

When applying learning to rank algorithms to Web search, a large number of features are usually designed to capture the relevance signals. Most of these features are computed based on the extracted textual elements, link analysis, and user logs. However, Web pages are not solely linked texts, but have structured layout organizing a large variety of elements in different styles. Such layout itself can convey useful visual information, indicating the relevance of a Web page. For example, the query-independent layout (i.e., raw page layout) can help identify the page quality, while the query-dependent layout (i.e., page rendered with matched query words) can further tell rich structural information (e.g., size, position and proximity) of the matching signals. However, such visual information of layout has been seldom utilized in Web search in the past. In this work, we propose to learn rich visual features automatically from the layout of Web pages (i.e., Web page snapshots) for relevance ranking. Both query-independent and query-dependent snapshots are considered as the new inputs. We then propose a novel visual perception model inspired by human's visual search behaviors on page viewing to extract the visual features. This model can be learned end-to-end together with traditional human-crafted features. We also show that such visual features can be efficiently acquired in the online setting with an extended inverted indexing scheme. Experiments on benchmark collections demonstrate that learning visual features from Web page snapshots can significantly improve the performance of relevance ranking in ad-hoc Web retrieval tasks.Comment: CIKM 201

Vision systems with the human in the loop

The emerging cognitive vision paradigm deals with vision systems that apply machine learning and automatic reasoning in order to learn from what they perceive. Cognitive vision systems can rate the relevance and consistency of newly acquired knowledge, they can adapt to their environment and thus will exhibit high robustness. This contribution presents vision systems that aim at flexibility and robustness. One is tailored for content-based image retrieval, the others are cognitive vision systems that constitute prototypes of visual active memories which evaluate, gather, and integrate contextual knowledge for visual analysis. All three systems are designed to interact with human users. After we will have discussed adaptive content-based image retrieval and object and action recognition in an office environment, the issue of assessing cognitive systems will be raised. Experiences from psychologically evaluated human-machine interactions will be reported and the promising potential of psychologically-based usability experiments will be stressed

Visibility graphs and landscape visibility analysis

Visibility analysis based on viewsheds is one of the most frequently used GIS analysis tools. In this paper we present an approach to visibility analysis based on the visibility graph. A visibility graph records the pattern of mutual visibility relations in a landscape, and provides a convenient way of storing and further analysing the results of multiple viewshed analyses for a particular landscape region. We describe how a visibility graph may be calculated for a landscape. We then give examples, which include the interactive exploration ofa landscape, and the calculation of new measures of a landscape?s visual properties based on graph metrics ? in particular, neighbourhood clustering coefficient and path length analysis. These analyses suggest that measures derived from the visibility graph may be of particular relevance to the growing interest in quantifying the perceptual characteristics of landscapes

Visual control of flight speed in Drosophila melanogaster

Flight control in insects depends on self-induced image motion (optic flow), which the visual system must process to generate appropriate corrective steering maneuvers. Classic experiments in tethered insects applied rigorous system identification techniques for the analysis of turning reactions in the presence of rotating pattern stimuli delivered in open-loop. However, the functional relevance of these measurements for visual free-flight control remains equivocal due to the largely unknown effects of the highly constrained experimental conditions. To perform a systems analysis of the visual flight speed response under free-flight conditions, we implemented a `one-parameter open-loop' paradigm using `TrackFly' in a wind tunnel equipped with real-time tracking and virtual reality display technology. Upwind flying flies were stimulated with sine gratings of varying temporal and spatial frequencies, and the resulting speed responses were measured from the resulting flight speed reactions. To control flight speed, the visual system of the fruit fly extracts linear pattern velocity robustly over a broad range of spatio–temporal frequencies. The speed signal is used for a proportional control of flight speed within locomotor limits. The extraction of pattern velocity over a broad spatio–temporal frequency range may require more sophisticated motion processing mechanisms than those identified in flies so far. In Drosophila, the neuromotor pathways underlying flight speed control may be suitably explored by applying advanced genetic techniques, for which our data can serve as a baseline. Finally, the high-level control principles identified in the fly can be meaningfully transferred into a robotic context, such as for the robust and efficient control of autonomous flying micro air vehicles

Visual Research: An Introduction to Research Methodologies in Graphic Design

"Visual Research: An Introduction to Research Methodologies in Graphic Design" is a guide to the practice of researching for graphic design projects. This book explains the key terms and theories that underlie design research; examining the importance of audience, communication theory, semiotics and semantics. It features a range of case studies that demonstrate how the use of rigorous research methods can form the basis of effective visual communication and design problem solving, eschewing end product analysis for a discussion of the way research feeds into the design process. Synopsis of Chapter 1: The Role of Research in Graphic Design. Research methodologies for graphic design is a broad field which encompasses a wide range of practical and theoretical applications. This chapter introduces the field of design research as both an analytical and a practical tool for graphic designers, and establishes the role of critical thinking as a support to the development of an engaged design practice. The primary theoretical models of design analysis are also introduced, including semiotics, communication theory, systematic approaches, semantics and discourse theory, and their relevance to the wider graphic design profession established. The emphasis here is on why we do what we do and how we can be sure it is effective, through testing, feedback and rigorous approaches to design. The second edition includes twelve new international case studies, end of chapter exercises, a new chapter on Visual Grammar and a foreword by Ellen Lupton, an internationally renowned graphic designer, writer, curator and educator

Dublin City University video track experiments for TREC 2003

In this paper, we describe our experiments for both the News Story Segmentation task and Interactive Search task for TRECVID 2003. Our News Story Segmentation task involved the use of a Support Vector Machine (SVM) to combine evidence from audio-visual analysis tools in order to generate a listing of news stories from a given news programme. Our Search task experiment compared a video retrieval system based on text, image and relevance feedback with a text-only video retrieval system in order to identify which was more effective. In order to do so we developed two variations of our Físchlár video retrieval system and conducted user testing in a controlled lab environment. In this paper we outline our work on both of these two tasks

Visual sociology: the visual content of internet for the investigation of the city image

The relevance of this article is to consider the potential of visual sociology method of content analysis of visual objects for investigation of the city image. A careful account is given to the experience of employment this method for researching the visual content of the Internet as the forming factor of Yekaterinburg’s image.В статье раскрываются возможности применения метода визуальной социологии – анализа содержания визуальных объектов – для исследования имиджа города, рассматривается опыт применения данного метода для изучения визуального контенета сети Интернет как фактора формирования имиджа Екатеринбурга

The potential of methods of visual sociology for the investigation of the city image

В статье раскрываются возможности применения метода визуальной социологии – анализа содержания визуальных объектов – для исследования имиджа города, рассматривается опыт применения данного метода для изучения визуального контента сети Интернет как фактора формирования имиджа Екатеринбурга.The relevance of this article is to consider the potential of visual sociology method of content analysis of visual objects for investigation of the city image. A careful account is given to the experience of employment this method for researching the visual content of the Internet as the forming factor of Yekaterinburg’s image