Vision systems with the human in the loop

The emerging cognitive vision paradigm deals with vision systems that apply machine learning and automatic reasoning in order to learn from what they perceive. Cognitive vision systems can rate the relevance and consistency of newly acquired knowledge, they can adapt to their environment and thus will exhibit high robustness. This contribution presents vision systems that aim at flexibility and robustness. One is tailored for content-based image retrieval, the others are cognitive vision systems that constitute prototypes of visual active memories which evaluate, gather, and integrate contextual knowledge for visual analysis. All three systems are designed to interact with human users. After we will have discussed adaptive content-based image retrieval and object and action recognition in an office environment, the issue of assessing cognitive systems will be raised. Experiences from psychologically evaluated human-machine interactions will be reported and the promising potential of psychologically-based usability experiments will be stressed

Research on Image Retrieval Optimization Based on Eye Movement Experiment Data

Satisfying a user's actual underlying needs in the image retrieval process is a difficult challenge facing image retrieval technology. The aim of this study is to improve the performance of a retrieval system and provide users with optimized search results using the feedback of eye movement. We analyzed the eye movement signals of the user’s image retrieval process from cognitive and mathematical perspectives. Data collected for 25 designers in eye tracking experiments were used to train and evaluate the model. In statistical analysis, eight eye movement features were statistically significantly different between selected and unselected groups of images (p < 0.05). An optimal selection of input features resulted in overall accuracy of the support vector machine prediction model of 87.16%. Judging the user’s requirements in the image retrieval process through eye movement behaviors was shown to be effective

A words-of-interest model of sketch representation for image retrieval

In this paper we propose a method for sketch-based image retrieval. Sketch is a magical medium which is capable of conveying semantic messages for user. It’s in accordance with user’s cognitive psychology to retrieve images with sketch. In order to narrow down the semantic gap between the user and the images in database, we preprocess all the images into sketches by the coherent line drawing algorithm. During the process of sketches extraction, saliency maps are used to filter out the redundant background information, while preserve the important semantic information. We use a variant of Words-of-Interest model to retrieve relevant images for the user according to the query. Words-of-Interest (WoI) model is based on Bag-ofvisual Words (BoW) model, which has been proven successfully for information retrieval. Bag-of-Words ignores the spatial relationships among visual words, which are important for sketch representation. Our method takes advantage of the spatial information of the query to select words of interest. Experimental results demonstrate that our sketch-based retrieval method achieves a good tradeoff between retrieval accuracy and semantic representation of users’ query

Representativeness and Diversity in Photos via Crowd-Sourced Media Analysis

In this paper we present a hybrid three steps mechanism for automated-human media analysis employed for selecting a small number of representative and diverse images in the context of a noisy set of images. The first step consists in the automatic retrieval from web of a large database of candidate images. In the second step, a proposed image analysis method is employed with the goal of diminishing the time, pay and cognitive load and implicitly people’s work. This is done by automatically selecting a set of potentially relevant and diverse images. Considering the semantic gap between low-level features and high-level semantics in images, the last step is necessary and consists in images being annotated and assessed by the crowd. The aim is to evaluate the level of representativeness and diversity of the selected set of images and providing images of highest quality. The method was validated in the context of the retrieval of images with monuments and using more than 30,000 images retrieved from various social image search platforms

Modeling human color categorization: color discrimination and color memory

Color matching in Content-Based Image Retrieval is done using a color space and measuring distances between colors. Such an approach yields non-intuitive results for the user. We introduce color categories (or focal colors), determine that they are valid, and use them in two experiments. The experiments conducted prove the difference between color categorization by the cognitive processes color discrimination and color memory. In addition, they yield a Color Look-Up Table, which can improve color matching, that can be seen as a model for human color matching

Cognitive abstraction approach to sketch-based image retrieval

Thesis (S.B. and M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1999.Includes bibliographical references (leaves 151-157).This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.As digital media become more popular, corporations and individuals gather an increasingly large number of digital images. As a collection grows to more than a few hundred images, the need for search becomes crucial. This thesis is addressing the problem of retrieving from a small database a particular image previously seen by the user. This thesis combines current findings in cognitive science with the knowledge of previous image retrieval systems to present a novel approach to content based image retrieval and indexing. We focus on algorithms which abstract away information from images in the same terms that a viewer abstracts information from an image. The focus in Imagina is on the matching of regions, instead of the matching of global measures. Multiple representations, focusing on shape and color, are used for every region. The matches of individual regions are combined using a saliency metric that accounts for differences in the distributions of metrics. Region matching along with configuration determines the overall match between a query and an image.by Manolis Kamvysselis and Ovidiu Marina.S.B.and M.Eng

Cognitive Reserve and Its Effect in Older Adults on Retrieval of Proper Names, Logo Names and Common Nouns

Previous studies showed that high Cognitive Reserve (CR, years of education and experience and knowledge acquired in life) is correlated with language proficiency as measured with vocabulary size, verbal analogy, and semantic processing. The aim of the present study is to investigate the relationship between CR and the ability in retrieving different categories of words: Proper Names, Logo Names, and Common Nouns. The hypothesis is that CR contributes more in retrieving Common Nouns and Logo Names which are highly semantically interconnected, than retrieving Proper Names which are pure referring expressions. Forty-six Italian healthy older adults underwent the Montreal Cognitive Assessment (MoCA) and their performances spanned from low to high global cognitive profile. They were also administered a picture naming task for Proper Names, Logo Names and Common Nouns. Latency and Accuracy were recorded. CR was measured with the Cognitive Reserve Index (CRI) questionnaire which provides a measure of education, working time activities, and leisure time activities. Participants were significantly faster and more accurate in name retrieval when CR was high. CRI and MoCA as interaction terms predicted naming Latency with a stronger effect of CRI when the global cognitive profile was in the low range. The effect of CRI on Accuracy was lower for Proper Names than for Common Nouns and Logo Names, which did not differ from each other. Our results show that name retrieval Accuracy can be predicted by CR, significantly more in the case of Logo Names and Common Nouns than in the case of Proper Names. As Proper Names have scarce semantic interconnections and are arbitrarily assigned to unique individuals, they are not much influenced by CR. Although Logo Names are also arbitrarily assigned to their bearers, they can be conceptually categorized and thus influenced by reserve. The weak relationship between Proper Names and CR might suggest a proper name task as a useful tool to detect early signs of dementia, in particular for persons with high CR

Visual object imagery and autobiographical memory: object imagers are better at remembering their personal past

In the present study we examined whether higher levels of object imagery, a stable characteristic that reflects the ability and preference in generating pictorial mental images of objects, facilitate involuntary and voluntary retrieval of autobiographical memories (ABMs). Individuals with high (High-OI) and low (Low-OI) levels of object imagery were asked to perform an involuntary and a voluntary ABM task in the laboratory. Results showed that High-OI participants generated more involuntary and voluntary ABMs than Low-OI, with faster retrieval times. High-OI also reported more detailed memories compared to Low-OI and retrieved memories as visual images. Theoretical implications of these findings for research on voluntary and involuntary ABMs are discussed