Image pattern recognition supporting interactive analysis and graphical visualization

Image Pattern Recognition attempts to infer properties of the world from image data. Such capabilities are crucial for making measurements from satellite or telescope images related to Earth and space science problems. Such measurements can be the required product itself, or the measurements can be used as input to a computer graphics system for visualization purposes. At present, the field of image pattern recognition lacks a unified scientific structure for developing and evaluating image pattern recognition applications. The overall goal of this project is to begin developing such a structure. This report summarizes results of a 3-year research effort in image pattern recognition addressing the following three principal aims: (1) to create a software foundation for the research and identify image pattern recognition problems in Earth and space science; (2) to develop image measurement operations based on Artificial Visual Systems; and (3) to develop multiscale image descriptions for use in interactive image analysis

Interactive Visual System

Mobile computing devices are being endowed with ever-increasing functionality. To demonstrate the augmentation of human cognition in an interactive visual recognition task, we reengineered a PC-based system called CAVIAR (Computer Assisted Visual Interactive System) for a handheld computer with camera system. The resulting Interactive Visual System exploits the pattern recognition capabilities of humans and the computational power of a computer to identify flowers based on features that are interactively extracted from an image and submitted for comparison to a species database. While IVS has similar functionality to that of CAVIAR, because it runs on a handheld computer, it offers complete portability for use in the field. We find that the handheld IVS and PC-based CAVIAR systems outperform humans alone both on speed and accuracy and machines alone on accuracy

An Integrated Assistance Tool for Visual Impairment

Although vision restoration is slowly becoming a reality through the various advancements in medical research and technology, the applicability of most methods are fairly limited as they only work in a few eye conditions and cannot provide help for most visually impaired people. Intelligent assistive devices however, albeit without offering visual experience, have three distinct advantages: they can be used by anyone, they have a huge potential cost advantage, and they can be brought to market much faster. We present a portable assistive device that processes the visual image flow provided by a built-in camera, and provides interactive detection and recognition functions. Pattern detection is designed to be universal, building on structural embeddedness and morphologic saliency including the Gestalt principles. Shape recognition in turn is trained for typical patterns appearing in specific situations proposed by blind experts. Testing is carried out regularly on blind subjects using experimental prototypes

Crowdsourcing in Computer Vision

Computer vision systems require large amounts of manually annotated data to properly learn challenging visual concepts. Crowdsourcing platforms offer an inexpensive method to capture human knowledge and understanding, for a vast number of visual perception tasks. In this survey, we describe the types of annotations computer vision researchers have collected using crowdsourcing, and how they have ensured that this data is of high quality while annotation effort is minimized. We begin by discussing data collection on both classic (e.g., object recognition) and recent (e.g., visual story-telling) vision tasks. We then summarize key design decisions for creating effective data collection interfaces and workflows, and present strategies for intelligently selecting the most important data instances to annotate. Finally, we conclude with some thoughts on the future of crowdsourcing in computer vision.Comment: A 69-page meta review of the field, Foundations and Trends in Computer Graphics and Vision, 201

From Codes to Patterns: Designing Interactive Decoration for Tableware

ABSTRACT We explore the idea of making aesthetic decorative patterns that contain multiple visual codes. We chart an iterative collaboration with ceramic designers and a restaurant to refine a recognition technology to work reliably on ceramics, produce a pattern book of designs, and prototype sets of tableware and a mobile app to enhance a dining experience. We document how the designers learned to work with and creatively exploit the technology, enriching their patterns with embellishments and backgrounds and developing strategies for embedding codes into complex designs. We discuss the potential and challenges of interacting with such patterns. We argue for a transition from designing ‘codes to patterns’ that reflects the skills of designers alongside the development of new technologies

Vision systems with the human in the loop

The emerging cognitive vision paradigm deals with vision systems that apply machine learning and automatic reasoning in order to learn from what they perceive. Cognitive vision systems can rate the relevance and consistency of newly acquired knowledge, they can adapt to their environment and thus will exhibit high robustness. This contribution presents vision systems that aim at flexibility and robustness. One is tailored for content-based image retrieval, the others are cognitive vision systems that constitute prototypes of visual active memories which evaluate, gather, and integrate contextual knowledge for visual analysis. All three systems are designed to interact with human users. After we will have discussed adaptive content-based image retrieval and object and action recognition in an office environment, the issue of assessing cognitive systems will be raised. Experiences from psychologically evaluated human-machine interactions will be reported and the promising potential of psychologically-based usability experiments will be stressed

Understanding person acquisition using an interactive activation and competition network

Face perception is one of the most developed visual skills that humans display, and recent work has attempted to examine the mechanisms involved in face perception through noting how neural networks achieve the same performance. The purpose of the present paper is to extend this approach to look not just at human face recognition, but also at human face acquisition. Experiment 1 presents empirical data to describe the acquisition over time of appropriate representations for newly encountered faces. These results are compared with those of Simulation 1, in which a modified IAC network capable of modelling the acquisition process is generated. Experiment 2 and Simulation 2 explore the mechanisms of learning further, and it is demonstrated that the acquisition of a set of associated new facts is easier than the acquisition of individual facts in isolation of one another. This is explained in terms of the advantage gained from additional inputs and mutual reinforcement of developing links within an interactive neural network system. <br/