569,825 research outputs found
GuessWhat?! Visual object discovery through multi-modal dialogue
We introduce GuessWhat?!, a two-player guessing game as a testbed for
research on the interplay of computer vision and dialogue systems. The goal of
the game is to locate an unknown object in a rich image scene by asking a
sequence of questions. Higher-level image understanding, like spatial reasoning
and language grounding, is required to solve the proposed task. Our key
contribution is the collection of a large-scale dataset consisting of 150K
human-played games with a total of 800K visual question-answer pairs on 66K
images. We explain our design decisions in collecting the dataset and introduce
the oracle and questioner tasks that are associated with the two players of the
game. We prototyped deep learning models to establish initial baselines of the
introduced tasks.Comment: 23 pages; CVPR 2017 submission; see https://guesswhat.a
Automatic Concept Discovery from Parallel Text and Visual Corpora
Humans connect language and vision to perceive the world. How to build a
similar connection for computers? One possible way is via visual concepts,
which are text terms that relate to visually discriminative entities. We
propose an automatic visual concept discovery algorithm using parallel text and
visual corpora; it filters text terms based on the visual discriminative power
of the associated images, and groups them into concepts using visual and
semantic similarities. We illustrate the applications of the discovered
concepts using bidirectional image and sentence retrieval task and image
tagging task, and show that the discovered concepts not only outperform several
large sets of manually selected concepts significantly, but also achieves the
state-of-the-art performance in the retrieval task.Comment: To appear in ICCV 201
AI for public health: Self-screening for eye diseases
A software-based visual-field testing (perimetry) system is described which incorporates several AI components, including machine learning, an intelligent user interface and pattern discovery. This system has been successfully used for self-screening in several different public environment
A new optical recording medium
Method has been developed for doping lithium niobiate crystals with transition metal to increase rate at which crystal can record optical data. Discovery may facilitate development of system for analog storage of TV frames, printed pages, photographs, and other visual information
- …