14,005 research outputs found
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
While it is nearly effortless for humans to quickly assess the perceptual
similarity between two images, the underlying processes are thought to be quite
complex. Despite this, the most widely used perceptual metrics today, such as
PSNR and SSIM, are simple, shallow functions, and fail to account for many
nuances of human perception. Recently, the deep learning community has found
that features of the VGG network trained on ImageNet classification has been
remarkably useful as a training loss for image synthesis. But how perceptual
are these so-called "perceptual losses"? What elements are critical for their
success? To answer these questions, we introduce a new dataset of human
perceptual similarity judgments. We systematically evaluate deep features
across different architectures and tasks and compare them with classic metrics.
We find that deep features outperform all previous metrics by large margins on
our dataset. More surprisingly, this result is not restricted to
ImageNet-trained VGG features, but holds across different deep architectures
and levels of supervision (supervised, self-supervised, or even unsupervised).
Our results suggest that perceptual similarity is an emergent property shared
across deep visual representations.Comment: Accepted to CVPR 2018; Code and data available at
https://www.github.com/richzhang/PerceptualSimilarit
Head-mounted spatial instruments II: Synthetic reality or impossible dream
A spatial instrument is defined as a spatial display which has been either geometrically or symbolically enhanced to enable a user to accomplish a particular task. Research conducted over the past several years on 3-D spatial instruments has shown that perspective displays, even when viewed from the correct viewpoint, are subject to systematic viewer biases. These biases interfere with correct spatial judgements of the presented pictorial information. The design of spatial instruments may not only require the introduction of compensatory distortions to remove the naturally occurring biases but also may significantly benefit from the introduction of artificial distortions which enhance performance. However, these image manipulations can cause a loss of visual-vestibular coordination and induce motion sickness. Consequently, the design of head-mounted spatial instruments will require an understanding of the tolerable limits of visual-vestibular discord
- …