27,635 research outputs found
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
While it is nearly effortless for humans to quickly assess the perceptual
similarity between two images, the underlying processes are thought to be quite
complex. Despite this, the most widely used perceptual metrics today, such as
PSNR and SSIM, are simple, shallow functions, and fail to account for many
nuances of human perception. Recently, the deep learning community has found
that features of the VGG network trained on ImageNet classification has been
remarkably useful as a training loss for image synthesis. But how perceptual
are these so-called "perceptual losses"? What elements are critical for their
success? To answer these questions, we introduce a new dataset of human
perceptual similarity judgments. We systematically evaluate deep features
across different architectures and tasks and compare them with classic metrics.
We find that deep features outperform all previous metrics by large margins on
our dataset. More surprisingly, this result is not restricted to
ImageNet-trained VGG features, but holds across different deep architectures
and levels of supervision (supervised, self-supervised, or even unsupervised).
Our results suggest that perceptual similarity is an emergent property shared
across deep visual representations.Comment: Accepted to CVPR 2018; Code and data available at
https://www.github.com/richzhang/PerceptualSimilarit
An Alarm System For Segmentation Algorithm Based On Shape Model
It is usually hard for a learning system to predict correctly on rare events
that never occur in the training data, and there is no exception for
segmentation algorithms. Meanwhile, manual inspection of each case to locate
the failures becomes infeasible due to the trend of large data scale and
limited human resource. Therefore, we build an alarm system that will set off
alerts when the segmentation result is possibly unsatisfactory, assuming no
corresponding ground truth mask is provided. One plausible solution is to
project the segmentation results into a low dimensional feature space; then
learn classifiers/regressors to predict their qualities. Motivated by this, in
this paper, we learn a feature space using the shape information which is a
strong prior shared among different datasets and robust to the appearance
variation of input data.The shape feature is captured using a Variational
Auto-Encoder (VAE) network that trained with only the ground truth masks.
During testing, the segmentation results with bad shapes shall not fit the
shape prior well, resulting in large loss values. Thus, the VAE is able to
evaluate the quality of segmentation result on unseen data, without using
ground truth. Finally, we learn a regressor in the one-dimensional feature
space to predict the qualities of segmentation results. Our alarm system is
evaluated on several recent state-of-art segmentation algorithms for 3D medical
segmentation tasks. Compared with other standard quality assessment methods,
our system consistently provides more reliable prediction on the qualities of
segmentation results.Comment: Accepted to ICCV 2019 (10 pages, 4 figures
- …