1,968 research outputs found
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
While it is nearly effortless for humans to quickly assess the perceptual
similarity between two images, the underlying processes are thought to be quite
complex. Despite this, the most widely used perceptual metrics today, such as
PSNR and SSIM, are simple, shallow functions, and fail to account for many
nuances of human perception. Recently, the deep learning community has found
that features of the VGG network trained on ImageNet classification has been
remarkably useful as a training loss for image synthesis. But how perceptual
are these so-called "perceptual losses"? What elements are critical for their
success? To answer these questions, we introduce a new dataset of human
perceptual similarity judgments. We systematically evaluate deep features
across different architectures and tasks and compare them with classic metrics.
We find that deep features outperform all previous metrics by large margins on
our dataset. More surprisingly, this result is not restricted to
ImageNet-trained VGG features, but holds across different deep architectures
and levels of supervision (supervised, self-supervised, or even unsupervised).
Our results suggest that perceptual similarity is an emergent property shared
across deep visual representations.Comment: Accepted to CVPR 2018; Code and data available at
https://www.github.com/richzhang/PerceptualSimilarit
Galaxy shape measurement with convolutional neural networks
We present our results from training and evaluating a convolutional neural
network (CNN) to predict galaxy shapes from wide-field survey images of the
first data release of the Dark Energy Survey (DES DR1). We use conventional
shape measurements as ground truth from an overlapping, deeper survey with less
sky coverage, the Canada-France Hawaii Telescope Lensing Survey (CFHTLenS). We
demonstrate that CNN predictions from single band DES images reproduce the
results of CFHTLenS at bright magnitudes and show higher correlation with
CFHTLenS at fainter magnitudes than maximum likelihood model fitting estimates
in the DES Y1 im3shape catalogue. Prediction of shape parameters with a CNN is
also extremely fast, it takes only 0.2 milliseconds per galaxy, improving more
than 4 orders of magnitudes over forward model fitting. The CNN can also
accurately predict shapes when using multiple images of the same galaxy, even
in different color bands, with no additional computational overhead. The CNN is
again more precise for faint objects, and the advantage of the CNN is more
pronounced for blue galaxies than red ones when compared to the DES Y1
metacalibration catalogue, which fits a single Gaussian profile using riz band
images. We demonstrate that CNN shape predictions within the metacalibration
self-calibrating framework yield shear estimates with negligible multiplicative
bias, , and no significant PSF leakage. Our proposed setup is
applicable to current and next generation weak lensing surveys where higher
quality ground truth shapes can be measured in dedicated deep fields
Pushing the Boundaries of Boundary Detection using Deep Learning
In this work we show that adapting Deep Convolutional Neural Network training
to the task of boundary detection can result in substantial improvements over
the current state-of-the-art in boundary detection.
Our contributions consist firstly in combining a careful design of the loss
for boundary detection training, a multi-resolution architecture and training
with external data to improve the detection accuracy of the current state of
the art. When measured on the standard Berkeley Segmentation Dataset, we
improve theoptimal dataset scale F-measure from 0.780 to 0.808 - while human
performance is at 0.803. We further improve performance to 0.813 by combining
deep learning with grouping, integrating the Normalized Cuts technique within a
deep network.
We also examine the potential of our boundary detector in conjunction with
the task of semantic segmentation and demonstrate clear improvements over
state-of-the-art systems. Our detector is fully integrated in the popular Caffe
framework and processes a 320x420 image in less than a second.Comment: The previous version reported large improvements w.r.t. the LPO
region proposal baseline, which turned out to be due to a wrong computation
for the baseline. The improvements are currently less important, and are
omitted. We are sorry if the reported results caused any confusion. We have
also integrated reviewer feedback regarding human performance on the BSD
benchmar
Stochastic Sampling Simulation for Pedestrian Trajectory Prediction
Urban environments pose a significant challenge for autonomous vehicles (AVs)
as they must safely navigate while in close proximity to many pedestrians. It
is crucial for the AV to correctly understand and predict the future
trajectories of pedestrians to avoid collision and plan a safe path. Deep
neural networks (DNNs) have shown promising results in accurately predicting
pedestrian trajectories, relying on large amounts of annotated real-world data
to learn pedestrian behavior. However, collecting and annotating these large
real-world pedestrian datasets is costly in both time and labor. This paper
describes a novel method using a stochastic sampling-based simulation to train
DNNs for pedestrian trajectory prediction with social interaction. Our novel
simulation method can generate vast amounts of automatically-annotated,
realistic, and naturalistic synthetic pedestrian trajectories based on small
amounts of real annotation. We then use such synthetic trajectories to train an
off-the-shelf state-of-the-art deep learning approach Social GAN (Generative
Adversarial Network) to perform pedestrian trajectory prediction. Our proposed
architecture, trained only using synthetic trajectories, achieves better
prediction results compared to those trained on human-annotated real-world data
using the same network. Our work demonstrates the effectiveness and potential
of using simulation as a substitution for human annotation efforts to train
high-performing prediction algorithms such as the DNNs.Comment: 8 pages, 6 figures and 2 table
Stochastic Sampling Simulation for Pedestrian Trajectory Prediction
Urban environments pose a significant challenge for autonomous vehicles (AVs)
as they must safely navigate while in close proximity to many pedestrians. It
is crucial for the AV to correctly understand and predict the future
trajectories of pedestrians to avoid collision and plan a safe path. Deep
neural networks (DNNs) have shown promising results in accurately predicting
pedestrian trajectories, relying on large amounts of annotated real-world data
to learn pedestrian behavior. However, collecting and annotating these large
real-world pedestrian datasets is costly in both time and labor. This paper
describes a novel method using a stochastic sampling-based simulation to train
DNNs for pedestrian trajectory prediction with social interaction. Our novel
simulation method can generate vast amounts of automatically-annotated,
realistic, and naturalistic synthetic pedestrian trajectories based on small
amounts of real annotation. We then use such synthetic trajectories to train an
off-the-shelf state-of-the-art deep learning approach Social GAN (Generative
Adversarial Network) to perform pedestrian trajectory prediction. Our proposed
architecture, trained only using synthetic trajectories, achieves better
prediction results compared to those trained on human-annotated real-world data
using the same network. Our work demonstrates the effectiveness and potential
of using simulation as a substitution for human annotation efforts to train
high-performing prediction algorithms such as the DNNs.Comment: 8 pages, 6 figures and 2 table
- …