6 research outputs found
An empirical evaluation of adversarial robustness under transfer learning
In this work, we evaluate adversarial robustness in the context of transfer
learning from a source trained on CIFAR 100 to a target network trained on
CIFAR 10. Specifically, we study the effects of using robust optimisation in
the source and target networks. This allows us to identify transfer learning
strategies under which adversarial defences are successfully retained, in
addition to revealing potential vulnerabilities. We study the extent to which
features learnt by a fast gradient sign method (FGSM) and its iterative
alternative (PGD) can preserve their defence properties against black and
white-box attacks under three different transfer learning strategies. We find
that using PGD examples during training on the source task leads to more
general robust features that are easier to transfer. Furthermore, under
successful transfer, it achieves 5.2% more accuracy against white-box PGD
attacks than suitable baselines. Overall, our empirical evaluations give
insights on how well adversarial robustness under transfer learning can
generalise
Score Normalization for a Faster Diffusion Exponential Integrator Sampler
Recently, Zhang et al. have proposed the Diffusion Exponential Integrator
Sampler (DEIS) for fast generation of samples from Diffusion Models. It
leverages the semi-linear nature of the probability flow ordinary differential
equation (ODE) in order to greatly reduce integration error and improve
generation quality at low numbers of function evaluations (NFEs). Key to this
approach is the score function reparameterisation, which reduces the
integration error incurred from using a fixed score function estimate over each
integration step. The original authors use the default parameterisation used by
models trained for noise prediction -- multiply the score by the standard
deviation of the conditional forward noising distribution. We find that
although the mean absolute value of this score parameterisation is close to
constant for a large portion of the reverse sampling process, it changes
rapidly at the end of sampling. As a simple fix, we propose to instead
reparameterise the score (at inference) by dividing it by the average absolute
value of previous score estimates at that time step collected from offline high
NFE generations. We find that our score normalisation (DEIS-SN) consistently
improves FID compared to vanilla DEIS, showing an improvement at 10 NFEs from
6.44 to 5.57 on CIFAR-10 and from 5.9 to 4.95 on LSUN-Church 64x64. Our code is
available at https://github.com/mtkresearch/Diffusion-DEIS-S
Image generation with shortest path diffusion
The field of image generation has made significant progress thanks to the
introduction of Diffusion Models, which learn to progressively reverse a given
image corruption. Recently, a few studies introduced alternative ways of
corrupting images in Diffusion Models, with an emphasis on blurring. However,
these studies are purely empirical and it remains unclear what is the optimal
procedure for corrupting an image. In this work, we hypothesize that the
optimal procedure minimizes the length of the path taken when corrupting an
image towards a given final state. We propose the Fisher metric for the path
length, measured in the space of probability distributions. We compute the
shortest path according to this metric, and we show that it corresponds to a
combination of image sharpening, rather than blurring, and noise deblurring.
While the corruption was chosen arbitrarily in previous work, our Shortest Path
Diffusion (SPD) determines uniquely the entire spatiotemporal structure of the
corruption. We show that SPD improves on strong baselines without any
hyperparameter tuning, and outperforms all previous Diffusion Models based on
image blurring. Furthermore, any small deviation from the shortest path leads
to worse performance, suggesting that SPD provides the optimal procedure to
corrupt images. Our work sheds new light on observations made in recent works
and provides a new approach to improve diffusion models on images and other
types of data.Comment: AD and SF contributed equall
Recommended from our members
Disentangled generative models for robust prediction of system dynamics
The use of deep neural networks for modelling system dynamics is increasingly popular, but long-term prediction accuracy and out-of-distribution generalization still present challenges. In this study, we address these challenges by considering the parameters of dynamical systems as factors of variation of the data and leverage their ground-truth values to disentangle the representations learned by generative models. Our experimental results in phase-space and observation-space dynamics, demonstrate the effectiveness of latent-space supervision in producing disentangled representations, leading to improved long-term prediction accuracy and out-of-distribution robustness