10 research outputs found
Towards Query-Efficient Black-Box Adversary with Zeroth-Order Natural Gradient Descent
Despite the great achievements of the modern deep neural networks (DNNs), the
vulnerability/robustness of state-of-the-art DNNs raises security concerns in
many application domains requiring high reliability. Various adversarial
attacks are proposed to sabotage the learning performance of DNN models. Among
those, the black-box adversarial attack methods have received special
attentions owing to their practicality and simplicity. Black-box attacks
usually prefer less queries in order to maintain stealthy and low costs.
However, most of the current black-box attack methods adopt the first-order
gradient descent method, which may come with certain deficiencies such as
relatively slow convergence and high sensitivity to hyper-parameter settings.
In this paper, we propose a zeroth-order natural gradient descent (ZO-NGD)
method to design the adversarial attacks, which incorporates the zeroth-order
gradient estimation technique catering to the black-box attack scenario and the
second-order natural gradient descent to achieve higher query efficiency. The
empirical evaluations on image classification datasets demonstrate that ZO-NGD
can obtain significantly lower model query complexities compared with
state-of-the-art attack methods.Comment: accepted by AAAI 202
Curvature-Aware Derivative-Free Optimization
The paper discusses derivative-free optimization (DFO), which involves
minimizing a function without access to gradients or directional derivatives,
only function evaluations. Classical DFO methods, which mimic gradient-based
methods, such as Nelder-Mead and direct search have limited scalability for
high-dimensional problems. Zeroth-order methods have been gaining popularity
due to the demands of large-scale machine learning applications, and the paper
focuses on the selection of the step size in these methods. The
proposed approach, called Curvature-Aware Random Search (CARS), uses first- and
second-order finite difference approximations to compute a candidate
. We prove that for strongly convex objective functions, CARS
converges linearly provided that the search direction is drawn from a
distribution satisfying very mild conditions. We also present a Cubic
Regularized variant of CARS, named CARS-CR, which converges in a rate of
without the assumption of strong convexity. Numerical
experiments show that CARS and CARS-CR match or exceed the state-of-the-arts on
benchmark problem sets.Comment: 31 pages, 9 figure