2,033,980 research outputs found
AdaBatch: Efficient Gradient Aggregation Rules for Sequential and Parallel Stochastic Gradient Methods
We study a new aggregation operator for gradients coming from a mini-batch
for stochastic gradient (SG) methods that allows a significant speed-up in the
case of sparse optimization problems. We call this method AdaBatch and it only
requires a few lines of code change compared to regular mini-batch SGD
algorithms. We provide a theoretical insight to understand how this new class
of algorithms is performing and show that it is equivalent to an implicit
per-coordinate rescaling of the gradients, similarly to what Adagrad methods
can do. In theory and in practice, this new aggregation allows to keep the same
sample efficiency of SG methods while increasing the batch size.
Experimentally, we also show that in the case of smooth convex optimization,
our procedure can even obtain a better loss when increasing the batch size for
a fixed number of samples. We then apply this new algorithm to obtain a
parallelizable stochastic gradient method that is synchronous but allows
speed-up on par with Hogwild! methods as convergence does not deteriorate with
the increase of the batch size. The same approach can be used to make
mini-batch provably efficient for variance-reduced SG methods such as SVRG
Numerically Computing QCD Laplace Sum-Rules Using pySecDec
pySecDec is a program that numerically calculates dimensionally regularized
integrals. We use pySecDec to compute QCD Laplace sum-rules for pseudoscalar
(i.e., ) charmonium hybrids, and compare the results to
sum-rules computed using analytic results for dimensionally regularized
integrals. We find that the errors due to the use of numerical integration
methods is negligible compared to the uncertainties in the sum-rules stemming
from the uncertainties in the parameters of QCD, e.g., the coupling constant,
quark masses, and condensate values. Also, we demonstrate that numerical
integration methods can be used to calculate finite-energy and Gaussian
sum-rules in addition to Laplace sum-rules.Comment: 10 pages, 5 figure
- …