Construction of quasi-Monte Carlo rules for multivariate integration in spaces of permutation-invariant functions

We study multivariate integration of functions that are invariant under the permutation (of a subset) of their arguments. Recently, in Nuyens, Suryanarayana, and Weimar (Adv. Comput. Math. (2016), 42(1):55--84), the authors derived an upper estimate for the $n$th minimal worst case error for such problems, and showed that under certain conditions this upper bound only weakly depends on the dimension. We extend these results by proposing two (semi-) explicit construction schemes. We develop a component-by-component algorithm to find the generating vector for a shifted rank-$1$ lattice rule that obtains a rate of convergence arbitrarily close to $\mathcal{O}(n^{-\alpha})$, where $\alpha>1/2$ denotes the smoothness of our function space and $n$ is the number of cubature nodes. Further, we develop a semi-constructive algorithm that builds on point sets which can be used to approximate the integrands of interest with a small error; the cubature error is then bounded by the error of approximation. Here the same rate of convergence is achieved while the dependence of the error bounds on the dimension $d$ is significantly improved

Some Results on the Complexity of Numerical Integration

This is a survey (21 pages, 124 references) written for the MCQMC 2014 conference in Leuven, April 2014. We start with the seminal paper of Bakhvalov (1959) and end with new results on the curse of dimension and on the complexity of oscillatory integrals. Some small errors of earlier versions are corrected

Average Case Tractability of Non-homogeneous Tensor Product Problems

We study d-variate approximation problems in the average case setting with respect to a zero-mean Gaussian measure. Our interest is focused on measures having a structure of non-homogeneous linear tensor product, where covariance kernel is a product of univariate kernels. We consider the normalized average error of algorithms that use finitely many evaluations of arbitrary linear functionals. The information complexity is defined as the minimal number n(h,d) of such evaluations for error in the d-variate case to be at most h. The growth of n(h,d) as a function of h^{-1} and d depends on the eigenvalues of the covariance operator and determines whether a problem is tractable or not. Four types of tractability are studied and for each of them we find the necessary and sufficient conditions in terms of the eigenvalues of univariate kernels. We illustrate our results by considering approximation problems related to the product of Korobov kernels characterized by a weights g_k and smoothnesses r_k. We assume that weights are non-increasing and smoothness parameters are non-decreasing. Furthermore they may be related, for instance g_k=g(r_k) for some non-increasing function g. In particular, we show that approximation problem is strongly polynomially tractable, i.e., n(h,d)\le C h^{-p} for all d and 0<h<1, where C and p are independent of h and d, iff liminf |ln g_k|/ln k >1. For other types of tractability we also show necessary and sufficient conditions in terms of the sequences g_k and r_k

Rank-1 lattice rules for multivariate integration in spaces of permutation-invariant functions: Error bounds and tractability

We study multivariate integration of functions that are invariant under permutations (of subsets) of their arguments. We find an upper bound for the $n$th minimal worst case error and show that under certain conditions, it can be bounded independent of the number of dimensions. In particular, we study the application of unshifted and randomly shifted rank-$1$ lattice rules in such a problem setting. We derive conditions under which multivariate integration is polynomially or strongly polynomially tractable with the Monte Carlo rate of convergence $O(n^{-1/2})$. Furthermore, we prove that those tractability results can be achieved with shifted lattice rules and that the shifts are indeed necessary. Finally, we show the existence of rank-$1$ lattice rules whose worst case error on the permutation- and shift-invariant spaces converge with (almost) optimal rate. That is, we derive error bounds of the form $O(n^{-\lambda/2})$ for all $1 \leq \lambda < 2 \alpha$, where $\alpha$ denotes the smoothness of the spaces. Keywords: Numerical integration, Quadrature, Cubature, Quasi-Monte Carlo methods, Rank-1 lattice rules.Comment: 26 pages; minor changes due to reviewer's comments; the final publication is available at link.springer.co

Bethe Projections for Non-Local Inference

Many inference problems in structured prediction are naturally solved by augmenting a tractable dependency structure with complex, non-local auxiliary objectives. This includes the mean field family of variational inference algorithms, soft- or hard-constrained inference using Lagrangian relaxation or linear programming, collective graphical models, and forms of semi-supervised learning such as posterior regularization. We present a method to discriminatively learn broad families of inference objectives, capturing powerful non-local statistics of the latent variables, while maintaining tractable and provably fast inference using non-Euclidean projected gradient descent with a distance-generating function given by the Bethe entropy. We demonstrate the performance and flexibility of our method by (1) extracting structured citations from research papers by learning soft global constraints, (2) achieving state-of-the-art results on a widely-used handwriting recognition task using a novel learned non-convex inference procedure, and (3) providing a fast and highly scalable algorithm for the challenging problem of inference in a collective graphical model applied to bird migration.Comment: minor bug fix to appendix. appeared in UAI 201