4,754 research outputs found
On the Properties of Simulation-based Estimators in High Dimensions
Considering the increasing size of available data, the need for statistical
methods that control the finite sample bias is growing. This is mainly due to
the frequent settings where the number of variables is large and allowed to
increase with the sample size bringing standard inferential procedures to incur
significant loss in terms of performance. Moreover, the complexity of
statistical models is also increasing thereby entailing important computational
challenges in constructing new estimators or in implementing classical ones. A
trade-off between numerical complexity and statistical properties is often
accepted. However, numerically efficient estimators that are altogether
unbiased, consistent and asymptotically normal in high dimensional problems
would generally be ideal. In this paper, we set a general framework from which
such estimators can easily be derived for wide classes of models. This
framework is based on the concepts that underlie simulation-based estimation
methods such as indirect inference. The approach allows various extensions
compared to previous results as it is adapted to possibly inconsistent
estimators and is applicable to discrete models and/or models with a large
number of parameters. We consider an algorithm, namely the Iterative Bootstrap
(IB), to efficiently compute simulation-based estimators by showing its
convergence properties. Within this framework we also prove the properties of
simulation-based estimators, more specifically the unbiasedness, consistency
and asymptotic normality when the number of parameters is allowed to increase
with the sample size. Therefore, an important implication of the proposed
approach is that it allows to obtain unbiased estimators in finite samples.
Finally, we study this approach when applied to three common models, namely
logistic regression, negative binomial regression and lasso regression
Quantization of Prior Probabilities for Hypothesis Testing
Bayesian hypothesis testing is investigated when the prior probabilities of
the hypotheses, taken as a random vector, are quantized. Nearest neighbor and
centroid conditions are derived using mean Bayes risk error as a distortion
measure for quantization. A high-resolution approximation to the
distortion-rate function is also obtained. Human decision making in segregated
populations is studied assuming Bayesian hypothesis testing with quantized
priors
Differentiable Game Mechanics
Deep learning is built on the foundational guarantee that gradient descent on
an objective function converges to local minima. Unfortunately, this guarantee
fails in settings, such as generative adversarial nets, that exhibit multiple
interacting losses. The behavior of gradient-based methods in games is not well
understood -- and is becoming increasingly important as adversarial and
multi-objective architectures proliferate. In this paper, we develop new tools
to understand and control the dynamics in n-player differentiable games.
The key result is to decompose the game Jacobian into two components. The
first, symmetric component, is related to potential games, which reduce to
gradient descent on an implicit function. The second, antisymmetric component,
relates to Hamiltonian games, a new class of games that obey a conservation law
akin to conservation laws in classical mechanical systems. The decomposition
motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding
stable fixed points in differentiable games. Basic experiments show SGA is
competitive with recently proposed algorithms for finding stable fixed points
in GANs -- while at the same time being applicable to, and having guarantees
in, much more general cases.Comment: JMLR 2019, journal version of arXiv:1802.0564
- …