Entropy Concentration and the Empirical Coding Game

We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's `conditional limit theorem'. The theorems characterize exactly in what sense a prior distribution Q conditioned on a given constraint, and the distribution P, minimizing the relative entropy D(P ||Q) over all distributions satisfying the constraint, are `close' to each other. We then apply our theorems to establish the relationship between entropy concentration and a game-theoretic characterization of Maximum Entropy Inference due to Topsoe and others.Comment: A somewhat modified version of this paper was published in Statistica Neerlandica 62(3), pages 374-392, 200

Taylor- and fugacity expansion for the effective center model of QCD at finite density

Using the effective center model of QCD we test series expansions for finite chemical potential $\mu$. In particular we study two variants of Taylor expansion as well as the fugacity series. The effective center model has a dual representation where the sign problem is absent and reliable Monte Carlo simulations are possible at arbitrary $\mu$. We use the results from the dual simulation as reference data to assess the Taylor- and fugacity series approaches. We find that for most of parameter space fugacity expansion is the best (but also numerically most expensive) choice for reproducing the dual simulation results, while conventional Taylor expansion is reliable only for very small $\mu$. We also discuss the results of a modified Taylor expansion in $e^{\pm \mu} - 1$ which at the same numerical effort clearly outperforms the conventional Taylor series.Comment: presented at the 31st International Symposium on Lattice Field Theory (Lattice 2013), 29 July - 3 August 2013, Mainz, Germany. Reference adde

PAC-Bayesian Theory Meets Bayesian Inference

We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization risk bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by an i.i.d distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.Comment: Published at NIPS 2015 (http://papers.nips.cc/paper/6569-pac-bayesian-theory-meets-bayesian-inference

Numerical simulations of anomalous diffusion

In this paper we present numerical methods - finite differences and finite elements - for solution of partial differential equation of fractional order in time for one-dimensional space. This equation describes anomalous diffusion which is a phenomenon connected with the interactions within the complex and non-homogeneous background. In order to consider physical initial-value conditions we use fractional derivative in the Caputo sense. In numerical analysis the boundary conditions of first kind are accounted and in the final part of this paper the result of simulations are presented.Comment: 5 pages, 2 figures, CMM 2003 Conference Gliwice/Wisla Polan