The Sliding Regret in Stochastic Bandits: Discriminating Index and Randomized Policies

This paper studies the one-shot behavior of no-regret algorithms for stochastic bandits. Although many algorithms are known to be asymptotically optimal with respect to the expected regret, over a single run, their pseudo-regret seems to follow one of two tendencies: it is either smooth or bumpy. To measure this tendency, we introduce a new notion: the sliding regret, that measures the worst pseudo-regret over a time-window of fixed length sliding to infinity. We show that randomized methods (e.g. Thompson Sampling and MED) have optimal sliding regret, while index policies, although possibly asymptotically optimal for the expected regret, have the worst possible sliding regret under regularity conditions on their index (e.g. UCB, UCB-V, KL-UCB, MOSS, IMED etc.). We further analyze the average bumpiness of the pseudo-regret of index policies via the regret of exploration, that we show to be suboptimal as well.Comment: 31 page

Distances between power spectral densities

We present several natural notions of distance between spectral density functions of (discrete-time) random processes. They are motivated by certain filtering problems. First we quantify the degradation of performance of a predictor which is designed for a particular spectral density function and then it is used to predict the values of a random process having a different spectral density. The logarithm of the ratio between the variance of the error, over the corresponding minimal (optimal) variance, produces a measure of distance between the two power spectra with several desirable properties. Analogous quantities based on smoothing problems produce alternative distances and suggest a class of measures based on fractions of generalized means of ratios of power spectral densities. These distance measures endow the manifold of spectral density functions with a (pseudo) Riemannian metric. We pursue one of the possible options for a distance measure, characterize the relevant geodesics, and compute corresponding distances.Comment: 16 pages, 4 figures; revision (July 29, 2006) includes two added section

Convex Optimization Based Bit Allocation for Light Field Compression under Weighting and Consistency Constraints

Compared with conventional image and video, light field images introduce the weight channel, as well as the visual consistency of rendered view, information that has to be taken into account when compressing the pseudo-temporal-sequence (PTS) created from light field images. In this paper, we propose a novel frame level bit allocation framework for PTS coding. A joint model that measures weighted distortion and visual consistency, combined with an iterative encoding system, yields the optimal bit allocation for each frame by solving a convex optimization problem. Experimental results show that the proposed framework is effective in producing desired distortion distribution based on weights, and achieves up to 24.7% BD-rate reduction comparing to the default rate control algorithm.Comment: published in IEEE Data Compression Conference, 201

Conditional convex orders and measurable martingale couplings

Strassen's classical martingale coupling theorem states that two real-valued random variables are ordered in the convex (resp.\ increasing convex) stochastic order if and only if they admit a martingale (resp.\ submartingale) coupling. By analyzing topological properties of spaces of probability measures equipped with a Wasserstein metric and applying a measurable selection theorem, we prove a conditional version of this result for real-valued random variables conditioned on a random element taking values in a general measurable space. We also provide an analogue of the conditional martingale coupling theorem in the language of probability kernels and illustrate how this result can be applied in the analysis of pseudo-marginal Markov chain Monte Carlo algorithms. We also illustrate how our results imply the existence of a measurable minimiser in the context of martingale optimal transport.Comment: 21 page

A remark on the optimal transport between two probability measures sharing the same copula

International audienceWe are interested in the Wasserstein distance between two probability measures on $\R^n$ sharing the same copula $C$. The image of the probability measure $dC$ by the vectors of pseudo-inverses of marginal distributions is a natural generalization of the coupling known to be optimal in dimension $n=1$. It turns out that for cost functions $c(x,y)$ equal to the $p$-th power of the $L^q$ norm of $x-y$ in $\R^n$, this coupling is optimal only when $p=q$ i.e. when $c(x,y)$ may be decomposed as the sum of coordinate-wise costs