Multi-Armed Bandits for Correlated Markovian Environments with Smoothed Reward Feedback

We study a multi-armed bandit problem in a dynamic environment where arm rewards evolve in a correlated fashion according to a Markov chain. Different than much of the work on related problems, in our formulation a learning algorithm does not have access to either a priori information or observations of the state of the Markov chain and only observes smoothed reward feedback following time intervals we refer to as epochs. We demonstrate that existing methods such as UCB and $\varepsilon$-greedy can suffer linear regret in such an environment. Employing mixing-time bounds on Markov chains, we develop algorithms called EpochUCB and EpochGreedy that draw inspiration from the aforementioned methods, yet which admit sublinear regret guarantees for the problem formulation. Our proposed algorithms proceed in epochs in which an arm is played repeatedly for a number of iterations that grows linearly as a function of the number of times an arm has been played in the past. We analyze these algorithms under two types of smoothed reward feedback at the end of each epoch: a reward that is the discount-average of the discounted rewards within an epoch, and a reward that is the time-average of the rewards within an epoch.Comment: Significant revision of prior version including deeper discussion of related work, gap-independent regret bounds, and regret bounds for discounted reward

Polya's inequalities, global uniform integrability and the size of plurisubharmonic lemniscates

First we prove a new inequality comparing uniformly the relative volume of a Borel subset with respect to any given complex euclidean ball \B \sub \C^n with its relative logarithmic capacity in \C^n with respect to the same ball \B. An analoguous comparison inequality for Borel subsets of euclidean balls of any generic real subspace of \C^n is also proved. Then we give several interesting applications of these inequalities. First we obtain sharp uniform estimates on the relative size of \psh lemniscates associated to the Lelong class of \psh functions of logarithmic singularities at infinity on \C^n as well as the Cegrell class of \psh functions of bounded Monge-Amp\`ere mass on a hyperconvex domain \W \Sub \C^n. Then we also deduce new results on the global behaviour of both the Lelong class and the Cegrell class of \psh functions.Comment: 25 page

Initial Data for General Relativity with Toroidal Conformal Symmetry

A new class of time-symmetric solutions to the initial value constraints of vacuum General Relativity is introduced. These data are globally regular, asymptotically flat (with possibly several asymptotic ends) and in general have no isometries, but a $U(1)\times U(1)$ group of conformal isometries. After decomposing the Lichnerowicz conformal factor in a double Fourier series on the group orbits, the solutions are given in terms of a countable family of uncoupled ODEs on the orbit space.Comment: REVTEX, 9 pages, ESI Preprint 12

Discrete complex analysis on planar quad-graphs

We develop a linear theory of discrete complex analysis on general quad-graphs, continuing and extending previous work of Duffin, Mercat, Kenyon, Chelkak and Smirnov on discrete complex analysis on rhombic quad-graphs. Our approach based on the medial graph yields more instructive proofs of discrete analogs of several classical theorems and even new results. We provide discrete counterparts of fundamental concepts in complex analysis such as holomorphic functions, derivatives, the Laplacian, and exterior calculus. Also, we discuss discrete versions of important basic theorems such as Green's identities and Cauchy's integral formulae. For the first time, we discretize Green's first identity and Cauchy's integral formula for the derivative of a holomorphic function. In this paper, we focus on planar quad-graphs, but we would like to mention that many notions and theorems can be adapted to discrete Riemann surfaces in a straightforward way. In the case of planar parallelogram-graphs with bounded interior angles and bounded ratio of side lengths, we construct a discrete Green's function and discrete Cauchy's kernels with asymptotics comparable to the smooth case. Further restricting to the integer lattice of a two-dimensional skew coordinate system yields appropriate discrete Cauchy's integral formulae for higher order derivatives.Comment: 49 pages, 8 figure

Forced Stratified Turbulence: Successive Transitions with Reynolds Number

Numerical simulations are made for forced turbulence at a sequence of increasing values of Reynolds number, R, keeping fixed a strongly stable, volume-mean density stratification. At smaller values of R, the turbulent velocity is mainly horizontal, and the momentum balance is approximately cyclostrophic and hydrostatic. This is a regime dominated by so-called pancake vortices, with only a weak excitation of internal gravity waves and large values of the local Richardson number, Ri, everywhere. At higher values of R there are successive transitions to (a) overturning motions with local reversals in the density stratification and small or negative values of Ri; (b) growth of a horizontally uniform vertical shear flow component; and (c) growth of a large-scale vertical flow component. Throughout these transitions, pancake vortices continue to dominate the large-scale part of the turbulence, and the gravity wave component remains weak except at small scales.Comment: 8 pages, 5 figures (submitted to Phys. Rev. E

Now the wars are over: The past, present and future of Scottish battlefields

Battlefield archaeology has provided a new way of appreciating historic battlefields. This paper provides a summary of the long history of warfare and conflict in Scotland which has given rise to a large number of battlefield sites. Recent moves to highlight the archaeological importance of these sites, in the form of Historic Scotland’s Battlefields Inventory are discussed, along with some of the problems associated with the preservation and management of these important cultural sites

Single and vertically coupled type II quantum dots in a perpendicular magnetic field: exciton groundstate properties

The properties of an exciton in a type II quantum dot are studied under the influence of a perpendicular applied magnetic field. The dot is modelled by a quantum disk with radius $R$, thickness $d$ and the electron is confined in the disk, whereas the hole is located in the barrier. The exciton energy and wavefunctions are calculated using a Hartree-Fock mesh method. We distinguish two different regimes, namely $d<<2R$ (the hole is located at the radial boundary of the disk) and $d>>2R$ (the hole is located above and below the disk), for which angular momentum $(l)$ transitions are predicted with increasing magnetic field. We also considered a system of two vertically coupled dots where now an extra parameter is introduced, namely the interdot distance $d_{z}$. For each $l_{h}$ and for a sufficient large magnetic field, the ground state becomes spontaneous symmetry broken in which the electron and the hole move towards one of the dots. This transition is induced by the Coulomb interaction and leads to a magnetic field induced dipole moment. No such symmetry broken ground states are found for a single dot (and for three vertically coupled symmetric quantum disks). For a system of two vertically coupled truncated cones, which is asymmetric from the start, we still find angular momentum transitions. For a symmetric system of three vertically coupled quantum disks, the system resembles for small $d_{z}$ the pillar-like regime of a single dot, where the hole tends to stay at the radial boundary, which induces angular momentum transitions with increasing magnetic field. For larger $d_{z}$ the hole can sit between the disks and the $l_{h}=0$ state remains the groundstate for the whole $B$-region.Comment: 11 pages, 16 figure

About curvature, conformal metrics and warped products

We consider the curvature of a family of warped products of two pseduo-Riemannian manifolds $(B,g_B)$ and $(F,g_F)$ furnished with metrics of the form $c^{2}g_B \oplus w^2 g_F$ and, in particular, of the type $w^{2 \mu}g_B \oplus w^2 g_F$, where $c, w \colon B \to (0,\infty)$ are smooth functions and $\mu$ is a real parameter. We obtain suitable expressions for the Ricci tensor and scalar curvature of such products that allow us to establish results about the existence of Einstein or constant scalar curvature structures in these categories. If $(B,g_B)$ is Riemannian, the latter question involves nonlinear elliptic partial differential equations with concave-convex nonlinearities and singular partial differential equations of the Lichnerowicz-York type among others.Comment: 32 pages, 3 figure

Polynomial diffeomorphisms of C^2, IV: The measure of maximal entropy and laminar currents

This paper concerns the dynamics of polynomial automorphisms of ${\bf C}^2$. One can associate to such an automorphism two currents $\mu^\pm$ and the equilibrium measure $\mu=\mu^+\wedge\mu^-$. In this paper we study some geometric and dynamical properties of these objects. First, we characterize $\mu$ as the unique measure of maximal entropy. Then we show that the measure $\mu$ has a local product structure and that the currents $\mu^\pm$ have a laminar structure. This allows us to deduce information about periodic points and heteroclinic intersections. For example, we prove that the support of $\mu$ coincides with the closure of the set of saddle points. The methods used combine the pluripotential theory with the theory of non-uniformly hyperbolic dynamical systems