An Analysis of the Value of Information when Exploring Stochastic, Discrete Multi-Armed Bandits

In this paper, we propose an information-theoretic exploration strategy for stochastic, discrete multi-armed bandits that achieves optimal regret. Our strategy is based on the value of information criterion. This criterion measures the trade-off between policy information and obtainable rewards. High amounts of policy information are associated with exploration-dominant searches of the space and yield high rewards. Low amounts of policy information favor the exploitation of existing knowledge. Information, in this criterion, is quantified by a parameter that can be varied during search. We demonstrate that a simulated-annealing-like update of this parameter, with a sufficiently fast cooling schedule, leads to an optimal regret that is logarithmic with respect to the number of episodes.Comment: Entrop

Beyond the Spectral Theorem: Spectrally Decomposing Arbitrary Functions of Nondiagonalizable Operators

Nonlinearities in finite dimensions can be linearized by projecting them into infinite dimensions. Unfortunately, often the linear operator techniques that one would then use simply fail since the operators cannot be diagonalized. This curse is well known. It also occurs for finite-dimensional linear operators. We circumvent it by developing a meromorphic functional calculus that can decompose arbitrary functions of nondiagonalizable linear operators in terms of their eigenvalues and projection operators. It extends the spectral theorem of normal operators to a much wider class, including circumstances in which poles and zeros of the function coincide with the operator spectrum. By allowing the direct manipulation of individual eigenspaces of nonnormal and nondiagonalizable operators, the new theory avoids spurious divergences. As such, it yields novel insights and closed-form expressions across several areas of physics in which nondiagonalizable dynamics are relevant, including memoryful stochastic processes, open non unitary quantum systems, and far-from-equilibrium thermodynamics. The technical contributions include the first full treatment of arbitrary powers of an operator. In particular, we show that the Drazin inverse, previously only defined axiomatically, can be derived as the negative-one power of singular operators within the meromorphic functional calculus and we give a general method to construct it. We provide new formulae for constructing projection operators and delineate the relations between projection operators, eigenvectors, and generalized eigenvectors. By way of illustrating its application, we explore several, rather distinct examples.Comment: 29 pages, 4 figures, expanded historical citations; http://csc.ucdavis.edu/~cmg/compmech/pubs/bst.ht

Geostatistical simulation of two-dimensional fields of raindrop size distributions at the meso-¿ scale

The large variability of the raindrop size distribution (DSD) in space and time must be taken into account to improve remote sensing of precipitation. The ability to simulate a large number of 2-D fields of DSDs sharing the same statistical properties provides a very useful simulation framework that nicely complements experimental approaches based on DSD ground measurements. These simulations can be used to investigate radar beam propagation through rain and to evaluate different radar retrieval techniques. The proposed approach uses geostatistical methods to provide structural analysis and stochastic simulation of DSD fields. First, the DSD is assumed to follow a Gamma distribution with three parameters. As a consequence, 2-D fields of DSDs can be described as a multivariate random function. The parameters are normalized using a Gaussian anamorphosis and simulated by taking advantage of fast Gaussian simulation algorithms. Variograms are used to characterize the spatial structure of the DSD fields. The generated fields have identical spatial structure and are consistent with the observations. Because intermittency cannot be simulated using this technique, the size of the simulation domain is limited to the meso-¿ scale (2-20 km). To assess the proposed approach, the method is applied to data collected during intense Mediterranean rainfall. Taylor's hypothesis is invoked to convert time series into 1-D range profiles. The anisotropy of the fields is derived from radar measurements. Simulated and measured reflectivity fields are in good agreement with respect to the mean, the standard deviation, and the spatial structure, demonstrating the promising potential of the proposed stochastic model of DSD field

Exact simulation pricing with Gamma processes and their extensions

Exact path simulation of the underlying state variable is of great practical importance in simulating prices of financial derivatives or their sensitivities when there are no analytical solutions for their pricing formulas. However, in general, the complex dependence structure inherent in most nontrivial stochastic volatility (SV) models makes exact simulation difficult. In this paper, we present a nontrivial SV model that parallels the notable Heston SV model in the sense of admitting exact path simulation as studied by Broadie and Kaya. The instantaneous volatility process of the proposed model is driven by a Gamma process. Extensions to the model including superposition of independent instantaneous volatility processes are studied. Numerical results show that the proposed model outperforms the Heston model and two other L\'evy driven SV models in terms of model fit to the real option data. The ability to exactly simulate some of the path-dependent derivative prices is emphasized. Moreover, this is the first instance where an infinite-activity volatility process can be applied exactly in such pricing contexts.Comment: Forthcoming The Journal of Computational Financ