Stochastic Reinforcement Learning

In reinforcement learning episodes, the rewards and punishments are often non-deterministic, and there are invariably stochastic elements governing the underlying situation. Such stochastic elements are often numerous and cannot be known in advance, and they have a tendency to obscure the underlying rewards and punishments patterns. Indeed, if stochastic elements were absent, the same outcome would occur every time and the learning problems involved could be greatly simplified. In addition, in most practical situations, the cost of an observation to receive either a reward or punishment can be significant, and one would wish to arrive at the correct learning conclusion by incurring minimum cost. In this paper, we present a stochastic approach to reinforcement learning which explicitly models the variability present in the learning environment and the cost of observation. Criteria and rules for learning success are quantitatively analyzed, and probabilities of exceeding the observation cost bounds are also obtained.Comment: AIKE 201

Variational data assimilation for the initial-value dynamo problem

The secular variation of the geomagnetic field as observed at the Earth's surface results from the complex magnetohydrodynamics taking place in the fluid core of the Earth. One way to analyze this system is to use the data in concert with an underlying dynamical model of the system through the technique of variational data assimilation, in much the same way as is employed in meteorology and oceanography. The aim is to discover an optimal initial condition that leads to a trajectory of the system in agreement with observations. Taking the Earth's core to be an electrically conducting fluid sphere in which convection takes place, we develop the continuous adjoint forms of the magnetohydrodynamic equations that govern the dynamical system together with the corresponding numerical algorithms appropriate for a fully spectral method. These adjoint equations enable a computationally fast iterative improvement of the initial condition that determines the system evolution. The initial condition depends on the three dimensional form of quantities such as the magnetic field in the entire sphere. For the magnetic field, conservation of the divergence-free condition for the adjoint magnetic field requires the introduction of an adjoint pressure term satisfying a zero boundary condition. We thus find that solving the forward and adjoint dynamo system requires different numerical algorithms. In this paper, an efficient algorithm for numerically solving this problem is developed and tested for two illustrative problems in a whole sphere: one is a kinematic problem with prescribed velocity field, and the second is associated with the Hall-effect dynamo, exhibiting considerable nonlinearity. The algorithm exhibits reliable numerical accuracy and stability. Using both the analytical and the numerical techniques of this paper, the adjoint dynamo system can be solved directly with the same order of computational complexity as that required to solve the forward problem. These numerical techniques form a foundation for ultimate application to observations of the geomagnetic field over the time scale of centuries

Search for Bc(ns)B_c(ns) via the Bc(ns)→Bc(ms)π+π−B_c(ns)\to B_c(ms)\pi^+\pi^- transition at LHCb and Z0Z_0 factory

It is interesting to study the characteristics of the whole family of $B_c$ which contains two different heavy flavors. LHC and the proposed $Z^0$ factory provide an opportunity because a large database on the $B_c$ family will be achieved. $B_c$ and its excited states can be identified via their decay modes. As suggested by experimentalists, $B_c^*(ns)\to B_c+\gamma$ is not easy to be clearly measured, instead, the trajectories of $\pi^+$ and $\pi^-$ occurring in the decay of $B_c(ns)\to B_c(ms)+\pi^+\pi^-$ ($n>m$) can be unambiguously identified, thus the measurement seems easier and more reliable, therefore this mode is more favorable at early running stage of LHCb and the proposed $Z^0$ factory. In this work, we calculate the rate of $B_c(ns)\to B_c(ms)+\pi^+\pi^-$ in terms of the QCD multipole-expansion and the numerical results indicate that the experimental measurements with the luminosity of LHC and $Z^0$ factory are feasible.Comment: 12 pages, 1 figures and 4 tables, acceptted by SCIENCE CHINA Physics, Mechanics & Astronomy (Science in China Series G

Prospects for detection of Υ(1D)→Υ(1S)ππ\Upsilon(1D) \to \Upsilon(1S) \pi \pi via Υ(3S)→Υ(1D)+X\Upsilon(3S) \to \Upsilon(1D) + X

At least one state in the first family of D-wave $b \bar b$ quarkonium levels has been discovered near the predicted mass of 10.16 GeV/$c^2$. This state is probably the one with J=2. This state and the ones with J=1 and J=3 may contribute a detectable amount to the decay $\Upsilon(1D) \to \Upsilon(1S) \pi \pi$, depending on the partial widths for these decays for which predictions vary considerably. The prospects for detection of the chain $\Upsilon(3S) \to \Upsilon(1D) + X \to \Upsilon \pi \pi + X$ are discussed.Comment: 4 pages, LaTeX, 1 figure, to be published in Phys. Rev. D, comment added after Eq. (2

The magnetic dipole transitions in the (cbˉ)(c\bar{b}) binding system

The magnetic dipole transitions between the vector mesons $B_c^*$ and their relevant pseudoscalar mesons $B_c$ ($B_c$, $B_c^*$, $B_c(2S)$, $B_c^*(2S)$, $B_c(3S)$ and $B_c^*(3S)$ etc, the binding states of $(c\bar{b})$ system) of the $B_c$ family are interesting. To see the `hyperfine' splitting due to spin-spin interaction is an important topic for understanding the spin-spin interaction and the spectrum of the the $(c\bar{b})$ binding system. The knowledge about the magnetic dipole transitions is also very useful for identifying the vector boson $B_c^*$ mesons experimentally, whose masses are just slightly above the masses of their relevant pseudoscalar mesons $B_c$ accordingly. Considering the possibility to observe the vector mesons via the transitions at $Z^0$ factory and the potentially usages of the theoretical estimate on the transitions, we fucus our efforts on calculating the magnetic dipole transitions, i.e. precisely to calculate the rates for the transitions such as decays $B_c^*\to B_c\gamma$ and $B_c^*\to B_c e^+e^-$, and particularly work in the Behte-Salpeter framework. In the estimate, as a typical example, we carefully investigate the dependance of the rate $\Gamma(B_c^*\to B_c\gamma)$ on the mass difference $\Delta M=M_{B_c^*}-M_{B_c}$ as well.Comment: 10 pages, 2 figures, 1 tabl

Quantum integrability and Bethe ansatz solution for interacting matter-radiation systems

A unified integrable system, generating a new series of interacting matter-radiation models with interatomic coupling and different atomic frequencies, is constructed and exactly solved through algebraic Bethe ansatz. Novel features in Rabi oscillation and vacuum Rabi splitting are shown on the example of an integrable two-atom Buck-Sukumar model with resolution of some important controversies in the Bethe ansatz solution including its possible degeneracy for such models.Comment: Latex, 7 pages, 1 figure. Final version to be published in J Phys A (as Letter

Counter Chemotactic Flow in Quasi-One-Dimensional Path

Quasi-one-dimensional bidirectional particle flow including the effect of chemotaxis is investigated through a modification of the John-Schadschneider-Chowdhury-Nishinari model. Specifically, we permit multiple lanes to be shared by both directionally traveling particles. The relation between particle density and flux is studied for several evaporation rates of pheromone, and the following results are obtained: i) in the low-particle-density range, the flux is enlarged by pheromone if the pheromone evaporation rate is sufficiently low, ii) in the high particle-density range, the flux is largest at a reasonably high evaporation rate and, iii) if the evaporation rate is at the level intermediate between the above two cases, the flux is kept small in the entire range of particle densities. The mechanism of these behaviors is investigated by observing the spatial-temporal evolution of particles and the average cluster size in the system.Comment: 4 pages, 9 figure

Cancellation of Infrared Divergences in Hadronic Annihilation Decays of Heavy Quarkonia

In the framework of a newly developed factorization formalism which is based on NRQCD, explicit cancellations are shown for the infrared divergences that appeared in the previously calculated hadronic annihilation decay rates of P-wave and D-wave heavy quarkonia. We extend them to a more general case that to leading order in $v^2$ and next-to-leading order in $\alpha_s$, the infrared divergences in the annihilation amplitudes of color-singlet $Q\bar{Q}(^{2S+1}L_J)$ pair can be removed by including the contributions of color-octet operators $Q\bar{Q}(^{2S+1}(L-1)_{J'})$, $Q\bar{Q}(^{2S+1}(L-3)_{J''})$, ... in NRQCD. We also give the decay widths of $^3D_J\rightarrow LH$ at leading order in $\alpha_s$.Comment: 8 pages, LaTex(3 figures included), to be publishe

A Crucial Test for Color-Octet Production Mechanism in Z^0 Decays

The direct production rates of $D$-wave charmonia in the decays of $Z^0$ is evaluated. The color-octet production processes $Z^0\rightarrow ^3D_J(c\bar c) q\bar q$ are shown to have distinctively large branching ratios, the same order of magnitude as that of $J/\psi$ prodution, as compared with other $D$-wave charmonium production mechanisms. This may suggest a crucial channel to test the color-octet mechanism as well as to observe the $D$-wave charmonium states in $Z^0$ decays. In addition, a signal for the $^3D_J$ charmonium as strong as $J/\psi$ or $\psi^\prime$ with large transverse momentum at the Tevatron should also be observed.Comment: 14 pages in LaTex (3 figures in PS-file