Second-order cosmological perturbations. III. Produced by scalar-scalar coupling during radiation-dominated stage

We study the 2nd-order scalar, vector and tensor metric perturbations in Robertson-Walker (RW) spacetime in synchronous coordinates during the radiation dominated (RD) stage. The dominant radiation is modeled by a relativistic fluid described by a stress tensor $T_{\mu\nu}=(\rho+p)U_\mu U_\nu+g_{\mu\nu}p$ with $p= c^2_s \rho$, and the 1st-order velocity is assumed to be curlless. We analyze the solutions of 1st-order perturbations, upon which the solutions of 2nd-order perturbation are based. We show that the 1st-order tensor modes propagate at the speed of light and are truly radiative, but the scalar and vector modes do not. The 2nd-order perturbed Einstein equation contains various couplings of 1st-order metric perturbations, and the scalar-scalar coupling is considered in this paper. We decompose the 2nd-order Einstein equation into the evolution equations of 2nd-order scalar, vector, and tensor perturbations, and the energy and momentum constraints. The coupling terms and the stress tensor of the fluid together serve as the effective source for the 2nd-order metric perturbations. The equation of covariant conservation of stress tensor is also needed to determine $\rho$ and $U^\mu$. By solving this set of equations up to 2nd order analytically, we obtain the 2nd-order integral solutions of all the metric perturbations, density contrast and velocity. To use these solutions in applications, one needs to carry out seven types of the numerical integrals. We perform the residual gauge transformations between synchronous coordinates up to 2nd order, and identify the gauge-invariant modes of 2nd-order solutions.Comment: 75 pages, 1 figure. We make the discussion below Eq.(6.8) clearer in this updated versio

On Bismut Flat Manifolds

In this paper, we give a classification of all compact Hermitian manifolds with flat Bismut connection. We show that the torsion tensor of such a manifold must be parallel, thus the universal cover of such a manifold is a Lie group equipped with a bi-invariant metric and a compatible left invariant complex structure. In particular, isosceles Hopf surfaces are the only Bismut flat compact non-K\"ahler surfaces, while central Calabi-Eckmann threefolds are the only simply-connected compact Bismut flat threefolds.Comment: In this 3rd version, we add a lemma on Hermitian surfaces with flat Riemannian connection. References are updated and typos correcte

Reinforcement Learning with Perturbed Rewards

Recent studies have shown that reinforcement learning (RL) models are vulnerable in various noisy scenarios. For instance, the observed reward channel is often subject to noise in practice (e.g., when rewards are collected through sensors), and is therefore not credible. In addition, for applications such as robotics, a deep reinforcement learning (DRL) algorithm can be manipulated to produce arbitrary errors by receiving corrupted rewards. In this paper, we consider noisy RL problems with perturbed rewards, which can be approximated with a confusion matrix. We develop a robust RL framework that enables agents to learn in noisy environments where only perturbed rewards are observed. Our solution framework builds on existing RL/DRL algorithms and firstly addresses the biased noisy reward setting without any assumptions on the true distribution (e.g., zero-mean Gaussian noise as made in previous works). The core ideas of our solution include estimating a reward confusion matrix and defining a set of unbiased surrogate rewards. We prove the convergence and sample complexity of our approach. Extensive experiments on different DRL platforms show that trained policies based on our estimated surrogate reward can achieve higher expected rewards, and converge faster than existing baselines. For instance, the state-of-the-art PPO algorithm is able to obtain 84.6% and 80.8% improvements on average score for five Atari games, with error rates as 10% and 30% respectively.Comment: AAAI 2020 (Spotlight