Model-Based Reinforcement Learning with Isolated Imaginations

World models learn the consequences of actions in vision-based interactive systems. However, in practical scenarios like autonomous driving, noncontrollable dynamics that are independent or sparsely dependent on action signals often exist, making it challenging to learn effective world models. To address this issue, we propose Iso-Dream++, a model-based reinforcement learning approach that has two main contributions. First, we optimize the inverse dynamics to encourage the world model to isolate controllable state transitions from the mixed spatiotemporal variations of the environment. Second, we perform policy optimization based on the decoupled latent imaginations, where we roll out noncontrollable states into the future and adaptively associate them with the current controllable state. This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild, such as self-driving cars that can anticipate the movement of other vehicles, thereby avoiding potential risks. On top of our previous work, we further consider the sparse dependencies between controllable and noncontrollable states, address the training collapse problem of state decoupling, and validate our approach in transfer learning setups. Our empirical study demonstrates that Iso-Dream++ outperforms existing reinforcement learning models significantly on CARLA and DeepMind Control.Comment: arXiv admin note: substantial text overlap with arXiv:2205.1381

Verification of Onsager reciprocal relation between anomalous transverse coefficients of an anisotropic antiferromagnet

Whenever two irreversible processes occur simultaneously, time-reversal symmetry of microscopic dynamics gives rise, on a macroscopic level, to Onsager's reciprocal relations, which impose constraints on the number of independent components of any transport coefficient tensor. Here, we show that in the antiferromagnetic YbMnBi$_2$, which displays a strong temperature-dependent anisotropy, the Onsager's reciprocal relations are strictly satisfied for anomalous electric, $\sigma^A_{ij}$, and anomalous thermoelectric, $\alpha^A_{ij}$, conductivity tensors. In contradiction with what was recently reported by Pan [1], we find that $\sigma^A_{ij} (H)= \sigma^A_{ji} (-H)$, and $\alpha^A_{ij} (H)= \alpha^A_{ji} (-H)$. This equality holds in the whole temperature window irrespective of the relative weights of the intrinsic or extrinsic mechanisms. The $\alpha^A_{ij}/\sigma^A_{ij}$ ratio is close $k_B/e$ at room temperature, but peaks to an unprecedented magnitude of 2.9 $k_B/e$ at $\sim$ 150 K, which may involve non-degenerate carriers of small Fermi surface pockets.Comment: 6 pages, 4 figures,supplemental materials include

Phonon thermal Hall effect in strontium titanate

It has been known for more than a decade that phonons can produce an off-diagonal thermal conductivity in presence of magnetic field. Recent studies of thermal Hall conductivity, $\kappa_{xy}$, in a variety of contexts, however, have assumed a negligibly small phonon contribution. We present a study of $\kappa_{xy}$ in quantum paraelectric SrTiO$_3$, which is a non-magnetic insulator and find that its peak value exceeds what has been reported in any other insulator, including those in which the signal has been qualified as 'giant'. Remarkably, $\kappa_{xy}(T)$ and $\kappa(T)$ peak at the same temperature and the former decreases faster than the latter at both sides of the peak. Interestingly, in the case of La$_2$CuO$_4$ and $\alpha$-RuCl$_3$, $\kappa_{xy}(T)$ and $\kappa(T)$ peak also at the same temperature. We also studied KTaO$_3$ and found a small signal, indicating that a sizable $\kappa_{xy}(T)$ is not a generic feature of quantum paraelectrics. Combined to other observations, this points to a crucial role played by antiferrodistortive domains in generating $\kappa_{xy}$ of this solid.Comment: Main text: 6 pages, 4 figures, Supplemental Material is included. Accepted by Phys. Rev. Let

Field-linear anomalous Hall effect and Berry curvature induced by spin chirality in the kagome antiferromagnet Mn3Sn

During the past two decades, it has been established that a non-trivial electron wave-function topology generates an anomalous Hall effect (AHE), which shows itself as a Hall conductivity non-linear in magnetic field. Here, we report on an unprecedented case of field-linear AHE. In Mn$_3$Sn, a kagome magnet, the out-of-plane Hall response, which shows an abrupt jump, was discovered to be a case of AHE. We find now that the in-plane Hall response, which is perfectly linear in magnetic field, is set by the Berry curvature of the wavefunction. The amplitude of the Hall response and its concomitant Nernst signal exceed by far what is expected in the semiclassical picture. We argue that magnetic field induces out-of-plane spin canting and thereafter gives rise to nontrivial spin chirality on the kagome lattice. In band structure, we find that the spin chirality modifies the topology by gapping out Weyl nodal lines unknown before, accounting for the AHE observed. Our work reveals intriguing unification of real-space Berry phase from spin chirality and momentum-space Berry curvature.Comment: 4 figures and 6 pages. The supplementary materials were attached to the en

Finite-temperature violation of the anomalous transverse Wiedemann-Franz law

The Wiedemann-Franz (WF) law links the ratio of electronic charge and heat conductivity to fundamental constants. It has been tested in numerous solids, but the extent of its relevance to the anomalous transverse transport, which represents the topological nature of the wave function, remains an open question. Here we present a study of anomalous transverse response in the noncollinear antiferromagnet Mn$_{3}$Ge extended from room temperature down to sub-Kelvin temperature and find that the anomalous Lorenz ratio remains close to the Sommerfeld value up to 100 K, but not above. The finite-temperature violation of the WF correlation is caused by a mismatch between the thermal and electrical summations of the Berry curvature, rather than the inelastic scattering as observed in ordinary metals. This interpretation is backed by our theoretical calculations, which reveals a competition between the temperature and the Berry curvature distribution. The accuracy of the experiment is supported by the verification of the Bridgman relation between the anomalous Ettingshausen and Nernst effects. Our results identify the anomalous Lorenz ratio as an extremely sensitive probe of Berry spectrum near the chemical potential.Comment: 9 pages,6 figures, Supplemental Material include