The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.Comment: Best Student Paper Award, Proceedings of the 38th SGAI International Conference on Artificial Intelligence, Cambridge, UK, 2018, Artificial Intelligence XXXV, 201

First-principle Wannier functions and effective lattice fermion models for narrow-band compounds

We propose a systematic procedure for constructing effective lattice fermion models for narrow-band compounds on the basis of first-principles electronic structure calculations. The method is illustrated for the series of transition-metal (TM) oxides: SrVO$_3$, YTiO$_3$, V$_2$O$_3$, and Y$_2$Mo$_2$O$_7$. It consists of three parts, starting from LDA. (i) construction of the kinetic energy Hamiltonian using downfolding method. (ii) solution of an inverse problem and construction of the Wannier functions (WFs) for the given kinetic energy Hamiltonian. (iii) calculation of screened Coulomb interactions in the basis of \textit{auxiliary} WFs, for which the kinetic-energy term is set to be zero. The last step is necessary in order to avoid the double counting of the kinetic-energy term, which is included explicitly into the model. The screened Coulomb interactions are calculated in a hybrid scheme. First, we evaluate the screening caused by the change of occupation numbers and the relaxation of the LMTO basis functions, using the conventional constraint-LDA approach, where all matrix elements of hybridization involving the TM $d$ orbitals are set to be zero. Then, we switch on the hybridization and evaluate the screening associated with the change of this hybridization in RPA. The second channel of screening is very important, and results in a relatively small value of the effective Coulomb interaction for isolated $t_{2g}$ bands. We discuss details of this screening and consider its band-filling dependence, frequency dependence, influence of the lattice distortion, proximity of other bands, and the dimensionality of the model Hamiltonian.Comment: 35 pages, 25 figure

The Structure of Barium in the hcp Phase Under High Pressure

Recent experimental results on two hcp phases of barium under high pressure show interesting variation of the lattice parameters. They are here interpreted in terms of electronic structure calculation by using the LMTO method and generalized pseudopotential theory (GPT) with a NFE-TBB approach. In phase II the dramatic drop in c/a is an instability analogous to that in the group II metals but with the transfer of s to d electrons playing a crucial role in Ba. Meanwhile in phase V, the instability decrease a lot due to the core repulsion at very high pressure. PACS numbers: 62.50+p, 61.66Bi, 71.15.Ap, 71.15Hx, 71.15LaComment: 29 pages, 8 figure

Electronic Structure of New LiFeAs High-Tc Superconductor

We present results of it ab initio LDA calculations of electronic structure of "next generation" layered ironpnictide High-Tc superconductor LiFeAs (Tc=18K). Obtained electronic structure of LiFeAs is very similar to recently studied ReOFeAs (Re=La,Ce,Pr,Nd,Sm) and AFe2As2 (A=Ba,Sr) compounds. Namely close to the Fermi level its electronic properties are also determined ma inly by Fe 3d-orbitals of FeAs4 two-dimensional layers. Band dispersions of LiFeAs are very similar to the LaOFeAs and BaFe2As2 systems as well as the shape of the Fe-3d density o f states and Fermi surface.Comment: 4 pages, 5 figures; Electronic structure improved with respect to new experimental crystal structure dat

Approximation of corner polyhedra with families of intersection cuts

We study the problem of approximating the corner polyhedron using intersection cuts derived from families of lattice-free sets in $\mathbb{R}^n$. In particular, we look at the problem of characterizing families that approximate the corner polyhedron up to a constant factor, which depends only on $n$ and not the data or dimension of the corner polyhedron. The literature already contains several results in this direction. In this paper, we use the maximum number of facets of lattice-free sets in a family as a measure of its complexity and precisely characterize the level of complexity of a family required for constant factor approximations. As one of the main results, we show that, for each natural number $n$, a corner polyhedron with $n$ basic integer variables and an arbitrary number of continuous non-basic variables is approximated up to a constant factor by intersection cuts from lattice-free sets with at most $i$ facets if $i> 2^{n-1}$ and that no such approximation is possible if $i \leq 2^{n-1}$. When the approximation factor is allowed to depend on the denominator of the fractional vertex of the linear relaxation of the corner polyhedron, we show that the threshold is $i > n$ versus $i \leq n$. The tools introduced for proving such results are of independent interest for studying intersection cuts

Comparative study of correlation effects in CaVO3 and SrVO3

We present parameter-free LDA+DMFT (local density approximation + dynamical mean field theory) results for the many-body spectra of cubic SrVO3 and orthorhombic CaVO3. Both systems are found to be strongly correlated metals, but not on the verge of a metal-insulator transition. In spite of the considerably smaller V-O-V bond angle in CaVO3 the LDA+DMFT spectra of the two systems for energies E<E_F are very similar, their quasiparticle parts being almost identical. The calculated spectrum for E>E_F shows more pronounced, albeit still small, differences. This is in contrast to earlier theoretical and experimental conclusions, but in good agreement with recent bulk-sensitive photoemission and x-ray absorption experiments.Comment: 15 pages, 6 figure

Adequacy of Approximations in GW Theory

We use an all-electron implementation of the GW approximation to analyze several possible sources of error in the theory and its implementation. Among these are convergence in the polarization and Green's functions, the dependence of QP levels on choice of basis sets, and differing approximations for dealing with core levels. In all GW calculations presented here, G and W are generated from the local-density approximation (LDA), which we denote as the \GLDA\WLDA approximation. To test its range of validity, the \GLDA\WLDA approximation is applied to a variety of materials systems. We show that for simple sp semiconductors, \GLDA\WLDA always underestimates bandgaps; however, better agreement with experiment is obtained when the self-energy is not renormalized, and we propose a justification for it. Some calculations for Si are compared to pseudopotential-based \GLDA\WLDA calculations, and some aspects of the suitability of pseudopotentials for GW calculations are discussed.Comment: 38 pages,6 figures. Minor Revision

Momentum-resolved spectral functions of SrVO3_3 calculated by LDA+DMFT

LDA+DMFT, the merger of density functional theory in the local density approximation and dynamical mean-field theory, has been mostly employed to calculate k-integrated spectra accessible by photoemission spectroscopy. In this paper, we calculate k-resolved spectral functions by LDA+DMFT. To this end, we employ the Nth order muffin-tin (NMTO) downfolding to set up an effective low-energy Hamiltonian with three t_2g orbitals. This downfolded Hamiltonian is solved by DMFT yielding k-dependent spectra. Our results show renormalized quasiparticle bands over a broad energy range from -0.7 eV to +0.9 eV with small ``kinks'', discernible in the dispersion below the Fermi energy.Comment: 21 pages, 8 figure

Bidirectional PageRank Estimation: From Average-Case to Worst-Case

We present a new algorithm for estimating the Personalized PageRank (PPR) between a source and target node on undirected graphs, with sublinear running-time guarantees over the worst-case choice of source and target nodes. Our work builds on a recent line of work on bidirectional estimators for PPR, which obtained sublinear running-time guarantees but in an average-case sense, for a uniformly random choice of target node. Crucially, we show how the reversibility of random walks on undirected networks can be exploited to convert average-case to worst-case guarantees. While past bidirectional methods combine forward random walks with reverse local pushes, our algorithm combines forward local pushes with reverse random walks. We also discuss how to modify our methods to estimate random-walk probabilities for any length distribution, thereby obtaining fast algorithms for estimating general graph diffusions, including the heat kernel, on undirected networks.Comment: Workshop on Algorithms and Models for the Web-Graph (WAW) 201