Quantum-enhanced reinforcement learning for finite-episode games with discrete state spaces

Quantum annealing algorithms belong to the class of metaheuristic tools, applicable for solving binary optimization problems. Hardware implementations of quantum annealing, such as the quantum annealing machines produced by D-Wave Systems, have been subject to multiple analyses in research, with the aim of characterizing the technology's usefulness for optimization and sampling tasks. Here, we present a way to partially embed both Monte Carlo policy iteration for finding an optimal policy on random observations, as well as how to embed (n) sub-optimal state-value functions for approximating an improved state-value function given a policy for finite horizon games with discrete state spaces on a D-Wave 2000Q quantum processing unit (QPU). We explain how both problems can be expressed as a quadratic unconstrained binary optimization (QUBO) problem, and show that quantum-enhanced Monte Carlo policy evaluation allows for finding equivalent or better state-value functions for a given policy with the same number episodes compared to a purely classical Monte Carlo algorithm. Additionally, we describe a quantum-classical policy learning algorithm. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of the QPU, and not to prove supremacy over every existing classical policy evaluation algorithm.Comment: 17 pages, 7 figure

Optimization and evaluation of variability in the programming window of a flash cell with molecular metal-oxide storage

We report a modeling study of a conceptual nonvolatile memory cell based on inorganic molecular metal-oxide clusters as a storage media embedded in the gate dielectric of a MOSFET. For the purpose of this paper, we developed a multiscale simulation framework that enables the evaluation of variability in the programming window of a flash cell with sub-20-nm gate length. Furthermore, we studied the threshold voltage variability due to random dopant fluctuations and fluctuations in the distribution of the molecular clusters in the cell. The simulation framework and the general conclusions of our work are transferrable to flash cells based on alternative molecules used for a storage media

Multi-level, Forming Free, Bulk Switching Trilayer RRAM for Neuromorphic Computing at the Edge

Resistive memory-based reconfigurable systems constructed by CMOS-RRAM integration hold great promise for low energy and high throughput neuromorphic computing. However, most RRAM technologies relying on filamentary switching suffer from variations and noise leading to computational accuracy loss, increased energy consumption, and overhead by expensive program and verify schemes. Low ON-state resistance of filamentary RRAM devices further increases the energy consumption due to high-current read and write operations, and limits the array size and parallel multiply & accumulate operations. High-forming voltages needed for filamentary RRAM are not compatible with advanced CMOS technology nodes. To address all these challenges, we developed a forming-free and bulk switching RRAM technology based on a trilayer metal-oxide stack. We systematically engineered a trilayer metal-oxide RRAM stack and investigated the switching characteristics of RRAM devices with varying thicknesses and oxygen vacancy distributions across the trilayer to achieve reliable bulk switching without any filament formation. We demonstrated bulk switching operation at megaohm regime with high current nonlinearity and programmed up to 100 levels without compliance current. We developed a neuromorphic compute-in-memory platform based on trilayer bulk RRAM crossbars by combining energy-efficient switched-capacitor voltage sensing circuits with differential encoding of weights to experimentally demonstrate high-accuracy matrix-vector multiplication. We showcased the computational capability of bulk RRAM crossbars by implementing a spiking neural network model for an autonomous navigation/racing task. Our work addresses challenges posed by existing RRAM technologies and paves the way for neuromorphic computing at the edge under strict size, weight, and power constraints

An overview of decision table literature 1982-1995.

This report gives an overview of the literature on decision tables over the past 15 years. As much as possible, for each reference, an author supplied abstract, a number of keywords and a classification are provided. In some cases own comments are added. The purpose of these comments is to show where, how and why decision tables are used. The literature is classified according to application area, theoretical versus practical character, year of publication, country or origin (not necessarily country of publication) and the language of the document. After a description of the scope of the interview, classification results and the classification by topic are presented. The main body of the paper is the ordered list of publications with abstract, classification and comments.

One-dimensional many-body entangled open quantum systems with tensor network methods

We present a collection of methods to simulate entangled dynamics of open quantum systems governed by the Lindblad equation with tensor network methods. Tensor network methods using matrix product states have been proven very useful to simulate many-body quantum systems and have driven many innovations in research. Since the matrix product state design is tailored for closed one-dimensional systems governed by the Schr\"odinger equation, the next step for many-body quantum dynamics is the simulation of open quantum systems. We review the three dominant approaches to the simulation of open quantum systems via the Lindblad master equation: quantum trajectories, matrix product density operators, and locally purified tensor networks. Selected examples guide possible applications of the methods and serve moreover as a benchmark between the techniques. These examples include the finite temperature states of the transverse quantum Ising model, the dynamics of an exciton traveling under the influence of spontaneous emission and dephasing, and a double-well potential simulated with the Bose-Hubbard model including dephasing. We analyze which approach is favorable leading to the conclusion that a complete set of all three methods is most beneficial, push- ing the limits of different scenarios. The convergence studies using analytical results for macroscopic variables and exact diagonalization methods as comparison, show, for example, that matrix product density operators are favorable for the exciton problem in our study. All three methods access the same library, i.e., the software package Open Source Matrix Product States, allowing us to have a meaningful comparison between the approaches based on the selected examples. For example, tensor operations are accessed from the same subroutines and with the same optimization eliminating one possible bias in a comparison of such numerical methods.Comment: 24 pages, 8 figures. Small extension of time evolution section and moving quantum simulators to introduction in comparison to v

Stochastics global optimization methods and their applications in Chemical Engineering

Ph.DDOCTOR OF PHILOSOPH