Quantum-enhanced reinforcement learning for finite-episode games with discrete state spaces

Quantum annealing algorithms belong to the class of metaheuristic tools, applicable for solving binary optimization problems. Hardware implementations of quantum annealing, such as the quantum annealing machines produced by D-Wave Systems, have been subject to multiple analyses in research, with the aim of characterizing the technology's usefulness for optimization and sampling tasks. Here, we present a way to partially embed both Monte Carlo policy iteration for finding an optimal policy on random observations, as well as how to embed (n) sub-optimal state-value functions for approximating an improved state-value function given a policy for finite horizon games with discrete state spaces on a D-Wave 2000Q quantum processing unit (QPU). We explain how both problems can be expressed as a quadratic unconstrained binary optimization (QUBO) problem, and show that quantum-enhanced Monte Carlo policy evaluation allows for finding equivalent or better state-value functions for a given policy with the same number episodes compared to a purely classical Monte Carlo algorithm. Additionally, we describe a quantum-classical policy learning algorithm. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of the QPU, and not to prove supremacy over every existing classical policy evaluation algorithm.Comment: 17 pages, 7 figure

Quantum-assisted finite-element design optimization

Quantum annealing devices such as the ones produced by D-Wave systems are typically used for solving optimization and sampling tasks, and in both academia and industry the characterization of their usefulness is subject to active research. Any problem that can naturally be described as a weighted, undirected graph may be a particularly interesting candidate, since such a problem may be formulated a as quadratic unconstrained binary optimization (QUBO) instance, which is solvable on D-Wave's Chimera graph architecture. In this paper, we introduce a quantum-assisted finite-element method for design optimization. We show that we can minimize a shape-specific quantity, in our case a ray approximation of sound pressure at a specific position around an object, by manipulating the shape of this object. Our algorithm belongs to the class of quantum-assisted algorithms, as the optimization task runs iteratively on a D-Wave 2000Q quantum processing unit (QPU), whereby the evaluation and interpretation of the results happens classically. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of a QPU, and not to prove supremacy over existing classical finite-element algorithms for design optimization.Comment: 17 pages, 5 figure

Quantum-assisted finite-element design optimization

Quantum annealing devices such as the ones produced by D-Wave systems are typically used for solving optimization and sampling tasks, and in both academia and industry the characterization of their usefulness is subject to active research. Any problem that can naturally be described as a weighted, undirected graph may be a particularly interesting candidate, since such a problem may be formulated a as quadratic unconstrained binary optimization (QUBO) instance, which is solvable on D-Wave's Chimera graph architecture. In this paper, we introduce a quantum-assisted finite-element method for design optimization. We show that we can minimize a shape-specific quantity, in our case a ray approximation of sound pressure at a specific position around an object, by manipulating the shape of this object. Our algorithm belongs to the class of quantum-assisted algorithms, as the optimization task runs iteratively on a D-Wave 2000Q quantum processing unit (QPU), whereby the evaluation and interpretation of the results happens classically. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of a QPU, and not to prove supremacy over existing classical finite-element algorithms for design optimization.Algorithms and the Foundations of Software technolog

Using copies can improve precision in continuous-time quantum computing

In the quantum optimisation setting, we build on a scheme introduced by Young et al (2013 Phys. Rev. A 88 062314), where physical qubits in multiple copies of a problem encoded into an Ising spin Hamiltonian are linked together to increase the logical system's robustness to error. We introduce several innovations that improve the error suppression of this scheme under a special model of control noise, designed to understand how limited precision could be overcome. First, we note that only one copy needs to be correct by the end of the computation, since solution quality can be checked efficiently. Second, we find that ferromagnetic links do not generally help in this 'one correct copy' setting, but anti-ferromagnetic links do help on average, by suppressing the chance of the same error being present on all of the copies. Third, we find that minimum-strength anti-ferromagnetic links perform best, by counteracting the spin-flips induced by the errors. We have numerically tested our innovations on small instances of spin glasses from Callison et al (2019 New J. Phys.21 123022), and we find improved error tolerance for three or more copies in configurations that include frustration. Interpreted as an effective precision increase, we obtain several extra bits of precision on average for three copies connected in a triangle. This provides proof-of-concept of a method for scaling quantum annealing beyond the precision limits of hardware, a step towards fault tolerance in this setting