Quantum-enhanced reinforcement learning for finite-episode games with discrete state spaces

Quantum annealing algorithms belong to the class of metaheuristic tools, applicable for solving binary optimization problems. Hardware implementations of quantum annealing, such as the quantum annealing machines produced by D-Wave Systems, have been subject to multiple analyses in research, with the aim of characterizing the technology's usefulness for optimization and sampling tasks. Here, we present a way to partially embed both Monte Carlo policy iteration for finding an optimal policy on random observations, as well as how to embed (n) sub-optimal state-value functions for approximating an improved state-value function given a policy for finite horizon games with discrete state spaces on a D-Wave 2000Q quantum processing unit (QPU). We explain how both problems can be expressed as a quadratic unconstrained binary optimization (QUBO) problem, and show that quantum-enhanced Monte Carlo policy evaluation allows for finding equivalent or better state-value functions for a given policy with the same number episodes compared to a purely classical Monte Carlo algorithm. Additionally, we describe a quantum-classical policy learning algorithm. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of the QPU, and not to prove supremacy over every existing classical policy evaluation algorithm.Comment: 17 pages, 7 figure

Cooperative medium access control based on spectrum leasing

Based on cooperative spectrum leasing, a distributed “win–win” (WW) cooperative framework is designed to encourage the licensed source node (SN) to lease some part of its spectral resources to the unlicensed relay node (RN) for the sake of simultaneously improving the SN’s achievable rate and for reducing the energy consumption (EC). The potential candidate RNs carry out autonomous decisions concerning whether to contend for a cooperative transmission opportunity, which could dissipate some of their battery power, while conveying their traffic in light of their individual service requirements. Furthermore, a WW cooperative medium-access-control (MAC) protocol is designed to implement the proposed distributed WW cooperative framework. Simulation results demonstrate that our WW cooperative MAC protocol is capable of providing both substantial rate improvements and considerable energy savings for the cooperative spectrum leasing system

Multi-Player and Multi-Choice Quantum Game

We investigate a multi-player and multi-choice quantum game. We start from two-player and two-choice game and the result is better than its classical version. Then we extend it to N-player and N-choice cases. In the quantum domain, we provide a strategy with which players can always avoid the worst outcome. Also, by changing the value of the parameter of the initial state, the probabilities for players to obtain the best payoff will be much higher that in its classical version.Comment: 4 pages, 1 figur

On Factors Affecting the Usage and Adoption of a Nation-wide TV Streaming Service

Using nine months of access logs comprising 1.9 Billion sessions to BBC iPlayer, we survey the UK ISP ecosystem to understand the factors affecting adoption and usage of a high bandwidth TV streaming application across different providers. We find evidence that connection speeds are important and that external events can have a huge impact for live TV usage. Then, through a temporal analysis of the access logs, we demonstrate that data usage caps imposed by mobile ISPs significantly affect usage patterns, and look for solutions. We show that product bundle discounts with a related fixed-line ISP, a strategy already employed by some mobile providers, can better support user needs and capture a bigger share of accesses. We observe that users regularly split their sessions between mobile and fixed-line connections, suggesting a straightforward strategy for offloading by speculatively pre-fetching content from a fixed-line ISP before access on mobile devices.Comment: In Proceedings of IEEE INFOCOM 201