Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks

This paper investigates the use of deep reinforcement learning (DRL) in a MAC protocol for heterogeneous wireless networking referred to as Deep-reinforcement Learning Multiple Access (DLMA). The thrust of this work is partially inspired by the vision of DARPA SC2, a 3-year competition whereby competitors are to come up with a clean-slate design that "best share spectrum with any network(s), in any environment, without prior knowledge, leveraging on machine-learning technique". Specifically, this paper considers the problem of sharing time slots among a multiple of time-slotted networks that adopt different MAC protocols. One of the MAC protocols is DLMA. The other two are TDMA and ALOHA. The nodes operating DLMA do not know that the other two MAC protocols are TDMA and ALOHA. Yet, by a series of observations of the environment, its own actions, and the resulting rewards, a DLMA node can learn an optimal MAC strategy to coexist harmoniously with the TDMA and ALOHA nodes according to a specified objective (e.g., the objective could be the sum throughput of all networks, or a general alpha-fairness objective)

Free Energy Approximations for CSMA networks

In this paper we study how to estimate the back-off rates in an idealized CSMA network consisting of $n$ links to achieve a given throughput vector using free energy approximations. More specifically, we introduce the class of region-based free energy approximations with clique belief and present a closed form expression for the back-off rates based on the zero gradient points of the free energy approximation (in terms of the conflict graph, target throughput vector and counting numbers). Next we introduce the size $k_{max}$ clique free energy approximation as a special case and derive an explicit expression for the counting numbers, as well as a recursion to compute the back-off rates. We subsequently show that the size $k_{max}$ clique approximation coincides with a Kikuchi free energy approximation and prove that it is exact on chordal conflict graphs when $k_{max} = n$. As a by-product these results provide us with an explicit expression of a fixed point of the inverse generalized belief propagation algorithm for CSMA networks. Using numerical experiments we compare the accuracy of the novel approximation method with existing methods

Throughput Analysis of CSMA Wireless Networks with Finite Offered-load

This paper proposes an approximate method, equivalent access intensity (EAI), for the throughput analysis of CSMA wireless networks in which links have finite offered-load and their MAC-layer transmit buffers may be empty from time to time. Different from prior works that mainly considered the saturated network, we take into account in our analysis the impacts of empty transmit buffers on the interactions and dependencies among links in the network that is more common in practice. It is known that the empty transmit buffer incurs extra waiting time for a link to compete for the channel airtime usage, since when it has no packet waiting for transmission, the link will not perform channel competition. The basic idea behind EAI is that this extra waiting time can be mapped to an equivalent "longer" backoff countdown time for the unsaturated link, yielding a lower link access intensity that is defined as the mean packet transmission time divided by the mean backoff countdown time. That is, we can compute the "equivalent access intensity" of an unsaturated link to incorporate the effects of the empty transmit buffer on its behavior of channel competition. Then, prior saturated ideal CSMA network (ICN) model can be adopted for link throughput computation. Specifically, we propose an iterative algorithm, "Compute-and-Compare", to identify which links are unsaturated under current offered-load and protocol settings, compute their "equivalent access intensities" and calculate link throughputs. Simulation shows that our algorithm has high accuracy under various offered-load and protocol settings. We believe the ability to identify unsaturated links and compute links throughputs as established in this paper will serve an important first step toward the design and optimization of general CSMA wireless networks with offered-load control.Comment: 6 pages. arXiv admin note: text overlap with arXiv:1007.5255 by other author