2 research outputs found
Carrier-Sense Multiple Access for Heterogeneous Wireless Networks Using Deep Reinforcement Learning
This paper investigates a new class of carrier-sense multiple access (CSMA)
protocols that employ deep reinforcement learning (DRL) techniques for
heterogeneous wireless networking, referred to as carrier-sense
deep-reinforcement learning multiple access (CS-DLMA). Existing CSMA protocols,
such as the medium access control (MAC) of WiFi, are designed for a homogeneous
network environment in which all nodes adopt the same protocol. Such protocols
suffer from severe performance degradation in a heterogeneous environment where
there are nodes adopting other MAC protocols. This paper shows that DRL
techniques can be used to design efficient MAC protocols for heterogeneous
networking. In particular, in a heterogeneous environment with nodes adopting
different MAC protocols (e.g., CS-DLMA, TDMA, and ALOHA), a CS-DLMA node can
learn to maximize the sum throughput of all nodes. Furthermore, compared with
WiFi's CSMA, CS-DLMA can achieve both higher sum throughput and individual
throughputs when coexisting with other MAC protocols. Last but not least, a
salient feature of CS-DLMA is that it does not need to know the operating
mechanisms of the co-existing MACs. Neither does it need to know the number of
nodes using these other MACs.Comment: 8 page
Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks
This paper investigates a new class of carrier-sense multiple access (CSMA)
protocols that employ deep reinforcement learning (DRL) techniques, referred to
as carrier-sense deep-reinforcement learning multiple access (CS-DLMA). The
goal of CS-DLMA is to enable efficient and equitable spectrum sharing among a
group of co-located heterogeneous wireless networks. Existing CSMA protocols,
such as the medium access control (MAC) of WiFi, are designed for a homogeneous
network in which all nodes adopt the same protocol. Such protocols suffer from
severe performance degradation in a heterogeneous environment where there are
nodes adopting other MAC protocols. CS-DLMA aims to circumvent this problem by
making use of DRL. In particular, this paper adopts alpha-fairness as the
general objective of CS-DLMA. With alpha-fairness, CS-DLMA can achieve a range
of different objectives when coexisting with other MACs by changing the value
of alpha. A salient feature of CS-DLMA is that it can achieve these objectives
without knowing the coexisting MACs through a learning process based on DRL.
The underpinning DRL technique in CS-DLMA is deep Q-network (DQN). However, the
conventional DQN algorithms are not suitable for CS-DLMA due to their uniform
time-step assumption. In CSMA protocols, time steps are non-uniform in that the
time duration required for carrier sensing is smaller than the duration of data
transmission. This paper introduces a non-uniform time-step formulation of DQN
to address this issue. Our simulation results show that CS-DLMA can achieve the
general alpha-fairness objective when coexisting with TDMA, ALOHA, and WiFi
protocols by adjusting its own transmission strategy. Interestingly, we also
find that CS-DLMA is more Pareto efficient than other CSMA protocols when
coexisting with WiFi.Comment: 14 pages, 11 figure