12,509 research outputs found
Adaptive Channel Recommendation For Opportunistic Spectrum Access
We propose a dynamic spectrum access scheme where secondary users recommend
"good" channels to each other and access accordingly. We formulate the problem
as an average reward based Markov decision process. We show the existence of
the optimal stationary spectrum access policy, and explore its structure
properties in two asymptotic cases. Since the action space of the Markov
decision process is continuous, it is difficult to find the optimal policy by
simply discretizing the action space and use the policy iteration, value
iteration, or Q-learning methods. Instead, we propose a new algorithm based on
the Model Reference Adaptive Search method, and prove its convergence to the
optimal policy. Numerical results show that the proposed algorithms achieve up
to 18% and 100% performance improvement than the static channel recommendation
scheme in homogeneous and heterogeneous channel environments, respectively, and
is more robust to channel dynamics
Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks
This paper investigates the use of deep reinforcement learning (DRL) in a MAC
protocol for heterogeneous wireless networking referred to as
Deep-reinforcement Learning Multiple Access (DLMA). The thrust of this work is
partially inspired by the vision of DARPA SC2, a 3-year competition whereby
competitors are to come up with a clean-slate design that "best share spectrum
with any network(s), in any environment, without prior knowledge, leveraging on
machine-learning technique". Specifically, this paper considers the problem of
sharing time slots among a multiple of time-slotted networks that adopt
different MAC protocols. One of the MAC protocols is DLMA. The other two are
TDMA and ALOHA. The nodes operating DLMA do not know that the other two MAC
protocols are TDMA and ALOHA. Yet, by a series of observations of the
environment, its own actions, and the resulting rewards, a DLMA node can learn
an optimal MAC strategy to coexist harmoniously with the TDMA and ALOHA nodes
according to a specified objective (e.g., the objective could be the sum
throughput of all networks, or a general alpha-fairness objective)
Survey of Inter-satellite Communication for Small Satellite Systems: Physical Layer to Network Layer View
Small satellite systems enable whole new class of missions for navigation,
communications, remote sensing and scientific research for both civilian and
military purposes. As individual spacecraft are limited by the size, mass and
power constraints, mass-produced small satellites in large constellations or
clusters could be useful in many science missions such as gravity mapping,
tracking of forest fires, finding water resources, etc. Constellation of
satellites provide improved spatial and temporal resolution of the target.
Small satellite constellations contribute innovative applications by replacing
a single asset with several very capable spacecraft which opens the door to new
applications. With increasing levels of autonomy, there will be a need for
remote communication networks to enable communication between spacecraft. These
space based networks will need to configure and maintain dynamic routes, manage
intermediate nodes, and reconfigure themselves to achieve mission objectives.
Hence, inter-satellite communication is a key aspect when satellites fly in
formation. In this paper, we present the various researches being conducted in
the small satellite community for implementing inter-satellite communications
based on the Open System Interconnection (OSI) model. This paper also reviews
the various design parameters applicable to the first three layers of the OSI
model, i.e., physical, data link and network layer. Based on the survey, we
also present a comprehensive list of design parameters useful for achieving
inter-satellite communications for multiple small satellite missions. Specific
topics include proposed solutions for some of the challenges faced by small
satellite systems, enabling operations using a network of small satellites, and
some examples of small satellite missions involving formation flying aspects.Comment: 51 pages, 21 Figures, 11 Tables, accepted in IEEE Communications
Surveys and Tutorial
- …