Decentralized Delay Optimal Control for Interference Networks with Limited Renewable Energy Storage

In this paper, we consider delay minimization for interference networks with renewable energy source, where the transmission power of a node comes from both the conventional utility power (AC power) and the renewable energy source. We assume the transmission power of each node is a function of the local channel state, local data queue state and local energy queue state only. In turn, we consider two delay optimization formulations, namely the decentralized partially observable Markov decision process (DEC-POMDP) and Non-cooperative partially observable stochastic game (POSG). In DEC-POMDP formulation, we derive a decentralized online learning algorithm to determine the control actions and Lagrangian multipliers (LMs) simultaneously, based on the policy gradient approach. Under some mild technical conditions, the proposed decentralized policy gradient algorithm converges almost surely to a local optimal solution. On the other hand, in the non-cooperative POSG formulation, the transmitter nodes are non-cooperative. We extend the decentralized policy gradient solution and establish the technical proof for almost-sure convergence of the learning algorithms. In both cases, the solutions are very robust to model variations. Finally, the delay performance of the proposed solutions are compared with conventional baseline schemes for interference networks and it is illustrated that substantial delay performance gain and energy savings can be achieved

Throughput Scaling of Wireless Networks With Random Connections

This work studies the throughput scaling laws of ad hoc wireless networks in the limit of a large number of nodes. A random connections model is assumed in which the channel connections between the nodes are drawn independently from a common distribution. Transmitting nodes are subject to an on-off strategy, and receiving nodes employ conventional single-user decoding. The following results are proven: 1) For a class of connection models with finite mean and variance, the throughput scaling is upper-bounded by $O(n^{1/3})$ for single-hop schemes, and $O(n^{1/2})$ for two-hop (and multihop) schemes. 2) The $\Theta (n^{1/2})$ throughput scaling is achievable for a specific connection model by a two-hop opportunistic relaying scheme, which employs full, but only local channel state information (CSI) at the receivers, and partial CSI at the transmitters. 3) By relaxing the constraints of finite mean and variance of the connection model, linear throughput scaling $\Theta (n)$ is achievable with Pareto-type fading models.Comment: 13 pages, 4 figures, To appear in IEEE Transactions on Information Theor

Transmit Power Minimization in Small Cell Networks Under Time Average QoS Constraints

We consider a small cell network (SCN) consisting of N cells, with the small cell base stations (SCBSs) equipped with Nt \geq 1 antennas each, serving K single antenna user terminals (UTs) per cell. Under this set up, we address the following question: given certain time average quality of service (QoS) targets for the UTs, what is the minimum transmit power expenditure with which they can be met? Our motivation to consider time average QoS constraint comes from the fact that modern wireless applications such as file sharing, multi-media etc. allow some flexibility in terms of their delay tolerance. Time average QoS constraints can lead to greater transmit power savings as compared to instantaneous QoS constraints since it provides the flexibility to dynamically allocate resources over the fading channel states. We formulate the problem as a stochastic optimization problem whose solution is the design of the downlink beamforming vectors during each time slot. We solve this problem using the approach of Lyapunov optimization and characterize the performance of the proposed algorithm. With this algorithm as the reference, we present two main contributions that incorporate practical design considerations in SCNs. First, we analyze the impact of delays incurred in information exchange between the SCBSs. Second, we impose channel state information (CSI) feedback constraints, and formulate a joint CSI feedback and beamforming strategy. In both cases, we provide performance bounds of the algorithm in terms of satisfying the QoS constraints and the time average power expenditure. Our simulation results show that solving the problem with time average QoS constraints provide greater savings in the transmit power as compared to the instantaneous QoS constraints.Comment: in Journal on Selected Areas of Communications (JSAC), 201

Opportunistic Relaying in Wireless Networks

Relay networks having $n$ source-to-destination pairs and $m$ half-duplex relays, all operating in the same frequency band in the presence of block fading, are analyzed. This setup has attracted significant attention and several relaying protocols have been reported in the literature. However, most of the proposed solutions require either centrally coordinated scheduling or detailed channel state information (CSI) at the transmitter side. Here, an opportunistic relaying scheme is proposed, which alleviates these limitations. The scheme entails a two-hop communication protocol, in which sources communicate with destinations only through half-duplex relays. The key idea is to schedule at each hop only a subset of nodes that can benefit from \emph{multiuser diversity}. To select the source and destination nodes for each hop, it requires only CSI at receivers (relays for the first hop, and destination nodes for the second hop) and an integer-value CSI feedback to the transmitters. For the case when $n$ is large and $m$ is fixed, it is shown that the proposed scheme achieves a system throughput of $m/2$ bits/s/Hz. In contrast, the information-theoretic upper bound of $(m/2)\log \log n$ bits/s/Hz is achievable only with more demanding CSI assumptions and cooperation between the relays. Furthermore, it is shown that, under the condition that the product of block duration and system bandwidth scales faster than $\log n$, the achievable throughput of the proposed scheme scales as $\Theta ({\log n})$. Notably, this is proven to be the optimal throughput scaling even if centralized scheduling is allowed, thus proving the optimality of the proposed scheme in the scaling law sense.Comment: 17 pages, 8 figures, To appear in IEEE Transactions on Information Theor

Delay-Optimal User Scheduling and Inter-Cell Interference Management in Cellular Network via Distributive Stochastic Learning

In this paper, we propose a distributive queueaware intra-cell user scheduling and inter-cell interference (ICI) management control design for a delay-optimal celluar downlink system with M base stations (BSs), and K users in each cell. Each BS has K downlink queues for K users respectively with heterogeneous arrivals and delay requirements. The ICI management control is adaptive to joint queue state information (QSI) over a slow time scale, while the user scheduling control is adaptive to both the joint QSI and the joint channel state information (CSI) over a faster time scale. We show that the problem can be modeled as an infinite horizon average cost Partially Observed Markov Decision Problem (POMDP), which is NP-hard in general. By exploiting the special structure of the problem, we shall derive an equivalent Bellman equation to solve the POMDP problem. To address the distributive requirement and the issue of dimensionality and computation complexity, we derive a distributive online stochastic learning algorithm, which only requires local QSI and local CSI at each of the M BSs. We show that the proposed learning algorithm converges almost surely (with probability 1) and has significant gain compared with various baselines. The proposed solution only has linear complexity order O(MK)

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Uplink CoMP under a Constrained Backhaul and Imperfect Channel Knowledge

Coordinated Multi-Point (CoMP) is known to be a key technology for next generation mobile communications systems, as it allows to overcome the burden of inter-cell interference. Especially in the uplink, it is likely that interference exploitation schemes will be used in the near future, as they can be used with legacy terminals and require no or little changes in standardization. Major drawbacks, however, are the extent of additional backhaul infrastructure needed, and the sensitivity to imperfect channel knowledge. This paper jointly addresses both issues in a new framework incorporating a multitude of proposed theoretical uplink CoMP concepts, which are then put into perspective with practical CoMP algorithms. This comprehensive analysis provides new insight into the potential usage of uplink CoMP in next generation wireless communications systems.Comment: Submitted to IEEE Transactions on Wireless Communications in February 201