Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

A Low-Complexity Approach to Distributed Cooperative Caching with Geographic Constraints

We consider caching in cellular networks in which each base station is equipped with a cache that can store a limited number of files. The popularity of the files is known and the goal is to place files in the caches such that the probability that a user at an arbitrary location in the plane will find the file that she requires in one of the covering caches is maximized. We develop distributed asynchronous algorithms for deciding which contents to store in which cache. Such cooperative algorithms require communication only between caches with overlapping coverage areas and can operate in asynchronous manner. The development of the algorithms is principally based on an observation that the problem can be viewed as a potential game. Our basic algorithm is derived from the best response dynamics. We demonstrate that the complexity of each best response step is independent of the number of files, linear in the cache capacity and linear in the maximum number of base stations that cover a certain area. Then, we show that the overall algorithm complexity for a discrete cache placement is polynomial in both network size and catalog size. In practical examples, the algorithm converges in just a few iterations. Also, in most cases of interest, the basic algorithm finds the best Nash equilibrium corresponding to the global optimum. We provide two extensions of our basic algorithm based on stochastic and deterministic simulated annealing which find the global optimum. Finally, we demonstrate the hit probability evolution on real and synthetic networks numerically and show that our distributed caching algorithm performs significantly better than storing the most popular content, probabilistic content placement policy and Multi-LRU caching policies.Comment: 24 pages, 9 figures, presented at SIGMETRICS'1

Pricing and Resource Allocation via Game Theory for a Small-Cell Video Caching System

Evidence indicates that downloading on-demand videos accounts for a dramatic increase in data traffic over cellular networks. Caching popular videos in the storage of small-cell base stations (SBS), namely, small-cell caching, is an efficient technology for reducing the transmission latency whilst mitigating the redundant transmissions of popular videos over back-haul channels. In this paper, we consider a commercialized small-cell caching system consisting of a network service provider (NSP), several video retailers (VR), and mobile users (MU). The NSP leases its SBSs to the VRs for the purpose of making profits, and the VRs, after storing popular videos in the rented SBSs, can provide faster local video transmissions to the MUs, thereby gaining more profits. We conceive this system within the framework of Stackelberg game by treating the SBSs as a specific type of resources. We first model the MUs and SBSs as two independent Poisson point processes, and develop, via stochastic geometry theory, the probability of the specific event that an MU obtains the video of its choice directly from the memory of an SBS. Then, based on the probability derived, we formulate a Stackelberg game to jointly maximize the average profit of both the NSP and the VRs. Also, we investigate the Stackelberg equilibrium by solving a non-convex optimization problem. With the aid of this game theoretic framework, we shed light on the relationship between four important factors: the optimal pricing of leasing an SBS, the SBSs allocation among the VRs, the storage size of the SBSs, and the popularity distribution of the VRs. Monte-Carlo simulations show that our stochastic geometry-based analytical results closely match the empirical ones. Numerical results are also provided for quantifying the proposed game-theoretic framework by showing its efficiency on pricing and resource allocation.Comment: Accepted to appear in IEEE Journal on Selected Areas in Communications, special issue on Video Distribution over Future Interne

Approaches for Future Internet architecture design and Quality of Experience (QoE) Control

Researching a Future Internet capable of overcoming the current Internet limitations is a strategic investment. In this respect, this paper presents some concepts that can contribute to provide some guidelines to overcome the above-mentioned limitations. In the authors' vision, a key Future Internet target is to allow applications to transparently, efficiently and flexibly exploit the available network resources with the aim to match the users' expectations. Such expectations could be expressed in terms of a properly defined Quality of Experience (QoE). In this respect, this paper provides some approaches for coping with the QoE provision problem