Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Spectrum Map and its Application in Cognitive Radio Networks

Recent measurements on radio spectrum usage have revealed the abundance of underutilized bands of spectrum that belong to licensed users. This necessitated the paradigm shift from static to dynamic spectrum access. Cognitive radio based secondary networks that utilize such unused spectrum holes in the licensed band, have been proposed as a possible solution to the spectrum crisis. The idea is to detect times when a particular licensed band is unused and use it for transmission without causing interference to the licensed user. We argue that prior knowledge about occupancy of such bands and the corresponding achievable performance metrics can potentially help secondary networks to devise effective strategies to improve utilization. In this work, we use Shepard\u27s method of interpolation to create a spectrum map that provides a spatial distribution of spectrum usage over a region of interest. It is achieved by intelligently fusing the spectrum usage reports shared by the secondary nodes at various locations. The obtained spectrum map is a continuous and differentiable 2-dimension distribution function in space. With the spectrum usage distribution known, we show how different radio spectrum and network performance metrics like channel capacity, secondary network throughput, spectral efficiency, and bit error rate can be estimated. We show the applicability of the spectrum map in solving the intra-cell channel allocation problem in centralized cognitive radio networks, such as IEEE 802.22. We propose a channel allocation scheme where the base station allocates interference free channels to the consumer premise equipments (CPE) using the spectrum map that it creates by fusing the spectrum usage information shared by some CPEs. The most suitable CPEs for information sharing are chosen on a dynamic basis using an iterative clustering algorithm. Next, we present a contention based media access control (MAC) protocol for distributed cognitive radio network. The unlicensed secondary users contend among themselves over a common control channel. Winners of the contention get to access the available channels ensuring high utilization and minimum collision with primary incumbent. Last, we propose a multi-channel, multi-hop routing protocol with secondary transmission power control. The spectrum map, created and maintained by a set of sensors, acts as the basis of finding the best route for every source destination pair. The proposed routing protocol ensures primary receiver protection and maximizes achievable link capacity. Through simulation experiments we show the correctness of the prediction model and how it can be used by secondary networks for strategic positioning of secondary transmitter-receiver pairs and selecting the best candidate channels. The simulation model mimics realistic distribution of TV stations for urban and non-urban areas. Results validate the nature and accuracy of estimation, prediction of performance metrics, and efficiency of the allocation process in an IEEE 802.22 network. Results for the proposed MAC protocol show high channel utilization with primary quality of service degradation within a tolerable limit. Performance evaluation of the proposed routing scheme reveals that it ensures primary receiver protection through secondary power control and maximizes route capacity

On Random Sampling for Compliance Monitoring in Opportunistic Spectrum Access Networks

In the expanding spectrum marketplace, there has been a long term evolution towards more market€“oriented mechanisms, such as Opportunistic Spectrum Access (OSA), enabled through Cognitive Radio (CR) technology. However, the potential of CR technologies to revolutionize wireless communications, also introduces challenges based upon the potentially non€“deterministic CR behaviour in the Electrospace. While establishing and enforcing compliance to spectrum etiquette rules are essential to realization of successful OSA networks in the future, there has only been recent increased research activity into enforcement. This dissertation presents novel work on the spectrum monitoring aspect, which is crucial to effective enforcement of OSA. An overview of the challenges faced by current compliance monitoring methods is first presented. A framework is then proposed for the use of random spectral sampling techniques to reduce data collection complexity in wideband sensing scenarios. This approach is recommended as an alternative to Compressed Sensing (CS) techniques for wideband spectral occupancy estimation, which may be difficult to utilize in many practical congested scenarios where compliance monitoring is required. Next, a low€“cost computational approach to online randomized temporal sensing deployment is presented for characterization of temporal spectrum occupancy in cognitive radio scenarios. The random sensing approach is demonstrated and its performance is compared to CS€“based approach for occupancy estimation. A novel frame€“based sampling inversion technique is then presented for cases when it is necessary to track the temporal behaviour of individual CRs or CR networks. Parameters from randomly sampled Physical Layer Convergence Protocol (PLCP) data frames are used to reconstruct occupancy statistics, taking account of missed frames due to sampling design, sensor limitations and frame errors. Finally, investigations into the use of distributed and mobile spectrum sensing to collect spatial diversity to improve the above techniques are presented, for several common monitoring tasks in spectrum enforcement. Specifically, focus is upon techniques for achieving consensus in dynamic topologies such as in mobile sensing scenarios

Comprehensive survey on quality of service provisioning approaches in cognitive radio networks : part one

Much interest in Cognitive Radio Networks (CRNs) has been raised recently by enabling unlicensed (secondary) users to utilize the unused portions of the licensed spectrum. CRN utilization of residual spectrum bands of Primary (licensed) Networks (PNs) must avoid harmful interference to the users of PNs and other overlapping CRNs. The coexisting of CRNs depends on four components: Spectrum Sensing, Spectrum Decision, Spectrum Sharing, and Spectrum Mobility. Various approaches have been proposed to improve Quality of Service (QoS) provisioning in CRNs within fluctuating spectrum availability. However, CRN implementation poses many technical challenges due to a sporadic usage of licensed spectrum bands, which will be increased after deploying CRNs. Unlike traditional surveys of CRNs, this paper addresses QoS provisioning approaches of CRN components and provides an up-to-date comprehensive survey of the recent improvement in these approaches. Major features of the open research challenges of each approach are investigated. Due to the extensive nature of the topic, this paper is the first part of the survey which investigates QoS approaches on spectrum sensing and decision components respectively. The remaining approaches of spectrum sharing and mobility components will be investigated in the next part

Spectrum-efficient Architecture for Cognitive Wireless Sensor Networks

Projecte realitzat en col.laboració amb el centre Université Libre de BruxellesHoy en día existe la creencia de que en unos pocos años las actuales Redes Inalámbricas de Sensores estarán presentes en muchas aplicaciones. Mientras estas sigan actuando en la banda sin licencia de ISM 2,4GHz, tendrán que coexistir con otras exitosas tecnologías como Wi-Fi o Bluetooth. En consecuencia, resulta obvio asegurar que la banda en cuestión estará superpoblada en un futuro próximo. Sin embargo y gracias a las nuevas técnicas de Radio Cognitiva, que permitirán la aplicación de un eficiente Acceso al Espectro Dinámico, se conseguirá una distribución racional, dentro del espectro disponible en ese momento y lugar, de las comunicaciones inalámbricas que se estén llevando a cabo. Esta actuación permitirá acceder a frecuencias menos pobladas para poder transmitir con menos interferencias e incluso con menos pérdidas de propagación. A lo largo de este trabajo se va a presentar una arquitectura eficiente, espectralmente hablando, para Redes Inalámbricas de Sensores y Cognitivas. Este esquema desarrolla un protocolo de recolección de datos, para una red con topología de árbol, totalmente escalable y con finalidades genéricas. A través de las pruebas realizadas, podemos afirmar que nuestro esquema, sin alterar el ciclo normal de recolección de datos, puede detectar la presencia de otras Redes Inalámbricas de Sensores y, consecuentemente, migrar la red a nueva frecuencia mientras que todas estas operaciones están ocultas al usuario final. También es eficiente a nivel de energía, ya que no se realizan comprobaciones redundantes de la presencia de otras redes. De esta manera, nuestra propuesta asegura un mejor comportamiento en caso de la existencia de una Red Inalámbrica de Sensores externa, sin realizar operaciones complicadas ni añadiendo más tráfico a la red