Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Route selection for multi-hop cognitive radio networks using reinforcement learning: an experimental study

Cognitive radio (CR) enables unlicensed users to explore and exploit underutilized licensed channels (or white spaces). While multi-hop CR network has drawn significant research interest in recent years, majority work has been validated through simulation. A key challenge in multi-hop CR network is to select a route with high quality of service (QoS) and lesser number of route breakages. In this paper, we propose three route selection schemes to enhance the network performance of CR networks, and investigate them using a real testbed environment, which consists of universal software radio peripheral and GNU radio units. Two schemes are based on reinforcement learning (RL), while a scheme is based on spectrum leasing (SL). RL is an artificial intelligence technique, whereas SL is a new paradigm that allows communication between licensed and unlicensed users in CR networks. We compare the route selection schemes with an existing route selection scheme in the literature, called highest-channel (HC), in a multi-hop CR network. With respect to the QoS parameters (i.e., throughput, packet delivery ratio, and the number of route breakages), the experimental results show that RL approaches achieve a better performance in comparison with the HC approach, and also achieve close to the performance achieved by the SL approach

Designing and Implementing Future Aerial Communication Networks

Providing "connectivity from the sky" is the new innovative trend in wireless communications. High and low altitude platforms, drones, aircrafts and airships are being considered as the candidates for deploying wireless communications complementing the terrestrial communication infrastructure. In this article, we report the detailed account of the design and implementation challenges of an aerial network consisting of LTE Advanced (LTE-A) base stations. In particular, we review achievements and innovations harnessed by an aerial network composed of Helikite platforms. Helikites can be raised in the sky to bring Internet access during special events and in the aftermath of an emergency. The trial phase of the system mounting LTE-A technology onboard Helikites to serve users on the ground showed not only to be very encouraging but that such a system could offer even a longer lasting solution provided that inefficiency in powering the radio frequency equipment in the Helikite can be overcome.Comment: IEEE Communications Magazine 201

Modelling, Dimensioning and Optimization of 5G Communication Networks, Resources and Services

This reprint aims to collect state-of-the-art research contributions that address challenges in the emerging 5G networks design, dimensioning and optimization. Designing, dimensioning and optimization of communication networks resources and services have been an inseparable part of telecom network development. The latter must convey a large volume of traffic, providing service to traffic streams with highly differentiated requirements in terms of bit-rate and service time, required quality of service and quality of experience parameters. Such a communication infrastructure presents many important challenges, such as the study of necessary multi-layer cooperation, new protocols, performance evaluation of different network parts, low layer network design, network management and security issues, and new technologies in general, which will be discussed in this book

Supporting unified distributed management and autonomic decisions: design, implementation and deployment

Nowadays, the prevailing use of networks based on traditional centralized management systems reflects on a fast increase of the management costs. The growth in the number of network equipments and services reinforces the need to distribute the management responsibilities throughout the network devices. In this approach, each device executes common network management functionalities, being part of the overall network management platform. In this paper, we present a Unified Distributed Network Management (UDNM) framework that provides a unified (wired and wireless) management network solution, where further different network services can take part of this infrastructure, e.g., flow monitoring, accurate routing decisions, distributed policies dissemination, etc. This framework is divided in two main components: (A) Situation awareness, which sets up initial information through bootstrapping, discovery, fault-management process and exchange of management information; (B) Autonomic Decision System (ADS) that performs distributed decisions in the network with incomplete information. We deploy the UDNM framework in a testbed which involves two cities ( ≈ ≈ 250 km between), different standards (IEEE 802.3, IEEE 802.11 and IEEE 802.16e) and network technologies, such as, wired virtual grid, wireless ad-hoc gateways, ad-hoc mobile access devices. The UDNM framework integrates management functionalities into the managed devices, proving to be a lightweight and easy-respond framework. The performance analysis shows that the UDNM framework is feasible to unify devices management functionalities and to take accurate decisions on top of a real network.info:eu-repo/semantics/publishedVersio

Wireless Sensor Networking in Challenging Environments

Recent years have witnessed growing interest in deploying wireless sensing applications in real-world environments. For example, home automation systems provide fine-grained metering and control of home appliances in residential settings. Similarly, assisted living applications employ wireless sensors to provide continuous health and wellness monitoring in homes. However, real deployments of Wireless Sensor Networks (WSNs) pose significant challenges due to their low-power radios and uncontrolled ambient environments. Our empirical study in over 15 real-world apartments shows that low-power WSNs based on the IEEE 802.15.4 standard are highly susceptible to external interference beyond user control, such as Wi-Fi access points, Bluetooth peripherals, cordless phones, and numerous other devices prevalent in residential environments that share the unlicensed 2.4 GHz ISM band with IEEE 802.15.4 radios. To address these real-world challenges, we developed two practical wireless network protocols including the Adaptive and Robust Channel Hopping (ARCH) protocol and the Adaptive Energy Detection Protocol (AEDP). ARCH enhances network reliability through opportunistically changing radio\u27s frequency to avoid interference and environmental noise and AEDP reduces false wakeups in noisy wireless environments by dynamically adjusting the wakeup threshold of low-power radios. Another major trend in WSNs is the convergence with smart phones. To deal with the dynamic wireless conditions and varying application requirements of mobile users, we developed the Self-Adapting MAC Layer (SAML) to support adaptive communication between smart phones and wireless sensors. SAML dynamically selects and switches Medium Access Control protocols to accommodate changes in ambient conditions and application requirements. Compared with the residential and personal wireless systems, industrial applications pose unique challenges due to their critical demands on reliability and real-time performance. We developed an experimental testbed by realizing key network mechanisms of industrial Wireless Sensor and Actuator Networks (WSANs) and conducted an empirical study that revealed the limitations and potential enhancements of those mechanisms. Our study shows that graph routing is more resilient to interference and its backup routes may be heavily used in noisy environments, which demonstrate the necessity of path diversity for reliable WSANs. Our study also suggests that combining channel diversity with retransmission may effectively reduce the burstiness of transmission failures and judicious allocation of multiple transmissions in a shared slot can effectively improve network capacity without significantly impacting reliability