An LSPI based reinforcement learning approach to enable network cooperation in cognitive wireless sensor networks

The number of wirelessly communicating devices increases every day, along with the number of communication standards and technologies that they use to exchange data. A relatively new form of research is trying to find a way to make all these co-located devices not only capable of detecting each other's presence, but to go one step further - to make them cooperate. One recently proposed way to tackle this problem is to engage into cooperation by activating 'network services' (such as internet sharing, interference avoidance, etc.) that offer benefits for other co-located networks. This approach reduces the problem to the following research topic: how to determine which network services would be beneficial for all the cooperating networks. In this paper we analyze and propose a conceptual solution for this problem using the reinforcement learning technique known as the Least Square Policy Iteration (LSPI). The proposes solution uses a self-learning entity that negotiates between different independent and co-located networks. First, the reasoning entity uses self-learning techniques to determine which service configuration should be used to optimize the network performance of each single network. Afterwards, this performance is used as a reference point and LSPI is used to deduce if cooperating with other co-located networks can lead to even further performance improvements

Cooperation techniques between LTE in unlicensed spectrum and Wi-Fi towards fair spectral efficiency

On the road towards 5G, a proliferation of Heterogeneous Networks (HetNets) is expected. Sensor networks are of great importance in this new wireless era, as they allow interaction with the environment. Additionally, the establishment of the Internet of Things (IoT) has incredibly increased the number of interconnected devices and consequently the already massive wirelessly transmitted traffic. The exponential growth of wireless traffic is pushing the wireless community to investigate solutions that maximally exploit the available spectrum. Recently, 3rd Generation Partnership Project (3GPP) announced standards that permit the operation of Long Term Evolution (LTE) in the unlicensed spectrum in addition to the exclusive use of the licensed spectrum owned by a mobile operator. Alternatively, leading wireless technology developers examine standalone LTE operation in the unlicensed spectrum without any involvement of a mobile operator. In this article, we present a classification of different techniques that can be applied on co-located LTE and Wi-Fi networks. Up to today, Wi-Fi is the most widely-used wireless technology in the unlicensed spectrum. A review of the current state of the art further reveals the lack of cooperation schemes among co-located networks that can lead to more optimal usage of the available spectrum. This article fills this gap in the literature by conceptually describing different classes of cooperation between LTE and Wi-Fi. For each class, we provide a detailed presentation of possible cooperation techniques that can provide spectral efficiency in a fair manner

Concept and framework of a self-regulating symbiotic network

The concept and framework of a self-regulating symbiotic network planner is introduced as a way to improve the use of available resources and infrastructure and the overall performance of co-located wireless networks. A framework for physical-layer optimization is proposed, based on an advanced and reliable network planner. Besides an optimal network planning including the adjustment of transmit powers, also a symbiotic optimization over different networks and network layers is implemented, a new concept in network cooperation based on shared and variable incentives. In this article, specifically, it is assumed that the co-located networks share the incentive of a lower global power consumption and the newly created symbiotic network is optimized accordingly. Feedback about the signal quality parameters allows optimizing path loss models, finetuning device transmit powers, coping with a changing propagation environment, and improving network reliability. The concept is applied to and experimentally validated with a real-life wireless test environment and a power consumption reduction of 79.5% is obtained, by consecutively enabling energy-saving features of the network planner: intelligent cognitive network planning, symbiotic network cooperation, and transmit power adjustments

Internet of robotic things : converging sensing/actuating, hypoconnectivity, artificial intelligence and IoT Platforms

The Internet of Things (IoT) concept is evolving rapidly and influencing newdevelopments in various application domains, such as the Internet of MobileThings (IoMT), Autonomous Internet of Things (A-IoT), Autonomous Systemof Things (ASoT), Internet of Autonomous Things (IoAT), Internetof Things Clouds (IoT-C) and the Internet of Robotic Things (IoRT) etc.that are progressing/advancing by using IoT technology. The IoT influencerepresents new development and deployment challenges in different areassuch as seamless platform integration, context based cognitive network integration,new mobile sensor/actuator network paradigms, things identification(addressing, naming in IoT) and dynamic things discoverability and manyothers. The IoRT represents new convergence challenges and their need to be addressed, in one side the programmability and the communication ofmultiple heterogeneous mobile/autonomous/robotic things for cooperating,their coordination, configuration, exchange of information, security, safetyand protection. Developments in IoT heterogeneous parallel processing/communication and dynamic systems based on parallelism and concurrencyrequire new ideas for integrating the intelligent “devices”, collaborativerobots (COBOTS), into IoT applications. Dynamic maintainability, selfhealing,self-repair of resources, changing resource state, (re-) configurationand context based IoT systems for service implementation and integrationwith IoT network service composition are of paramount importance whennew “cognitive devices” are becoming active participants in IoT applications.This chapter aims to be an overview of the IoRT concept, technologies,architectures and applications and to provide a comprehensive coverage offuture challenges, developments and applications