Enhancing graph routing algorithm of industrial wireless sensor networks using the covariance-matrix adaptation evolution strategy

The emergence of the Industrial Internet of Things (IIoT) has accelerated the adoption of Industrial Wireless Sensor Networks (IWSNs) for numerous applications. Effective communication in such applications requires reduced end-to-end transmission time, balanced energy consumption and increased communication reliability. Graph routing, the main routing method in IWSNs, has a significant impact on achieving effective communication in terms of satisfying these requirements. Graph routing algorithms involve applying the first-path available approach and using path redundancy to transmit data packets from a source sensor node to the gateway. However, this approach can affect end-to-end transmission time by creating conflicts among transmissions involving a common sensor node and promoting imbalanced energy consumption due to centralised management. The characteristics and requirements of these networks encounter further complications due to the need to find the best path on the basis of the requirements of IWSNs to overcome these challenges rather than using the available first-path. Such a requirement affects the network performance and prolongs the network lifetime. To address this problem, we adopt a Covariance-Matrix Adaptation Evolution Strategy (CMA-ES) to create and select the graph paths. Firstly, this article proposes three best single-objective graph routing paths according to the IWSN requirements that this research focused on. The sensor nodes select best paths based on three objective functions of CMA-ES: the best Path based on Distance (PODis), the best Path based on residual Energy (POEng) and the best Path based on End-to-End transmission time (POE2E). Secondly, to enhance energy consumption balance and achieve a balance among IWSN requirements, we adapt the CMA-ES to select the best path with multiple-objectives, otherwise known as the Best Path of Graph Routing with a CMA-ES (BPGR-ES). A simulation using MATALB with different configurations and parameters is applied to evaluate the enhanced graph routing algorithms. Furthermore, the performance of PODis, POEng, POE2E and BPGR-ES is compared with existing state-of-the-art graph routing algorithms. The simulation results reveal that the BPGR-ES algorithm achieved 87.53% more balanced energy consumption among sensor nodes in the network compared to other algorithms, and the delivery of data packets of BPGR-ES reached 99.86%, indicating more reliable communication

Multipath QoS-Driven Routing Protocol for Industrial Wireless Networks

Abstract—The Industry 4.0 concept targets the interconnection and computerization of traditional industries to improve their adaptability and utilize efficiently their resources. Industrial wireless networks will play a key role within the Industry 4.0 as they will facilitate the deployment of novel industrial applications thanks to the flexible and reconfigurable wireless connection of industrial devices. Significant advances are still necessary to enable the deployment of reliable industrial wireless networks capable to guarantee the strict QoS (Quality of Service) requirements of industrial applications in harsh propagation conditions. This paper contributes towards this objective with a novel multipath routing protocol that identifies and establishes the necessary redundant routes between any pair of nodes of a wireless network in order to satisfy the reliability and delay QoS levels demanded by industrial applications. The proposed protocol is here presented and analyzed under the framework of the WirelessHART standard given its important industrial adoption. However, it can also be adapted to other centralized TDMA-based multi-hop wireless networks.This work was supported by the Spanish Ministry of Economy and Competitiveness and FEDER funds under the projects TEC2011-26109, TEC2014-5716-R and TEC2014-56469-RED

Application of reinforcement learning with Q-learning for the routing in industrial wireless sensors networks

Industrial Wireless Sensor Networks (IWSN) usually have a centralized management approach, where a device known as Network Manager is responsible for the overall configuration, definition of routes, and allocation of communication resources. The routing algorithms need to ensure path redundancy while reducing latency, power consumption, and resource usage. Graph routing algorithms are used to address these requirements. The dynamicity of wireless networks has been a challenge for tuning and developing routing algorithms, and Machine Learning models such as Reinforcement Learning have been applied in a promising way in Wireless Sensor Networks to select, adapt and optimize routes. The basic concept of Reinforcement Learning is the existence of a learning agent that acts and changes the state of the environment, and receives rewards. However, the existing approaches do not meet some of the requirements of the IWSN standards. In this context, this thesis proposes the Q-Learning Reliable Routing approach, where the Q-Learning model is used to build graph routes. Two approaches are presented: QLRR-WA and QLRR-MA. QLRR-WA uses a learning agent that adjusts the weights of the cost equation of a state-of-the-art routing algorithm to reduce the latency and increase the network lifetime. QLRR-MA uses several learning agents so nodes can choose connections in the graph trying to reduce the latency. Other contributions of this thesis are the performance comparison of the state-of-the-art graph-routing algorithms and the evaluation methodology proposed. The QLRR algorithms were evaluated in a WirelessHART simulator, considering industrial monitoring applications with random topologies. The performance was analyzed considering the average network latency, network lifetime, packet delivery ratio and the reliability of the graphs. The results showed that, when compared to the state of the art, QLRR-WA reduced the average network latency and improved the lifetime while keeping high reliability, while QLRR-MA reduced latency and increased packet delivery ratio with a reduction in the network lifetime. These results indicate that Reinforcement Learning may be helpful to optimize and improve network performance.As Redes Industriais de Sensores Sem Fio (IWSN) geralmente têm uma abordagem de gerenciamento centralizado, onde um dispositivo conhecido como Gerenciador de Rede é responsável pela configuração geral, definição de rotas e alocação de recursos de comunicação. Os algoritmos de roteamento precisam garantir a redundância de caminhos para as mensagens, e também reduzir a latência, o consumo de energia e o uso de recursos. O roteamento por grafos é usado para alcançar estes requisitos. A dinamicidade das redes sem fio tem sido um desafio para o ajuste e o desenvolvimento de algoritmos de roteamento, e modelos de Aprendizado de Máquina como o Aprendizado por Reforço têm sido aplicados de maneira promissora nas Redes de Sensores Sem Fio para selecionar, adaptar e otimizar rotas. O conceito básico do Aprendizado por Reforço envolve a existência de um agente de aprendizado que atua em um ambiente, altera o estado do ambiente e recebe recompensas. No entanto, as abordagens existentes não atendem a alguns dos requisitos dos padrões das IWSN. Nesse contexto, esta tese propõe a abordagem Q-Learning Reliable Routing, onde o modelo Q-Learning é usado para construir os grafos de roteamento. Duas abordagens são propostas: QLRR-WA e QLRR-MA. A abordagem QLRR-WA utiliza um agente de aprendizado que ajusta os pesos da equação de custo de um algoritmo de roteamento de estado da arte, com o objetivo de reduzir a latência e aumentar a vida útil da rede. A abordagem QLRR-MA utiliza diversos agente de aprendizado de forma que cada dispositivo na rede pode escolher suas conexões tentando reduzir a latência. Outras contribuições desta tese são a comparação de desempenho das abordagens com os algoritmos de roteamento de estado da arte e a metodologia de avaliação proposta. As abordagens do QLRR foram avaliadas com um simulador WirelessHART, considerando aplicações de monitoramento industrial com diversas topologias. O desempenho foi analisado considerando a latência média da rede, o tempo de vida esperado da rede, a taxa de entrega de pacotes e a confiabilidade dos grafos. Os resultados mostraram que, quando comparado com o estado da arte, o QLRR-WA reduziu a latência média da rede e melhorou o tempo de vida esperado, mantendo alta confiabilidade, enquanto o QLRR-MA reduziu a latência e aumentou a taxa de entrega de pacotes, ao custo de uma redução no tempo de vida esperado da rede. Esses resultados indicam que o Aprendizado por Reforço pode ser útil para otimizar e melhorar o desempenho destas redes