Wireless Sensor Networking in Challenging Environments

Recent years have witnessed growing interest in deploying wireless sensing applications in real-world environments. For example, home automation systems provide fine-grained metering and control of home appliances in residential settings. Similarly, assisted living applications employ wireless sensors to provide continuous health and wellness monitoring in homes. However, real deployments of Wireless Sensor Networks (WSNs) pose significant challenges due to their low-power radios and uncontrolled ambient environments. Our empirical study in over 15 real-world apartments shows that low-power WSNs based on the IEEE 802.15.4 standard are highly susceptible to external interference beyond user control, such as Wi-Fi access points, Bluetooth peripherals, cordless phones, and numerous other devices prevalent in residential environments that share the unlicensed 2.4 GHz ISM band with IEEE 802.15.4 radios. To address these real-world challenges, we developed two practical wireless network protocols including the Adaptive and Robust Channel Hopping (ARCH) protocol and the Adaptive Energy Detection Protocol (AEDP). ARCH enhances network reliability through opportunistically changing radio\u27s frequency to avoid interference and environmental noise and AEDP reduces false wakeups in noisy wireless environments by dynamically adjusting the wakeup threshold of low-power radios. Another major trend in WSNs is the convergence with smart phones. To deal with the dynamic wireless conditions and varying application requirements of mobile users, we developed the Self-Adapting MAC Layer (SAML) to support adaptive communication between smart phones and wireless sensors. SAML dynamically selects and switches Medium Access Control protocols to accommodate changes in ambient conditions and application requirements. Compared with the residential and personal wireless systems, industrial applications pose unique challenges due to their critical demands on reliability and real-time performance. We developed an experimental testbed by realizing key network mechanisms of industrial Wireless Sensor and Actuator Networks (WSANs) and conducted an empirical study that revealed the limitations and potential enhancements of those mechanisms. Our study shows that graph routing is more resilient to interference and its backup routes may be heavily used in noisy environments, which demonstrate the necessity of path diversity for reliable WSANs. Our study also suggests that combining channel diversity with retransmission may effectively reduce the burstiness of transmission failures and judicious allocation of multiple transmissions in a shared slot can effectively improve network capacity without significantly impacting reliability

Latency and Lifetime Enhancements in Industrial Wireless Sensor Networks : A Q-Learning Approach for Graph Routing

Industrial wireless sensor networks usually have a centralized management approach, where a device known as network manager is responsible for the overall configuration, definition of routes, and allocation of communication resources. Graph routing is used to increase the reliability of communication through path redundancy. Some of the state-of-the-art graph-routing algorithms use weighted cost equations to define preferences on how the routes are constructed. The characteristics and requirements of these networks complicate to find a proper set of weight values to enhance network performance. Reinforcement learning can be useful to adjust these weights according to the current operating conditions of the network. In this article, we present the Q-learning reliable routing with a weighting agent approach, where an agent adjusts the weights of a state-of-the-art graph-routing algorithm. The states of the agent represent sets of weights, and the actions change the weights during network operation. Rewards are given to the agent when the average network latency decreases or the expected network lifetime increases. Simulations were conducted on a WirelessHART simulator considering industrial monitoring applications with random topologies. Results show, in most cases, a reduction of the average network latency while the expected network lifetime and the communication reliability are at least as good as what is obtained by the state-of-the-art graph-routing algorithms

High Performance Wireless Sensor-Actuator Networks for Industrial Internet of Things

Wireless Sensor-Actuator Networks (WSANs) enable cost-effective communication for Industrial Internet of Things (IIoT). To achieve predictability and reliability demanded by industrial applications, industrial wireless standards (e.g., WirelessHART) incorporate a set of unique features such as a centralized management architecture, Time Slotted Channel Hopping (TSCH), and conservative channel selection. However, those features also incur significant degradation in performance, efficiency, and agility. To overcome these key limitations of existing industrial wireless technologies, this thesis work develops and empirically evaluates a suite of novel network protocols and algorithms. The primary contributions of this thesis are four-fold. (1) We first build an experimental testbed realizing key features of the WirelessHART protocol stack, and perform a series of empirical studies to uncover the limitations and potential improvements of existing network features. (2) We then investigate the impacts of the industrial WSAN protocol’s channel selection mechanism on routing and real-time performance, and present new channel and link selection strategies that improve route diversity and real-time performance. (3) To further enhance performance, we propose and design conservative channel reuse, a novel approach to support concurrent transmissions in a same wireless channel while maintaining a high degree of reliability. (4) Lastly, to address the limitation of the centralized architecture in handling network dynamics, we develop REACT, a Reliable, Efficient, and Adaptive Control Plane for centralized network management. REACT is designed to reduce the latency and energy cost of network reconfiguration by incorporating a reconfiguration planner to reduce a rescheduling cost, and an update engine providing efficient and reliable mechanisms to support schedule reconfiguration. All the network protocols and algorithms developed in this thesis have been empirically evaluated on the wireless testbed. This thesis represents a step toward next-generation IIoT for industrial automation that demands high-performance and agile wireless communication

Application of reinforcement learning with Q-learning for the routing in industrial wireless sensors networks

Industrial Wireless Sensor Networks (IWSN) usually have a centralized management approach, where a device known as Network Manager is responsible for the overall configuration, definition of routes, and allocation of communication resources. The routing algorithms need to ensure path redundancy while reducing latency, power consumption, and resource usage. Graph routing algorithms are used to address these requirements. The dynamicity of wireless networks has been a challenge for tuning and developing routing algorithms, and Machine Learning models such as Reinforcement Learning have been applied in a promising way in Wireless Sensor Networks to select, adapt and optimize routes. The basic concept of Reinforcement Learning is the existence of a learning agent that acts and changes the state of the environment, and receives rewards. However, the existing approaches do not meet some of the requirements of the IWSN standards. In this context, this thesis proposes the Q-Learning Reliable Routing approach, where the Q-Learning model is used to build graph routes. Two approaches are presented: QLRR-WA and QLRR-MA. QLRR-WA uses a learning agent that adjusts the weights of the cost equation of a state-of-the-art routing algorithm to reduce the latency and increase the network lifetime. QLRR-MA uses several learning agents so nodes can choose connections in the graph trying to reduce the latency. Other contributions of this thesis are the performance comparison of the state-of-the-art graph-routing algorithms and the evaluation methodology proposed. The QLRR algorithms were evaluated in a WirelessHART simulator, considering industrial monitoring applications with random topologies. The performance was analyzed considering the average network latency, network lifetime, packet delivery ratio and the reliability of the graphs. The results showed that, when compared to the state of the art, QLRR-WA reduced the average network latency and improved the lifetime while keeping high reliability, while QLRR-MA reduced latency and increased packet delivery ratio with a reduction in the network lifetime. These results indicate that Reinforcement Learning may be helpful to optimize and improve network performance.As Redes Industriais de Sensores Sem Fio (IWSN) geralmente têm uma abordagem de gerenciamento centralizado, onde um dispositivo conhecido como Gerenciador de Rede é responsável pela configuração geral, definição de rotas e alocação de recursos de comunicação. Os algoritmos de roteamento precisam garantir a redundância de caminhos para as mensagens, e também reduzir a latência, o consumo de energia e o uso de recursos. O roteamento por grafos é usado para alcançar estes requisitos. A dinamicidade das redes sem fio tem sido um desafio para o ajuste e o desenvolvimento de algoritmos de roteamento, e modelos de Aprendizado de Máquina como o Aprendizado por Reforço têm sido aplicados de maneira promissora nas Redes de Sensores Sem Fio para selecionar, adaptar e otimizar rotas. O conceito básico do Aprendizado por Reforço envolve a existência de um agente de aprendizado que atua em um ambiente, altera o estado do ambiente e recebe recompensas. No entanto, as abordagens existentes não atendem a alguns dos requisitos dos padrões das IWSN. Nesse contexto, esta tese propõe a abordagem Q-Learning Reliable Routing, onde o modelo Q-Learning é usado para construir os grafos de roteamento. Duas abordagens são propostas: QLRR-WA e QLRR-MA. A abordagem QLRR-WA utiliza um agente de aprendizado que ajusta os pesos da equação de custo de um algoritmo de roteamento de estado da arte, com o objetivo de reduzir a latência e aumentar a vida útil da rede. A abordagem QLRR-MA utiliza diversos agente de aprendizado de forma que cada dispositivo na rede pode escolher suas conexões tentando reduzir a latência. Outras contribuições desta tese são a comparação de desempenho das abordagens com os algoritmos de roteamento de estado da arte e a metodologia de avaliação proposta. As abordagens do QLRR foram avaliadas com um simulador WirelessHART, considerando aplicações de monitoramento industrial com diversas topologias. O desempenho foi analisado considerando a latência média da rede, o tempo de vida esperado da rede, a taxa de entrega de pacotes e a confiabilidade dos grafos. Os resultados mostraram que, quando comparado com o estado da arte, o QLRR-WA reduziu a latência média da rede e melhorou o tempo de vida esperado, mantendo alta confiabilidade, enquanto o QLRR-MA reduziu a latência e aumentou a taxa de entrega de pacotes, ao custo de uma redução no tempo de vida esperado da rede. Esses resultados indicam que o Aprendizado por Reforço pode ser útil para otimizar e melhorar o desempenho destas redes

Quality-of-service in wireless sensor networks: state-of-the-art and future directions

Wireless sensor networks (WSNs) are one of today’s most prominent instantiations of the ubiquituous computing paradigm. In order to achieve high levels of integration, WSNs need to be conceived considering requirements beyond the mere system’s functionality. While Quality-of-Service (QoS) is traditionally associated with bit/data rate, network throughput, message delay and bit/packet error rate, we believe that this concept is too strict, in the sense that these properties alone do not reflect the overall quality-ofservice provided to the user/application. Other non-functional properties such as scalability, security or energy sustainability must also be considered in the system design. This paper identifies the most important non-functional properties that affect the overall quality of the service provided to the users, outlining their relevance, state-of-the-art and future research directions