Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Optimized Cluster-Based Dynamic Energy-Aware Routing Protocol for Wireless Sensor Networks in Agriculture Precision

[EN] Wireless sensor networks (WSNs) are becoming one of the demanding platforms, where sensor nodes are sensing and monitoring the physical or environmental conditions and transmit the data to the base station via multihop routing. Agriculture sector also adopted these networks to promote innovations for environmental friendly farming methods, lower the management cost, and achieve scientific cultivation. Due to limited capabilities, the sensor nodes have suffered with energy issues and complex routing processes and lead to data transmission failure and delay in the sensor-based agriculture fields. Due to these limitations, the sensor nodes near the base station are always relaying on it and cause extra burden on base station or going into useless state. To address these issues, this study proposes a Gateway Clustering Energy-Efficient Centroid- (GCEEC-) based routing protocol where cluster head is selected from the centroid position and gateway nodes are selected from each cluster. Gateway node reduces the data load from cluster head nodes and forwards the data towards the base station. Simulation has performed to evaluate the proposed protocol with state-of-the-art protocols. The experimental results indicated the better performance of proposed protocol and provide more feasible WSN-based monitoring for temperature, humidity, and illumination in agriculture sector.This work has also been partially supported by the European Union through the ERANETMED (Euromediterranean Cooperation through ERANET joint activities and beyond) project ERANETMED3-227 SMARTWATIR.Qureshi, KN.; Bashir, MU.; Lloret, J.; León Fernández, A. (2020). Optimized Cluster-Based Dynamic Energy-Aware Routing Protocol for Wireless Sensor Networks in Agriculture Precision. Journal of Sensors. 2020:1-19. https://doi.org/10.1155/2020/9040395S1192020Sneha, K., Kamath, R., Balachandra, M., & Prabhu, S. (2019). New Gossiping Protocol for Routing Data in Sensor Networks for Precision Agriculture. Soft Computing and Signal Processing, 139-152. doi:10.1007/978-981-13-3393-4_15Qureshi, K. N., Abdullah, A. H., Bashir, F., Iqbal, S., & Awan, K. M. (2018). Cluster-based data dissemination, cluster head formation under sparse, and dense traffic conditions for vehicular ad hoc networks. International Journal of Communication Systems, 31(8), e3533. doi:10.1002/dac.3533Rault, T., Bouabdallah, A., & Challal, Y. (2014). Energy efficiency in wireless sensor networks: A top-down survey. Computer Networks, 67, 104-122. doi:10.1016/j.comnet.2014.03.027Feng, X., Zhang, J., Ren, C., & Guan, T. (2018). An Unequal Clustering Algorithm Concerned With Time-Delay for Internet of Things. IEEE Access, 6, 33895-33909. doi:10.1109/access.2018.2847036Savaglio, C., Pace, P., Aloi, G., Liotta, A., & Fortino, G. (2019). Lightweight Reinforcement Learning for Energy Efficient Communications in Wireless Sensor Networks. IEEE Access, 7, 29355-29364. doi:10.1109/access.2019.2902371Srbinovska, M., Gavrovski, C., Dimcev, V., Krkoleva, A., & Borozan, V. (2015). Environmental parameters monitoring in precision agriculture using wireless sensor networks. Journal of Cleaner Production, 88, 297-307. doi:10.1016/j.jclepro.2014.04.036Lloret, J., Garcia, M., Bri, D., & Diaz, J. (2009). A Cluster-Based Architecture to Structure the Topology of Parallel Wireless Sensor Networks. Sensors, 9(12), 10513-10544. doi:10.3390/s91210513Qureshi, K. N., Din, S., Jeon, G., & Piccialli, F. (2020). Link quality and energy utilization based preferable next hop selection routing for wireless body area networks. Computer Communications, 149, 382-392. doi:10.1016/j.comcom.2019.10.030Kumar, S. A., & Ilango, P. (2017). The Impact of Wireless Sensor Network in the Field of Precision Agriculture: A Review. Wireless Personal Communications, 98(1), 685-698. doi:10.1007/s11277-017-4890-zAnisi, M. H., Abdul-Salaam, G., & Abdullah, A. H. (2014). A survey of wireless sensor network approaches and their energy consumption for monitoring farm fields in precision agriculture. Precision Agriculture, 16(2), 216-238. doi:10.1007/s11119-014-9371-8Long, D. S., & McCallum, J. D. (2015). On-combine, multi-sensor data collection for post-harvest assessment of environmental stress in wheat. Precision Agriculture, 16(5), 492-504. doi:10.1007/s11119-015-9391-zFu, X., Fortino, G., Li, W., Pace, P., & Yang, Y. (2019). WSNs-assisted opportunistic network for low-latency message forwarding in sparse settings. Future Generation Computer Systems, 91, 223-237. doi:10.1016/j.future.2018.08.031Mehmood, A., Khan, S., Shams, B., & Lloret, J. (2013). Energy-efficient multi-level and distance-aware clustering mechanism for WSNs. International Journal of Communication Systems, 28(5), 972-989. doi:10.1002/dac.2720Pantazis, N. A., Nikolidakis, S. A., & Vergados, D. D. (2013). Energy-Efficient Routing Protocols in Wireless Sensor Networks: A Survey. IEEE Communications Surveys & Tutorials, 15(2), 551-591. doi:10.1109/surv.2012.062612.00084De Farias, C. M., Pirmez, L., Fortino, G., & Guerrieri, A. (2019). A multi-sensor data fusion technique using data correlations among multiple applications. Future Generation Computer Systems, 92, 109-118. doi:10.1016/j.future.2018.09.034Rao, P. C. S., Jana, P. K., & Banka, H. (2016). A particle swarm optimization based energy efficient cluster head selection algorithm for wireless sensor networks. Wireless Networks, 23(7), 2005-2020. doi:10.1007/s11276-016-1270-7Fu, X., Fortino, G., Pace, P., Aloi, G., & Li, W. (2020). Environment-fusion multipath routing protocol for wireless sensor networks. Information Fusion, 53, 4-19. doi:10.1016/j.inffus.2019.06.001Liu, X. (2015). Atypical Hierarchical Routing Protocols for Wireless Sensor Networks: A Review. IEEE Sensors Journal, 15(10), 5372-5383. doi:10.1109/jsen.2015.2445796Jan, N., Javaid, N., Javaid, Q., Alrajeh, N., Alam, M., Khan, Z. A., & Niaz, I. A. (2017). A Balanced Energy-Consuming and Hole-Alleviating Algorithm for Wireless Sensor Networks. IEEE Access, 5, 6134-6150. doi:10.1109/access.2017.2676004Gupta, G. P., Misra, M., & Garg, K. (2014). Energy and trust aware mobile agent migration protocol for data aggregation in wireless sensor networks. Journal of Network and Computer Applications, 41, 300-311. doi:10.1016/j.jnca.2014.01.003Safa, H., Karam, M., & Moussa, B. (2014). PHAODV: Power aware heterogeneous routing protocol for MANETs. Journal of Network and Computer Applications, 46, 60-71. doi:10.1016/j.jnca.2014.07.035Liu, X. (2015). An Optimal-Distance-Based Transmission Strategy for Lifetime Maximization of Wireless Sensor Networks. IEEE Sensors Journal, 15(6), 3484-3491. doi:10.1109/jsen.2014.2372340Brar, G. S., Rani, S., Chopra, V., Malhotra, R., Song, H., & Ahmed, S. H. (2016). Energy Efficient Direction-Based PDORP Routing Protocol for WSN. IEEE Access, 4, 3182-3194. doi:10.1109/access.2016.2576475Abo-Zahhad, M., Ahmed, S. M., Sabor, N., & Sasaki, S. (2015). Mobile Sink-Based Adaptive Immune Energy-Efficient Clustering Protocol for Improving the Lifetime and Stability Period of Wireless Sensor Networks. IEEE Sensors Journal, 15(8), 4576-4586. doi:10.1109/jsen.2015.2424296Huynh, T.-T., Dinh-Duc, A.-V., & Tran, C.-H. (2016). Delay-constrained energy-efficient cluster-based multi-hop routing in wireless sensor networks. Journal of Communications and Networks, 18(4), 580-588. doi:10.1109/jcn.2016.000081Shen, J., Wang, A., Wang, C., Hung, P. C. K., & Lai, C.-F. (2017). An Efficient Centroid-Based Routing Protocol for Energy Management in WSN-Assisted IoT. IEEE Access, 5, 18469-18479. doi:10.1109/access.2017.2749606Awan, K. M., Shah, P. A., Iqbal, K., Gillani, S., Ahmad, W., & Nam, Y. (2019). Underwater Wireless Sensor Networks: A Review of Recent Issues and Challenges. Wireless Communications and Mobile Computing, 2019, 1-20. doi:10.1155/2019/6470359Sajwan, M., Gosain, D., & Sharma, A. K. (2018). CAMP: cluster aided multi-path routing protocol for wireless sensor networks. Wireless Networks, 25(5), 2603-2620. doi:10.1007/s11276-018-1689-0Varga, A. (2010). OMNeT++. Modeling and Tools for Network Simulation, 35-59. doi:10.1007/978-3-642-12331-3_3Lartillot, O., Toiviainen, P., & Eerola, T. (2008). A Matlab Toolbox for Music Information Retrieval. Studies in Classification, Data Analysis, and Knowledge Organization, 261-268. doi:10.1007/978-3-540-78246-9_31Mathur, P., Nielsen, R. H., Prasad, N. R., & Prasad, R. (2016). Data collection using miniature aerial vehicles in wireless sensor networks. IET Wireless Sensor Systems, 6(1), 17-25. doi:10.1049/iet-wss.2014.0120Zou, T., Lin, S., Feng, Q., & Chen, Y. (2016). Energy-Efficient Control with Harvesting Predictions for Solar-Powered Wireless Sensor Networks. Sensors, 16(1), 53. doi:10.3390/s16010053Song, Y., Ma, J., Zhang, X., & Feng, Y. (2012). Design of Wireless Sensor Network-Based Greenhouse Environment Monitoring and Automatic Control System. Journal of Networks, 7(5). doi:10.4304/jnw.7.5.838-844Nikolidakis, S., Kandris, D., Vergados, D., & Douligeris, C. (2013). Energy Efficient Routing in Wireless Sensor Networks Through Balanced Clustering. Algorithms, 6(1), 29-42. doi:10.3390/a6010029Ndzi, D. L., Harun, A., Ramli, F. M., Kamarudin, M. L., Zakaria, A., Shakaff, A. Y. M., … Farook, R. S. (2014). Wireless sensor network coverage measurement and planning in mixed crop farming. Computers and Electronics in Agriculture, 105, 83-94. doi:10.1016/j.compag.2014.04.01

3R: a reliable multi-agent reinforcement learning based routing protocol for wireless medical sensor networks.

Interest in the Wireless Medical Sensor Network (WMSN) is rapidly gaining attention thanks to recent advances in semiconductors and wireless communication. However, by virtue of the sensitive medical applications and the stringent resource constraints, there is a need to develop a routing protocol to fulfill WMSN requirements in terms of delivery reliability, attack resiliency, computational overhead and energy efficiency. This paper proposes 3R, a reliable multi-agent reinforcement learning routing protocol for WMSN. 3R uses a novel resource-conservative Reinforcement Learning (RL) model to reduce the computational overhead, along with two updating methods to speed up the algorithm convergence. The reward function is re-defined as a punishment, combining the proposed trust management system to defend against well-known dropping attacks. Furthermore, an energy model is integrated with the reward function to enhance the network lifetime and balance energy consumption across the network. The proposed energy model uses only local information to avoid the resource burdens and the security concerns of exchanging energy information. Experimental results prove the lightweightness, attacks resiliency and energy efficiency of 3R, making it a potential routing candidate for WMSN

Energy-aware peering routing protocol for indoor hospital body area network communication

The recent research in Body Area Networks (BAN) is focused on making its communication more reliable, energy efficient, secure, and to better utilize system resources. In this paper we propose a novel BAN network architecture for indoor hospital environments, and a new mechanism of peer discovery with routing table construction that helps to reduce network traffic load, energy consumption, and improves BAN reliability. We have performed extensive simulations in the Castalia simulation environment to show that our proposed protocol has better performance in terms of reduced BAN traffic load, increased number of successful packets received by nodes, reduced number of packets forwarded by intermediate nodes, and overall lower energy consumption compared to other protocols

A reliable trust-aware reinforcement learning based routing protocol for wireless medical sensor networks.

Interest in the Wireless Medical Sensor Network (WMSN) is rapidly gaining attention thanks to recent advances in semiconductors and wireless communication. However, by virtue of the sensitive medical applications and the stringent resource constraints, there is a need to develop a routing protocol to fulfill WMSN requirements in terms of delivery reliability, attack resiliency, computational overhead and energy efficiency. This doctoral research therefore aims to advance the state of the art in routing by proposing a lightweight, reliable routing protocol for WMSN. Ensuring a reliable path between the source and the destination requires making trustaware routing decisions to avoid untrustworthy paths. A lightweight and effective Trust Management System (TMS) has been developed to evaluate the trust relationship between the sensor nodes with a view to differentiating between trustworthy nodes and untrustworthy ones. Moreover, a resource-conservative Reinforcement Learning (RL) model has been proposed to reduce the computational overhead, along with two updating methods to speed up the algorithm convergence. The reward function is re-defined as a punishment, combining the proposed trust management system to defend against well-known dropping attacks. Furthermore, with a view to addressing the inborn overestimation problem in Q-learning-based routing protocols, we adopted double Q-learning to overcome the positive bias of using a single estimator. An energy model is integrated with the reward function to enhance the network lifetime and balance energy consumption across the network. The proposed energy model uses only local information to avoid the resource burdens and the security concerns of exchanging energy information. Finally, a realistic trust management testbed has been developed to overcome the limitations of using numerical analysis to evaluate proposed trust management schemes, particularly in the context of WMSN. The proposed testbed has been developed as an additional module to the NS-3 simulator to fulfill usability, generalisability, flexibility, scalability and high-performance requirements

RRP: a reliable reinforcement learning based routing protocol for wireless medical sensor networks.

Wireless medical sensor networks (WMSNs) offer innovative healthcare applications that improve patients' quality of life, provide timely monitoring tools for physicians, and support national healthcare systems. However, despite these benefits, widespread adoption of WMSN advancements is still hampered by security concerns and limitations of routing protocols. Routing in WMSNs is a challenging task due to the fact that some WMSN requirements are overlooked by existing routing proposals. To overcome these challenges, this paper proposes a reliable multi-agent reinforcement learning based routing protocol (RRP). RRP is a lightweight attacks-resistant routing protocol designed to meet the unique requirements of WMSN. It uses a novel Q-learning model to reduce resource consumption combined with an effective trust management system to defend against various packet-dropping attacks. Experimental results prove the lightweightness of RRP and its robustness against blackhole, selective forwarding, sinkhole and complicated on-off attacks