A reliable trust-aware reinforcement learning based routing protocol for wireless medical sensor networks.

Interest in the Wireless Medical Sensor Network (WMSN) is rapidly gaining attention thanks to recent advances in semiconductors and wireless communication. However, by virtue of the sensitive medical applications and the stringent resource constraints, there is a need to develop a routing protocol to fulfill WMSN requirements in terms of delivery reliability, attack resiliency, computational overhead and energy efficiency. This doctoral research therefore aims to advance the state of the art in routing by proposing a lightweight, reliable routing protocol for WMSN. Ensuring a reliable path between the source and the destination requires making trustaware routing decisions to avoid untrustworthy paths. A lightweight and effective Trust Management System (TMS) has been developed to evaluate the trust relationship between the sensor nodes with a view to differentiating between trustworthy nodes and untrustworthy ones. Moreover, a resource-conservative Reinforcement Learning (RL) model has been proposed to reduce the computational overhead, along with two updating methods to speed up the algorithm convergence. The reward function is re-defined as a punishment, combining the proposed trust management system to defend against well-known dropping attacks. Furthermore, with a view to addressing the inborn overestimation problem in Q-learning-based routing protocols, we adopted double Q-learning to overcome the positive bias of using a single estimator. An energy model is integrated with the reward function to enhance the network lifetime and balance energy consumption across the network. The proposed energy model uses only local information to avoid the resource burdens and the security concerns of exchanging energy information. Finally, a realistic trust management testbed has been developed to overcome the limitations of using numerical analysis to evaluate proposed trust management schemes, particularly in the context of WMSN. The proposed testbed has been developed as an additional module to the NS-3 simulator to fulfill usability, generalisability, flexibility, scalability and high-performance requirements

3R: a reliable multi-agent reinforcement learning based routing protocol for wireless medical sensor networks.

Interest in the Wireless Medical Sensor Network (WMSN) is rapidly gaining attention thanks to recent advances in semiconductors and wireless communication. However, by virtue of the sensitive medical applications and the stringent resource constraints, there is a need to develop a routing protocol to fulfill WMSN requirements in terms of delivery reliability, attack resiliency, computational overhead and energy efficiency. This paper proposes 3R, a reliable multi-agent reinforcement learning routing protocol for WMSN. 3R uses a novel resource-conservative Reinforcement Learning (RL) model to reduce the computational overhead, along with two updating methods to speed up the algorithm convergence. The reward function is re-defined as a punishment, combining the proposed trust management system to defend against well-known dropping attacks. Furthermore, an energy model is integrated with the reward function to enhance the network lifetime and balance energy consumption across the network. The proposed energy model uses only local information to avoid the resource burdens and the security concerns of exchanging energy information. Experimental results prove the lightweightness, attacks resiliency and energy efficiency of 3R, making it a potential routing candidate for WMSN

RRP: a reliable reinforcement learning based routing protocol for wireless medical sensor networks.

Wireless medical sensor networks (WMSNs) offer innovative healthcare applications that improve patients' quality of life, provide timely monitoring tools for physicians, and support national healthcare systems. However, despite these benefits, widespread adoption of WMSN advancements is still hampered by security concerns and limitations of routing protocols. Routing in WMSNs is a challenging task due to the fact that some WMSN requirements are overlooked by existing routing proposals. To overcome these challenges, this paper proposes a reliable multi-agent reinforcement learning based routing protocol (RRP). RRP is a lightweight attacks-resistant routing protocol designed to meet the unique requirements of WMSN. It uses a novel Q-learning model to reduce resource consumption combined with an effective trust management system to defend against various packet-dropping attacks. Experimental results prove the lightweightness of RRP and its robustness against blackhole, selective forwarding, sinkhole and complicated on-off attacks

Performance evaluation of cooperation strategies for m-health services and applications

Health telematics are becoming a major improvement for patients’ lives, especially for disabled, elderly, and chronically ill people. Information and communication technologies have rapidly grown along with the mobile Internet concept of anywhere and anytime connection. In this context, Mobile Health (m-Health) proposes healthcare services delivering, overcoming geographical, temporal and even organizational barriers. Pervasive and m-Health services aim to respond several emerging problems in health services, including the increasing number of chronic diseases related to lifestyle, high costs in existing national health services, the need to empower patients and families to self-care and manage their own healthcare, and the need to provide direct access to health services, regardless the time and place. Mobile Health (m- Health) systems include the use of mobile devices and applications that interact with patients and caretakers. However, mobile devices have several constraints (such as, processor, energy, and storage resource limitations), affecting the quality of service and user experience. Architectures based on mobile devices and wireless communications presents several challenged issues and constraints, such as, battery and storage capacity, broadcast constraints, interferences, disconnections, noises, limited bandwidths, and network delays. In this sense, cooperation-based approaches are presented as a solution to solve such limitations, focusing on increasing network connectivity, communication rates, and reliability. Cooperation is an important research topic that has been growing in recent years. With the advent of wireless networks, several recent studies present cooperation mechanisms and algorithms as a solution to improve wireless networks performance. In the absence of a stable network infrastructure, mobile nodes cooperate with each other performing all networking functionalities. For example, it can support intermediate nodes forwarding packets between two distant nodes. This Thesis proposes a novel cooperation strategy for m-Health services and applications. This reputation-based scheme uses a Web-service to handle all the nodes reputation and networking permissions. Its main goal is to provide Internet services to mobile devices without network connectivity through cooperation with neighbor devices. Therefore resolving the above mentioned network problems and resulting in a major improvement for m-Health network architectures performances. A performance evaluation of this proposal through a real network scenario demonstrating and validating this cooperative scheme using a real m-Health application is presented. A cryptography solution for m-Health applications under cooperative environments, called DE4MHA, is also proposed and evaluated using the same real network scenario and the same m-Health application. Finally, this work proposes, a generalized cooperative application framework, called MobiCoop, that extends the incentive-based cooperative scheme for m-Health applications for all mobile applications. Its performance evaluation is also presented through a real network scenario demonstrating and validating MobiCoop using different mobile applications

The Internet of Everything

In the era before IoT, the world wide web, internet, web 2.0 and social media made people’s lives comfortable by providing web services and enabling access personal data irrespective of their location. Further, to save time and improve efficiency, there is a need for machine to machine communication, automation, smart computing and ubiquitous access to personal devices. This need gave birth to the phenomenon of Internet of Things (IoT) and further to the concept of Internet of Everything (IoE)

A patient agent controlled customized blockchain based framework for internet of things

Although Blockchain implementations have emerged as revolutionary technologies for various industrial applications including cryptocurrencies, they have not been widely deployed to store data streaming from sensors to remote servers in architectures known as Internet of Things. New Blockchain for the Internet of Things models promise secure solutions for eHealth, smart cities, and other applications. These models pave the way for continuous monitoring of patient’s physiological signs with wearable sensors to augment traditional medical practice without recourse to storing data with a trusted authority. However, existing Blockchain algorithms cannot accommodate the huge volumes, security, and privacy requirements of health data. In this thesis, our first contribution is an End-to-End secure eHealth architecture that introduces an intelligent Patient Centric Agent. The Patient Centric Agent executing on dedicated hardware manages the storage and access of streams of sensors generated health data, into a customized Blockchain and other less secure repositories. As IoT devices cannot host Blockchain technology due to their limited memory, power, and computational resources, the Patient Centric Agent coordinates and communicates with a private customized Blockchain on behalf of the wearable devices. While the adoption of a Patient Centric Agent offers solutions for addressing continuous monitoring of patients’ health, dealing with storage, data privacy and network security issues, the architecture is vulnerable to Denial of Services(DoS) and single point of failure attacks. To address this issue, we advance a second contribution; a decentralised eHealth system in which the Patient Centric Agent is replicated at three levels: Sensing Layer, NEAR Processing Layer and FAR Processing Layer. The functionalities of the Patient Centric Agent are customized to manage the tasks of the three levels. Simulations confirm protection of the architecture against DoS attacks. Few patients require all their health data to be stored in Blockchain repositories but instead need to select an appropriate storage medium for each chunk of data by matching their personal needs and preferences with features of candidate storage mediums. Motivated by this context, we advance third contribution; a recommendation model for health data storage that can accommodate patient preferences and make storage decisions rapidly, in real-time, even with streamed data. The mapping between health data features and characteristics of each repository is learned using machine learning. The Blockchain’s capacity to make transactions and store records without central oversight enables its application for IoT networks outside health such as underwater IoT networks where the unattended nature of the nodes threatens their security and privacy. However, underwater IoT differs from ground IoT as acoustics signals are the communication media leading to high propagation delays, high error rates exacerbated by turbulent water currents. Our fourth contribution is a customized Blockchain leveraged framework with the model of Patient-Centric Agent renamed as Smart Agent for securely monitoring underwater IoT. Finally, the smart Agent has been investigated in developing an IoT smart home or cities monitoring framework. The key algorithms underpinning to each contribution have been implemented and analysed using simulators.Doctor of Philosoph

A Hybrid Reinforcement Learning-Based Trust Model for 5G Networks

Trust investigation in 5G, which is the next-generation wireless network, is still at its infancy. This research proposes a hybrid trust model for the selection of a legitimate (or trusted) forwarding (relay) entity and to countermeasure intelligent attacks against a route selection scheme. The hybrid trust model has a centralized entity (i.e., a centralized controller C c ) that provides the security level of the operating environment to network entities, and distributed entities that identify malicious or legitimate entities in the network. When the security level of the operating environment is high (low), the distributed entities learn more (lesser) from their respective operating environment. While legitimate entities can use artificial intelligence, such as reinforcement learning (RL), to enhance their trust models, the malicious entities can also use artificial intelligence to increase the detrimental effects of their attacks and minimize their likelihood of being detected. Our proposed trust model is feasible with the introduction of artificial intelligence and the central controller (C c ) in 5G to support the hybrid trust model. We have explained in detail our proposed model that reinforcement learning based hybridization of trust model can enhance the performance of the network interms of learning and tackling intelligent attacks, whereby achieving context awareness and detection of malicious entities

A COGNITIVE ARCHITECTURE FOR AMBIENT INTELLIGENCE

L’Ambient Intelligence (AmI) è caratterizzata dall’uso di sistemi pervasivi per monitorare l’ambiente e modificarlo secondo le esigenze degli utenti e rispettando vincoli definiti globalmente. Questi sistemi non possono prescindere da requisiti come la scalabilità e la trasparenza per l’utente. Una tecnologia che consente di raggiungere questi obiettivi è rappresentata dalle reti di sensori wireless (WSN), caratterizzate da bassi costi e bassa intrusività. Tuttavia, sebbene in grado di effettuare elaborazioni a bordo dei singoli nodi, le WSN non hanno da sole le capacità di elaborazione necessarie a supportare un sistema intelligente; d’altra parte senza questa attività di pre-elaborazione la mole di dati sensoriali può facilmente sopraffare un sistema centralizzato con un’eccessiva quantità di dettagli superflui. Questo lavoro presenta un’architettura cognitiva in grado di percepire e controllare l’ambiente di cui fa parte, basata su un nuovo approccio per l’estrazione di conoscenza a partire dai dati grezzi, attraverso livelli crescenti di astrazione. Le WSN sono utilizzate come strumento sensoriale pervasivo, le cui capacità computazionali vengono utilizzate per pre-elaborare i dati rilevati, in modo da consentire ad un sistema centralizzato intelligente di effettuare ragionamenti di alto livello. L’architettura proposta è stata utilizzata per sviluppare un testbed dotato degli strumenti hardware e software necessari allo sviluppo e alla gestione di applicazioni di AmI basate su WSN, il cui obiettivo principale sia il risparmio energetico. Per fare in modo che le applicazioni di AmI siano in grado di comunicare con il mondo esterno in maniera affidabile, per richiedere servizi ad agenti esterni, l’architettura è stata arricchita con un protocollo di gestione distribuita della reputazione. È stata inoltre sviluppata un’applicazione di esempio che sfrutta le caratteristiche del testbed, con l’obiettivo di controllare la temperatura in un ambiente lavorativo. Quest’applicazione rileva la presenza dell’utente attraverso un modulo per la fusione di dati multi-sensoriali basato su reti bayesiane, e sfrutta questa informazione in un controllore fuzzy multi-obiettivo che controlla gli attuatori sulla base delle preferenze dell’utente e del risparmio energetico.Ambient Intelligence (AmI) systems are characterized by the use of pervasive equipments for monitoring and modifying the environment according to users’ needs, and to globally defined constraints. Furthermore, such systems cannot ignore requirements about ubiquity, scalability, and transparency to the user. An enabling technology capable of accomplishing these goals is represented by Wireless Sensor Networks (WSNs), characterized by low-costs and unintrusiveness. However, although provided of in-network processing capabilities, WSNs do not exhibit processing features able to support comprehensive intelligent systems; on the other hand, without this pre-processing activities the wealth of sensory data may easily overwhelm a centralized AmI system, clogging it with superfluous details. This work proposes a cognitive architecture able to perceive, decide upon, and control the environment of which the system is part, based on a new approach to knowledge extraction from raw data, that addresses this issue at different abstraction levels. WSNs are used as the pervasive sensory tool, and their computational capabilities are exploited to remotely perform preliminary data processing. A central intelligent unit subsequently extracts higher-level concepts in order to carry on symbolic reasoning. The aim of the reasoning is to plan a sequence of actions that will lead the environment to a state as close as possible to the users’ desires, taking into account both implicit and explicit feedbacks from the users, while considering global system-driven goals, such as energy saving. The proposed conceptual architecture was exploited to develop a testbed providing the hardware and software tools for the development and management of AmI applications based on WSNs, whose main goal is energy saving for global sustainability. In order to make the AmI system able to communicate with the external world in a reliable way, when some services are required to external agents, the architecture was enriched with a distributed reputation management protocol. A sample application exploiting the testbed features was implemented for addressing temperature control in a work environment. Knowledge about the user’s presence is obtained through a multi-sensor data fusion module based on Bayesian networks, and this information is exploited by a multi-objective fuzzy controller that operates on actuators taking into account users’ preference and energy consumption constraints

6G—Enabling the New Smart City: A Survey

Smart cities and 6G are technological areas that have the potential to transform the way we live and work in the years to come. Until this transformation comes into place, there is the need, underlined by research and market studies, for a critical reassessment of the entire wireless communication sector for smart cities, which should include the IoT infrastructure, economic factors that could improve their adoption rate, and strategies that enable smart city operations. Therefore, from a technical point of view, a series of stringent issues, such as interoperability, data privacy, security, the digital divide, and implementation issues have to be addressed. Notably, to concentrate the scrutiny on smart cities and the forthcoming influence of 6G, the groundwork laid by the current 5G, with its multifaceted role and inherent limitations within the domain of smart cities, is embraced as a foundational standpoint. This examination culminates in a panoramic exposition, extending beyond the mere delineation of the 6G standard toward the unveiling of the extensive gamut of potential applications that this emergent standard promises to introduce to the smart cities arena. This paper provides an update on the SC ecosystem around the novel paradigm of 6G, aggregating a series of enabling technologies accompanied by the descriptions of their roles and specific employment schemes