Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Health-aware predictive control schemes based on industrial processes

Aplicat embargament des de la data de defensa fins el dia 30 de desembre de 2021The research is motivated by real applications, such as pasteurization plant, water networks and autonomous system, which each of them require a specific control system to provide proper management able to take into account their particular features and operating limits in presence of uncertainties related to their operation and failures from component breakdowns. According to that most of the real systems have nonlinear behaviors, it can be approximated them by polytopic linear uncertain models such as Linear Parameter Varying (LPV) and Takagi-Sugeno (TS) models. Therefore, a new economic Model Predictive Control (MPC) approach based on LPV/TS models is proposed and the stability of the proposed approach is certified by using a region constraint on the terminal state. Besides, the MPC-LPV strategy is extended based on the system with varying delays affecting states and inputs. The control approach allows the controller to accommodate the scheduling parameters and delay change. By computing the prediction of the state variables and delay along a prediction time horizon, the system model can be modified according to the evaluation of the estimated state and delay at each time instant. To increase the system reliability, anticipate the appearance of faults and reduce the operational costs, actuator health monitoring should be considered. Regarding several types of system failures, different strategies are studied for obtaining system failures. First, the damage is assessed with the rainflow-counting algorithm that allows estimating the component’s fatigue and control objective is modified by adding an extra criterion that takes into account the accumulated damage. Besides, two different health-aware economic predictive control strategies that aim to minimize the damage of components are presented. Then, economic health-aware MPC controller is developed to compute the components and system reliability in the MPC model using an LPV modeling approach and maximizes the availability of the system by estimating system reliability. Additionally, another improvement considers chance-constraint programming to compute an optimal list replenishment policy based on a desired risk acceptability level, managing to dynamically designate safety stocks in flowbased networks to satisfy non-stationary flow demands. Finally, an innovative health-aware control approach for autonomous racing vehicles to simultaneously control it to the driving limits and to follow the desired path based on maximization of the battery RUL. The proposed approach is formulated as an optimal on-line robust LMI based MPC driven from Lyapunov stability and controller gain synthesis solved by LPV-LQR problem in LMI formulation with integral action for tracking the trajectory.Esta tesis pretende proporcionar contribuciones teóricas y prácticas sobre seguridad y control de sistemas industriales, especialmente en la forma maten ática de sistemas inciertos. La investigación está motivada por aplicaciones reales, como la planta de pasteurización, las redes de agua y el sistema autónomo, cada uno de los cuales requiere un sistema de control específico para proporcionar una gestión adecuada capaz de tener en cuenta sus características particulares y limites o de operación en presencia de incertidumbres relacionadas con su operación y fallas de averías de componentes. De acuerdo con que la mayoría de los sistemas reales tienen comportamientos no lineales, puede aproximarse a ellos mediante modelos inciertos lineales politopicos como los modelos de Lineal Variación de Parámetros (LPV) y Takagi-Sugeno (TS). Por lo tanto, se propone un nuevo enfoque de Control Predictivo del Modelo (MPC) económico basado en modelos LPV/TS y la estabilidad del enfoque propuesto se certifica mediante el uso de una restricción de región en el estado terminal. Además, la estrategia MPC-LPV se extiende en función del sistema con diferentes demoras que afectan los estados y las entradas. El enfoque de control permite al controlador acomodar los parámetros de programación y retrasar el cambio. Al calcular la predicción de las variables de estado y el retraso a lo largo de un horizonte de tiempo de predicción, el modelo del sistema se puede modificar de acuerdo con la evaluación del estado estimado y el retraso en cada instante de tiempo. Para aumentar la confiabilidad del sistema, anticipar la aparición de fallas y reducir los costos operativos, se debe considerar el monitoreo del estado del actuador. Con respecto a varios tipos de fallas del sistema, se estudian diferentes estrategias para obtener fallas del sistema. Primero, el daño se evalúa con el algoritmo de conteo de flujo de lluvia que permite estimar la fatiga del componente y el objetivo de control se modifica agregando un criterio adicional que tiene en cuenta el daño acumulado. Además, se presentan dos estrategias diferentes de control predictivo económico que tienen en cuenta la salud y tienen como objetivo minimizar el daño de los componentes. Luego, se desarrolla un controlador MPC económico con conciencia de salud para calcular los componentes y la confiabilidad del sistema en el modelo MPC utilizando un enfoque de modelado LPV y maximiza la disponibilidad del sistema mediante la estimación de la confiabilidad del sistema. Además, otra mejora considera la programación de restricción de posibilidades para calcular una política ´optima de reposición de listas basada en un nivel de aceptabilidad de riesgo deseado, logrando designar dinámicamente existencias de seguridad en redes basadas en flujo para satisfacer demandas de flujo no estacionarias. Finalmente, un enfoque innovador de control consciente de la salud para vehículos de carreras autónomos para controlarlo simultáneamente hasta los límites de conducción y seguir el camino deseado basado en la maximización de la bacteria RUL. El diseño del control se divide en dos capas con diferentes escalas de tiempo, planificador de ruta y controlador. El enfoque propuesto está formulado como un MPC robusto en línea optimo basado en LMI impulsado por la estabilidad de Lyapunov y la síntesis de ganancia del controlador resuelta por el problema LPV-LQR en la formulación de LMI con acción integral para el seguimiento de la trayectoria.Postprint (published version

The Wireless Body Area Sensor Networks and Routing Strategies: Nomenclature and Review of Literature

WBASN is an effective solution that has been proposed in terms of improving the solutions and there are varied benefits that have been achieved from the usage of WBASN solutions in communication, healthcare domain. From the review of stats on rising number of wireless devices and solutions that are coming up which is embraced by the people as wearable devices, implants for medical diagnostic solutions, etc. reflect upon the growing demand for effective models. However, the challenge is about effective performance of such solutions with optimal efficiency. Due to certain intrinsic factors like numerous standards that are available, and also due to the necessity for identifying the best solutions that are based on application requirements. Some of the key issues that have to be considered in the process of WBASN are about the impacts that are taking place from the wireless medium, the lifetime of batteries in the WBASN devices and the other significant condition like the coexistence of the systems among varied other wireless networks that are constituted in the proximity. In this study, scores of models that has been proposed pertaining to MAC protocols for WBASN solutions has been reviewed to understand the efficacy of the existing systems, and a scope for process improvement has been explored for conducting in detail research and developing a solution

Reliable on-demand routing protocols for mobile ad-hoc networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University London.Mobile Ad-Hoc Network (MANET) facilitates the creation of dynamic reconfigurable networks, without centralized infrastructure. MANET routing protocols have to face high challenges like link instability, node mobility, frequently changing topologies and energy consumption of node, due to these challenges routing becomes one of the core issues in MANETs. This Thesis mainly focuses on the reactive routing protocol such as Ad-Hoc On-Demand Distance Vector (AODV) routing protocol. Reliable and Efficient Reactive Routing Protocol (RERRP) for MANET has been proposed to reduce the link breakages between the moving nodes. This scheme selects a reliable route using Reliability Factor (RF); the RF considers Route Expiration Time and Hop Count to select a routing path with high reliability and have less number of hops. The simulation result shows that RERRP outperforms AODV and enhance the packet delivery fraction (PDF) by around 6% and reduces the network routing load (NRL) by around 30%. Broadcasting in MANET could cause serious redundancy, contention, and collision of the packets. A scheme, Effective Broadcast Control Routing Protocol (EBCRP) has been proposed for the controlling of broadcast storm problem in a MANET. The EBCRP is mainly selects the reliable node while controlling the redundant re-broadcast of the route request packet. The proposed algorithm EBCRP is an on-demand routing protocol, therefore AODV route discovery mechanism was selected as the base of this scheme. The analysis of the performance of EBCRP has revealed that the EBCRP have controlled the routing overhead significantly, reduces it around 70% and enhance the packet delivery by 13% as compared to AODV. An Energy Sensible and Route Stability Based Routing Protocol (ESRSBR) have also been proposed that mainly focuses on increasing the network lifetime with better packet delivery. The ESRSBR supports those nodes to participate in the data transfer that have more residual energy related to their neighbour nodes. The proposed protocol also keeps track of the stability of the links between the nodes. Finally, the ESRSBR selects those routes which consist of nodes that have more residual energy and have stable links. The comparative analysis of ESRSBR with AODV and recently proposed routing protocol called Link Stability and Energy Aware (LSEA) routing protocol revealed that the proposed protocol ESRSBR has a significantly affect the network lifetime, increases it around 10% and 13% as compared to LSEA and AODV protocols respectively. The ESRSBR also decreases the routing overhead by 22% over LSEA and by 38% over AODV.Higher Education Commission of Pakistan and NED University of Engineering and Technology Karach

Energy Efficient Neocortex-Inspired Systems with On-Device Learning

Shifting the compute workloads from cloud toward edge devices can significantly improve the overall latency for inference and learning. On the contrary this paradigm shift exacerbates the resource constraints on the edge devices. Neuromorphic computing architectures, inspired by the neural processes, are natural substrates for edge devices. They offer co-located memory, in-situ training, energy efficiency, high memory density, and compute capacity in a small form factor. Owing to these features, in the recent past, there has been a rapid proliferation of hybrid CMOS/Memristor neuromorphic computing systems. However, most of these systems offer limited plasticity, target either spatial or temporal input streams, and are not demonstrated on large scale heterogeneous tasks. There is a critical knowledge gap in designing scalable neuromorphic systems that can support hybrid plasticity for spatio-temporal input streams on edge devices. This research proposes Pyragrid, a low latency and energy efficient neuromorphic computing system for processing spatio-temporal information natively on the edge. Pyragrid is a full-scale custom hybrid CMOS/Memristor architecture with analog computational modules and an underlying digital communication scheme. Pyragrid is designed for hierarchical temporal memory, a biomimetic sequence memory algorithm inspired by the neocortex. It features a novel synthetic synapses representation that enables dynamic synaptic pathways with reduced memory usage and interconnects. The dynamic growth in the synaptic pathways is emulated in the memristor device physical behavior, while the synaptic modulation is enabled through a custom training scheme optimized for area and power. Pyragrid features data reuse, in-memory computing, and event-driven sparse local computing to reduce data movement by ~44x and maximize system throughput and power efficiency by ~3x and ~161x over custom CMOS digital design. The innate sparsity in Pyragrid results in overall robustness to noise and device failure, particularly when processing visual input and predicting time series sequences. Porting the proposed system on edge devices can enhance their computational capability, response time, and battery life

On the Support of Massive Machine-to-Machine Traffic in Heterogeneous Networks and Fifth-Generation Cellular Networks

The widespread availability of many emerging services enabled by the Internet of Things (IoT) paradigm passes through the capability to provide long-range connectivity to a massive number of things, overcoming the well-known issues of ad-hoc, short-range networks. This scenario entails a lot of challenges, ranging from the concerns about the radio access network efficiency to the threats about the security of IoT networks. In this thesis, we will focus on wireless communication standards for long-range IoT as well as on fundamental research outcomes about IoT networks. After investigating how Machine-Type Communication (MTC) is supported nowadays, we will provide innovative solutions that i) satisfy the requirements in terms of scalability and latency, ii) employ a combination of licensed and license-free frequency bands, and iii) assure energy-efficiency and security

On Enhancing Security of Password-Based Authentication

Password has been the dominant authentication scheme for more than 30 years, and it will not be easily replaced in the foreseeable future. However, password authentication has long been plagued by the dilemma between security and usability, mainly due to human memory limitations. For example, a user often chooses an easy-to-guess (weak) password since it is easier to remember. The ever increasing number of online accounts per user even exacerbates this problem. In this dissertation, we present four research projects that focus on the security of password authentication and its ecosystem. First, we observe that personal information plays a very important role when a user creates a password. Enlightened by this, we conduct a study on how users create their passwords using their personal information based on a leaked password dataset. We create a new metric---Coverage---to quantify the personal information in passwords. Armed with this knowledge, we develop a novel password cracker named Personal-PCFG (Probabilistic Context-Free Grammars) that leverages personal information for targeted password guessing. Experiments show that Personal-PCFG is much more efficient than the original PCFG in cracking passwords. The second project aims to ease the password management hassle for a user. Password managers are introduced so that users need only one password (master password) to access all their other passwords. However, the password manager induces a single point of failure and is potentially vulnerable to data breach. To address these issues, we propose BluePass, a decentralized password manager that features a dual-possession security that involves a master password and a mobile device. In addition, BluePass enables a hand-free user experience by retrieving passwords from the mobile device through Bluetooth communications. In the third project, we investigate an overlooked aspect in the password lifecycle, the password recovery procedure. We study the password recovery protocols in the Alexa top 500 websites, and report interesting findings on the de facto implementation. We observe that the backup email is the primary way for password recovery, and the email becomes a single point of failure. We assess the likelihood of an account recovery attack, analyze the security policy of major email providers, and propose a security enhancement protocol to help securing password recovery emails by two factor authentication. \newline Finally, we focus on a more fundamental level, user identity. Password-based authentication is just a one-time checking to ensure that a user is legitimate. However, a user\u27s identity could be hijacked at any step. For example, an attacker can leverage a zero-day vulnerability to take over the root privilege. Thus, tracking the user behavior is essential to examine the identity legitimacy. We develop a user tracking system based on OS-level logs inside an enterprise network, and apply a variety of techniques to generate a concise and salient user profile for identity examination

Mobile Ad-Hoc Networks

Being infrastructure-less and without central administration control, wireless ad-hoc networking is playing a more and more important role in extending the coverage of traditional wireless infrastructure (cellular networks, wireless LAN, etc). This book includes state-of-the-art techniques and solutions for wireless ad-hoc networks. It focuses on the following topics in ad-hoc networks: quality-of-service and video communication, routing protocol and cross-layer design. A few interesting problems about security and delay-tolerant networks are also discussed. This book is targeted to provide network engineers and researchers with design guidelines for large scale wireless ad hoc networks