Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Adoption of vehicular ad hoc networking protocols by networked robots

This paper focuses on the utilization of wireless networking in the robotics domain. Many researchers have already equipped their robots with wireless communication capabilities, stimulated by the observation that multi-robot systems tend to have several advantages over their single-robot counterparts. Typically, this integration of wireless communication is tackled in a quite pragmatic manner, only a few authors presented novel Robotic Ad Hoc Network (RANET) protocols that were designed specifically with robotic use cases in mind. This is in sharp contrast with the domain of vehicular ad hoc networks (VANET). This observation is the starting point of this paper. If the results of previous efforts focusing on VANET protocols could be reused in the RANET domain, this could lead to rapid progress in the field of networked robots. To investigate this possibility, this paper provides a thorough overview of the related work in the domain of robotic and vehicular ad hoc networks. Based on this information, an exhaustive list of requirements is defined for both types. It is concluded that the most significant difference lies in the fact that VANET protocols are oriented towards low throughput messaging, while RANET protocols have to support high throughput media streaming as well. Although not always with equal importance, all other defined requirements are valid for both protocols. This leads to the conclusion that cross-fertilization between them is an appealing approach for future RANET research. To support such developments, this paper concludes with the definition of an appropriate working plan

Persistent Localized Broadcasting in VANETs

We present a communication protocol, called LINGER, for persistent dissemination of delay-tolerant information to vehicular users, within a geographical area of interest. The goal of LINGER is to dispatch and confine information in localized areas of a mobile network with minimal protocol overhead and without requiring knowledge of the vehicles' routes or destinations. LINGER does not require roadside infrastructure support: it selects mobile nodes in a distributed, cooperative way and lets them act as "information bearers", providing uninterrupted information availability within a desired region. We analyze the performance of our dissemination mechanism through extensive simulations, in complex vehicular scenarios with realistic node mobility. The results demonstrate that LINGER represents a viable, appealing alternative to infrastructure-based solutions, as it can successfully drive the information toward a region of interest from a far away source and keep it local with negligible overhead. We show the effectiveness of such an approach in the support of localized broadcasting, in terms of both percentage of informed vehicles and information delivery delay, and we compare its performance to that of a dedicated, state-of-the-art protoco

Distributed Adaptation Techniques for Connected Vehicles

In this PhD dissertation, we propose distributed adaptation mechanisms for connected vehicles to deal with the connectivity challenges. To understand the system behavior of the solutions for connected vehicles, we first need to characterize the operational environment. Therefore, we devised a large scale fading model for various link types, including point-to-point vehicular communications and multi-hop connected vehicles. We explored two small scale fading models to define the characteristics of multi-hop connected vehicles. Taking our research into multi-hop connected vehicles one step further, we propose selective information relaying to avoid message congestion due to redundant messages received by the relay vehicle. Results show that the proposed mechanism reduces messaging load by up to 75% without sacrificing environmental awareness. Once we define the channel characteristics, we propose a distributed congestion control algorithm to solve the messaging overhead on the channels as the next research interest of this dissertation. We propose a combined transmit power and message rate adaptation for connected vehicles. The proposed algorithm increases the environmental awareness and achieves the application requirements by considering highly dynamic network characteristics. Both power and rate adaptation mechanisms are performed jointly to avoid one result affecting the other negatively. Results prove that the proposed algorithm can increase awareness by 20% while keeping the channel load and interference at almost the same level as well as improve the average message rate by 18%. As the last step of this dissertation, distributed cooperative dynamic spectrum access technique is proposed to solve the channel overhead and the limited resources issues. The adaptive energy detection threshold, which is used to decide whether the channel is busy, is optimized in this work by using a computationally efficient numerical approach. Each vehicle evaluates the available channels by voting on the information received from one-hop neighbors. An interdisciplinary approach referred to as entropy-based weighting is used for defining the neighbor credibility. Once the vehicle accesses the channel, we propose a decision mechanism for channel switching that is inspired by the optimal flower selection process employed by bumblebees foraging. Experimental results show that by using the proposed distributed cooperative spectrum sensing mechanism, spectrum detection error converges to zero

TDMP-Reliable Target Driven and Mobility Prediction based Routing Protocol in Complex VANET

Vehicle-to-everything (V2X) communication in the vehicular ad hoc network (VANET), an infrastructure-free mechanism, has emerged as a crucial component in the advanced Intelligent Transport System (ITS) for special information transmission and inter-vehicular communications. One of the main research challenges in VANET is the design and implementation of network routing protocols which manage to trigger V2X communication with the reliable end-to-end connectivity and efficient packet transmission. The organically changing nature of road transport vehicles poses a significant threat to VANET with respect to the accuracy and reliability of packet delivery. Therefore, a position-based routing protocol tends to be the predominant method in VANET as they overcome rapid changes in vehicle movements effectively. However, existing routing protocols have some limitations such as (i) inaccurate in high dynamic network topology, (ii) defective link-state estimation (iii) poor movement prediction in heterogeneous road layouts. In this paper, a target-driven and mobility prediction (TDMP) based routing protocol is therefore developed for high-speed mobility and dynamic topology of vehicles, fluctuant traffic flow and diverse road layouts in VANET. The primary idea in TDMP is that the destination target of a driver is included in the mobility prediction to assist the implementation of the routing protocol. Compared to existing geographic routing protocols which mainly greedily forward the packet to the next-hop based on its current position and partial road layout, TDMP is developed to enhance the packet transmission with the consideration of the estimation of inter-vehicles link status, and the prediction of vehicle positions dynamically in fluctuant mobility and global road layout.Comment: 35 pages,16 Figure

Quality of service aware data dissemination in vehicular Ad Hoc networks

Des systèmes de transport intelligents (STI) seront éventuellement fournis dans un proche avenir pour la sécurité et le confort des personnes lors de leurs déplacements sur les routes. Les réseaux ad-hoc véhiculaires (VANETs) représentent l'élément clé des STI. Les VANETs sont formés par des véhicules qui communiquent entre eux et avec l'infrastructure. En effet, les véhicules pourront échanger des messages qui comprennent, par exemple, des informations sur la circulation routière, les situations d'urgence et les divertissements. En particulier, les messages d'urgence sont diffusés par des véhicules en cas d'urgence (p.ex. un accident de voiture); afin de permettre aux conducteurs de réagir à temps (p.ex., ralentir), les messages d'urgence doivent être diffusés de manière fiable dans un délai très court. Dans les VANETs, il existe plusieurs facteurs, tels que le canal à pertes, les terminaux cachés, les interférences et la bande passante limitée, qui compliquent énormément la satisfaction des exigences de fiabilité et de délai des messages d'urgence. Dans cette thèse, en guise de première contribution, nous proposons un schéma de diffusion efficace à plusieurs sauts, appelé Dynamic Partitioning Scheme (DPS), pour diffuser les messages d'urgence. DPS calcule les tailles de partitions dynamiques et le calendrier de transmission pour chaque partition; à l'intérieur de la zone arrière de l'expéditeur, les partitions sont calculées de sorte qu'en moyenne chaque partition contient au moins un seul véhicule; l'objectif est de s'assurer que seul un véhicule dans la partition la plus éloignée (de l'expéditeur) est utilisé pour diffuser le message, jusqu'au saut suivant; ceci donne lieu à un délai d'un saut plus court. DPS assure une diffusion rapide des messages d'urgence. En outre, un nouveau mécanisme d'établissement de liaison, qui utilise des tonalités occupées, est proposé pour résoudre le problème du problème de terminal caché. Dans les VANETs, la Multidiffusion, c'est-à-dire la transmission d'un message d'une source à un nombre limité de véhicules connus en tant que destinations, est très importante. Par rapport à la diffusion unique, avec Multidiffusion, la source peut simultanément prendre en charge plusieurs destinations, via une arborescence de multidiffusion, ce qui permet d'économiser de la bande passante et de réduire la congestion du réseau. Cependant, puisque les VANETs ont une topologie dynamique, le maintien de la connectivité de l'arbre de multidiffusion est un problème majeur. Comme deuxième contribution, nous proposons deux approches pour modéliser l'utilisation totale de bande passante d'une arborescence de multidiffusion: (i) la première approche considère le nombre de segments de route impliqués dans l'arbre de multidiffusion et (ii) la seconde approche considère le nombre d'intersections relais dans l'arbre de multidiffusion. Une heuristique est proposée pour chaque approche. Pour assurer la qualité de service de l'arbre de multidiffusion, des procédures efficaces sont proposées pour le suivi des destinations et la surveillance de la qualité de service des segments de route. Comme troisième contribution, nous étudions le problème de la congestion causée par le routage du trafic de données dans les VANETs. Nous proposons (1) une approche de routage basée sur l’infonuagique qui, contrairement aux approches existantes, prend en compte les chemins de routage existants qui relaient déjà les données dans les VANETs. Les nouvelles demandes de routage sont traitées de sorte qu'aucun segment de route ne soit surchargé par plusieurs chemins de routage croisés. Au lieu d'acheminer les données en utilisant des chemins de routage sur un nombre limité de segments de route, notre approche équilibre la charge des données en utilisant des chemins de routage sur l'ensemble des tronçons routiers urbains, dans le but d'empêcher, dans la mesure du possible, les congestions locales dans les VANETs; et (2) une approche basée sur le réseau défini par logiciel (SDN) pour surveiller la connectivité VANET en temps réel et les délais de transmission sur chaque segment de route. Les données de surveillance sont utilisées en entrée de l'approche de routage.Intelligent Transportation Systems (ITS) will be eventually provided in the near future for both safety and comfort of people during their travel on the roads. Vehicular ad-hoc Networks (VANETs), represent the key component of ITS. VANETs consist of vehicles that communicate with each other and with the infrastructure. Indeed, vehicles will be able to exchange messages that include, for example, information about road traffic, emergency situations, and entertainment. Particularly, emergency messages are broadcasted by vehicles in case of an emergency (e.g., car accident); in order to allow drivers to react in time (e.g., slow down), emergency messages must be reliably disseminated with very short delay. In VANETs, there are several factors, such as lossy channel, hidden terminals, interferences and scarce bandwidth, which make satisfying reliability and delay requirements of emergency messages very challenging. In this thesis, as the first contribution, we propose a reliable time-efficient and multi-hop broadcasting scheme, called Dynamic Partitioning Scheme (DPS), to disseminate emergency messages. DPS computes dynamic partition sizes and the transmission schedule for each partition; inside the back area of the sender, the partitions are computed such that in average each partition contains at least a single vehicle; the objective is to ensure that only a vehicle in the farthest partition (from the sender) is used to disseminate the message, to next hop, resulting in shorter one hop delay. DPS ensures fast dissemination of emergency messages. Moreover, a new handshaking mechanism, that uses busy tones, is proposed to solve the problem of hidden terminal problem. In VANETs, Multicasting, i.e. delivering a message from a source to a limited known number of vehicles as destinations, is very important. Compared to Unicasting, with Multicasting, the source can simultaneously support multiple destinations, via a multicast tree, saving bandwidth and reducing overall communication congestion. However, since VANETs have a dynamic topology, maintaining the connectivity of the multicast tree is a major issue. As the second contribution, we propose two approaches to model total bandwidth usage of a multicast tree: (i) the first approach considers the number of road segments involved in the multicast tree and (ii) the second approach considers the number of relaying intersections involved in the multicast tree. A heuristic is proposed for each approach. To ensure QoS of the multicasting tree, efficient procedures are proposed for tracking destinations and monitoring QoS of road segments. As the third contribution, we study the problem of network congestion in routing data traffic in VANETs. We propose (1) a Cloud-based routing approach that, in opposition to existing approaches, takes into account existing routing paths which are already relaying data in VANETs. New routing requests are processed such that no road segment gets overloaded by multiple crossing routing paths. Instead of routing over a limited set of road segments, our approach balances the load of communication paths over the whole urban road segments, with the objective to prevent, whenever possible, local congestions in VANETs; and (2) a Software Defined Networking (SDN) based approach to monitor real-time VANETs connectivity and transmission delays on each road segment. The monitoring data is used as input to the routing approach