Quality of Service Issues for Reinforcement Learning Based Routing Algorithm for Ad-Hoc Networks

Mobile ad-hoc networks are dynamic networks which are decentralized and autonomous in nature. Many routing algorithms have been proposed for these dynamic networks. It is an important problem to model Quality of Service requirements on these types of algorithms which traditionally have certain limitations. To model this scenario we have considered a reinforcement learning algorithm SAMPLE. SAMPLE promises to deal effectively with congestion and under high traffic load. As it is natural for ad-hoc networks to move in groups, we have considered the various group mobility models. The Pursue Mobility Model with its superiormobilitymetrics exhibits better performance. At the data link layer we have considered IEEE 802.11e, a MAC layer which has provisions to support QoS. As mobile ad-hoc networks are constrained by resources like energy and bandwidth, it is imperative for them to cooperate in a reasonably selfish manner. Thus, in this paper we propose cooperation with a moderately punishing algorithm based on game theory. The proposed algorithm in synchronization with SAMPLE yields better results on IEEE 802.11e

Study on QoS support in 802.11e-based multi-hop vehicular wireless ad hoc networks

Multimedia communications over vehicular ad hoc networks (VANET) will play an important role in the future intelligent transport system (ITS). QoS support for VANET therefore becomes an essential problem. In this paper, we first study the QoS performance in multi-hop VANET by using the standard IEEE 802.11e EDCA MAC and our proposed triple-constraint QoS routing protocol, Delay-Reliability-Hop (DeReHQ). In particular, we evaluate the DeReHQ protocol together with EDCA in highway and urban areas. Simulation results show that end-to-end delay performance can sometimes be achieved when both 802.11e EDCA and DeReHQ extended AODV are used. However, further studies on cross-layer optimization for QoS support in multi-hop environment are required

Multipath routing and QoS provisioning in mobile ad hoc networks

PhDA Mobile Ad Hoc Networks (MANET) is a collection of mobile nodes that can communicate with each other using multihop wireless links without utilizing any fixed based-station infrastructure and centralized management. Each mobile node in the network acts as both a host generating flows or being destination of flows and a router forwarding flows directed to other nodes. Future applications of MANETs are expected to be based on all-IP architecture and be capable of carrying multitude real-time multimedia applications such as voice and video as well as data. It is very necessary for MANETs to have an efficient routing and quality of service (QoS) mechanism to support diverse applications. This thesis proposes an on-demand Node-Disjoint Multipath Routing protocol (NDMR) with low broadcast redundancy. Multipath routing allows the establishment of multiple paths between a single source and single destination node. It is also beneficial to avoid traffic congestion and frequent link breaks in communication because of the mobility of nodes. The important components of the protocol, such as path accumulation, decreasing routing overhead and selecting node-disjoint paths, are explained. Because the new protocol significantly reduces the total number of Route Request packets, this results in an increased delivery ratio, smaller end-to-end delays for data packets, lower control overhead and fewer collisions of packets. Although NDMR provides node-disjoint multipath routing with low route overhead in MANETs, it is only a best-effort routing approach, which is not enough to support QoS. DiffServ is a standard approach for a more scalable way to achieve QoS in any IP network and could potentially be used to provide QoS in MANETs because it minimises the need for signalling. However, one of the biggest drawbacks of DiffServ is that the QoS provisioning is separate from the routing process. This thesis presents a Multipath QoS Routing protocol for iv supporting DiffServ (MQRD), which combines the advantages of NDMR and DiffServ. The protocol can classify network traffic into different priority levels and apply priority scheduling and queuing management mechanisms to obtain QoS guarantees

Design of rate-adaptive MAC and medium aware routing protocols for multi-rate, multi-hop wireless networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The IEEE 802.11 standard conformant wireless communication stations have multi-rate transmission capability. To achieve greater communication efficiency, multi-rate capable stations use rateadaptation to select appropriate transmission rate according to variations in the channel quality. The thesis presents two rate-adaptation schemes, each belonging to one of the two classes of rateadaptation schemes i.e.(1) the frame-transmission statistics based schemes, and (2) Signal-to-Noise Ratio (SNR) based, closed loop schemes. The SNR-based rate-adaptation scheme, proposed in this thesis uses a novel mechanism of delivering a receiver’s feedback to a transmitter; without requiring any modification in the standard frames as suggested by existing research. The frame-transmissionstatistics based rate adaptation solution uses an on-demand incremental strategy for selecting a rate-selection threshold. This solution is based on a cross-layer communication framework, where the rate-adaptation module uses information to/from the Application layer along with relevant information from the Medium Access Control (MAC) sub-layer. The proposed solutions are highly responsive when compared with existing rate-adaptation schemes; responsiveness is one of the key factors in the design of such protocols. The novel feedback mechanism makes it possible to achieve frame-loss differentiation with just three frames, avoiding the use of Request To Send/ Clear To Send (RTS/CTS) frames and further delays in this process. Performance tests have affirmed that the proposed rate-adaptation schemes are energy efficient; with efficiency up to 19% in specific test scenarios. In terms of throughput and frame loss-differentiation mechanisms, the proposed schemes have shown significantly better performance.Routing protocols for Mobile Ad-Hoc Networks (MANETs) use broadcast frames during the route discovery process. The 802.11 mandates the use of different transmission rates for broadcast and unicast (data-) frames. In many cases it causes creation of communication gray zones, where stations which are marked as ‘reachable neighbours’ using the broadcast frames (using lower transmission rate) are not accessible during normal, unicast communication (mainly at a higher rate). Similarly, higher device density, interference and mobility cause variable medium access delays. The IEEE 802.11e introduces four different MAC level queues for four access categories, maintaining service priority within the queues; which implies that frames from a higher priority queue are serviced more frequently than those belonging to lower priority queues. Such an enhancement at the MAC sub-layer introduces uneven queuing delays. Conventional routing protocols are unaware of such MAC specific constraints and as a result these factors are not considered which result in severe performance deterioration. To meet such challenges, the thesis presents a medium aware distance vector (MADV) routing protocol for MANETs. MADV uses MAC and physical layer (PHY) specific information in the route metric and maintains a separate route per-AC-per-destination in its routing tables. The MADV-metric can be incorporated into various routing rotocols and its applicability is determined by the possibility of provision of MAC dependent arameters that are used to determine the hop-by-hop MADV-metric values. Simulation tests and omparison with existing MANET protocols demonstrate the effectiveness of incorporating the medium dependent parameters and show that MADV is significantly better in terms of end-to-end delay and throughput

Supporting Internet Access and Quality of Service in Distributed Wireless Ad Hoc Networks

In this era of wireless hysteria, with continuous technological advances in wireless communication and new wireless technologies becoming standardized at a fast rate, we can expect an increased interest for wireless networks, such as ad hoc and mesh networks. These networks operate in a distributed manner, independent of any centralized device. In order to realize the practical benefits of ad hoc networks, two challenges (among others) need to be considered: distributed QoS guarantees and multi-hop Internet access. In this thesis we present conceivable solutions to both of these problems. An autonomous, stand-alone ad hoc network is useful in many cases, such as search and rescue operations and meetings where participants wish to quickly share information. However, an ad hoc network connected to the Internet is even more desirable. This is because Internet plays an important role in the daily life of many people by offering a broad range of services. In this thesis we present AODV+, which is our solution to achieve this network interconnection between a wireless ad hoc network and the wired Internet. Providing QoS in distributed wireless networks is another challenging, but yet important, task mainly because there is no central device controlling the medium access. In this thesis we propose EDCA with Resource Reservation (EDCA/RR), which is a fully distributed MAC scheme that provides QoS guarantees by allowing applications with strict QoS requirements to reserve transmission time for contention-free medium access. Our scheme is compatible with existing standards and provides both parameterized and prioritized QoS. In addition, we present the Distributed Deterministic Channel Access (DDCA) scheme, which is a multi-hop extension of EDCA/RR and can be used in wireless mesh networks. Finally, we have complemented our simulation studies with real-world ad hoc and mesh network experiments. With the experience from these experiments, we obtained a clear insight into the limitations of wireless channels. We could conclude that a wise design of the network architecture that limits the number of consecutive wireless hops may result in a wireless mesh network that is able to satisfy users’ needs. Moreover, by using QoS mechanisms like EDCA/RR or DDCA we are able to provide different priorities to traffic flows and reserve resources for the most time-critical applications

Contributions to provide a QoS-aware self-configured framework for video-streaming services over ad hoc networks

Aplicat embargament des del dia 27 d'octubre de 2021 fins 31 de juliol de 2022Ad hoc networks have attracted much attention from the research community over the last years and important technical advances have risen as a consequence. These networks are foreseen as an important kind of next generation access networks, where multimedia services will be demanded by end users from their wireless devices everywhere. In this thesis, we specially focus our research work on mobile ad hoc networks (MANETs) and on vehicular ad hoc networks (VANETs), two kind of ad hoc networks over which interesting multimedia services can be provided. The special haracteristics of MANETs/VANETs, such as mobility, dynamic network topology (specially in VANETs), energy constraints (in case of MANETs), infrastructureless and variable link capacity, make the QoS (Quality of Service) provision over these networks an important challenge for the research community. Due to that, there is a need to develop new routing protocols specially designed for MANETs and VANETs able to provide multimedia services. The main objective of this thesis is to contribute in the development of the communication framework for MANETs and VANETs to improve decisions to select paths or next hops in the moment of forwarding video-reporting messages. In this way, it would be possible to have a quick answer to manage daily problems in the city and help the emergency units (e.g., police, ambulances, health care units) in case of incidents (e.g., traffic accidents). Furthermore, in case of VANETs, a real scenario must be created and thus we have analysed the presence of obstacles in real maps. Also, in case of an obstacle found between the current forwarding node and the candidate next forwarding node, the packet is stored in a buffer, for a maximum time, until a forwarding neighbour node is found; otherwise, the packet is dropped. To improve the communication framework for MANETs, we propose a new routing protocol based on a game-theoretical scheme for N users specially designed to transmit video-reporting messages. Our proposal makes the network more efficient and provides a higher degree of satisfaction of the users by receiving much more packets with a lower average end-to-end delay, lower jitter and higher PSNR (Peak Signal-to-Noise Ratio). In addition, we propose a geographical routing protocol for VANETs that considers multiple metrics named 3MRP (Multimedia Multimetric Map-Aware Routing Protocol) [1]. 3MRP is a geographical protocol based on hop-by-hop forwarding. The metrics considered in 3MRP are the distance, the density of vehicles in transmission range, the available bandwidth, the future trajectory of the neighbouring nodes and the MAC layer losses. Those metrics are weighted to obtain a multimetric score. Thus, a node selects another node among its neighbours as the best forwarding node to increase the percentage of successful packet delivery, minimizing the average packet delay and offering a certain level of quality and service. Furthermore, a new algorithm named DSW (Dynamic Self-configured Weights) computes for each metric its corresponding weight depending on the current network conditions. As a consequence, nodes are classiffied in a better way.Les xarxes sense fils ad hoc han captat molt l'atenció per part de la comunitat científica en els últims anys, a més dels importants avenços tècnics que han sorgit. Aquestes xarxes es preveuen com un tipus important de xarxes d'accés de nova generació, a on els serveis multimèdia seran requerits pels usuaris a través dels seus dispositius sense fils desde tot arreu. En aquesta tesi, centrem el nostre treball especialment en les xarxes mòbils ad hoc (MANET, Mobile Ad hoc Network) i en les xarxes vehiculars ad hoc (VANET, Vehicular Ad hoc Network). Les característiques especials de les MANETs i les VANETs, com la mobilitat, la topologia dinàmica de la xarxa (especialment en VANETs), les restriccions de bateria (en cas de MANETs), l'absència d'infrastructura i la capacitat variable de l'enllaç sense fil fa que la provisió de la qualitat de servei en aquestes xarxes sigui un repte important per a la comunitat científica. A causa d'això, hi ha la necessitat de desenvolupar nous protocols d'encaminament especialment dissenyats per a MANETs i VANETs capaços de proporcionar els serveis de multimèdia requerits. L'objectiu principal d'aquesta tesi és proveir millores en la comunicació per a les xarxes ad hoc MANET i VANET per millorar les decisions a l'hora de seleccionar els propers camins o nodes, respectivament, en el moment de l'enviament de les trames del video. D'aquesta manera serà possible tenir una resposta ràpida per resoldre els problema diaris a la ciutat i ajudar a les unitats d'emergència (per exemple, policia, ambulàncies, unitats de salut) en cas d'incidents, com ara els accidents de trànsit. Per analitzar adequadament les nostres propostes sobre VANETs hem dissenyat un entorn de simulació realista que incorpora la presència d'edificis en mapes reals i en el cas que hi hagi un obstacle entre el node actual i el candidat per a ser el pròxim salt el paquet s'emmagatzema en un buffer, per un temps màxim, fins a trobar un nou candidat; en cas contrari, es descarta el paquet. Per millorar les comunicacions en les MANETs, proposem un nou protocol d'encaminament basat en teoria de jocs per a N usuaris especialment dissenyat per a enviar missatges de vídeo. Això fa que la xarxa sigui més eficient, i així s'aconsegueix un major grau de satisfacció dels usuaris en rebre molts més paquets amb un menor retard mig extrem a extrem, menor variació del retard (jitter ) i major PSNR (Relació Senyal Soroll de Pic) . A més, es proposa un protocol d'encaminament geogràfic basat en el reenviament hop-by-hop per a VANETs anomenat 3MRP (Multimedia Multimetric Map-Aware Routing Protocol ) [1] que prèn en consideració múltiples mètriques. Les mètriques considerades en 3MRP són la distància a destinació, la densitat de vehicles en el rang de transmissió, l'ample de banda disponible, la trajectòria futura dels nodes veíns i la perdua de paquets a la capa MAC. Aquestes mètriques es ponderen per a obtenir una puntuació multimètrica. Així, un node pot seleccionar el millor node de reenviament entre tots els seus veíns per augmentar la probabilitat d' èxit de lliurament de paquets, minimitzant el retard mitjà dels paquets i oferint un cert nivell de qualitat de servei.Las redes ad hoc han llamado mucho la atención por parte de la comunidad científica en los últimos años, además de los importantes avances técnicos que han surgido. Estas redes se prevén como un tipo importante de redes de acceso de nueva generación, donde los servicios multimedia sean requeridos por los usuarios a través de sus dispositivos inalámbricos desde todas partes. En esta tesis, centramos nuestro trabajo de investigación especialmente en las redes móviles ad hoc (MANET, Mobile Ad hoc Network) y las redes vehiculares ad hoc (VANET, Vehicular Ad hoc Network). Las características especiales de las MANETs y las VANETs, como la movilidad, la topología dinámica de la red (especialmente en VANETs), las restricciones de batería (en caso de MANETs), la ausencia de infraestructura y la capacidad variable del enlace inalámbrico hace que la provisión de la calidad de servicio en estas redes sea un reto importante para la comunidad científica. Debido a esto, existe la necesidad de desarrollar nuevos protocolos de encaminamiento especialmente diseñados para MANETs y VANETs capaces de proporcionar los servicios de multimedia requeridos. El objetivo principal de esta tesis es proveer mejoras en la comunicación para las redes ad hoc MANET y VANET para mejorar las decisiones a la hora de seleccionar los próximos caminos o nodos, respectivamente, en el momento del envío de las tramas del video y de esta manera sería posible tener una respuesta rápida para resolver los problema diarios en la ciudad y ayudar a las unidades de emergencia (por ejemplo, policía, ambulancias, unidades de salud) en caso de incidentes, como accidentes de tráfico. Para analizar adecuadamente nuestras propuestas sobre VANETs hemos diseñado un entorno de simulación realista que incorpora la presencia de edificios en mapas reales. En el caso de que haya un obstáculo entre el nodo actual y el candidato para ser el próximo salto el paquete se almacena en un buer, durante un tiempo máximo, hasta encontrar un nuevo candidato; en caso contrario, se descarta el paquete. Para mejorar las comunicaciones en las MANETs, proponemos un nuevo protocolo de encaminamiento basado en teoría de juegos para N usuarios especialmente diseñados para enviar mensajes de video. Esto hace que la red sea más eficiente, y así se consigue un mayor grado de satisfacción de los usuarios al recibir muchos más paquetes con un menor promedio de retardo de extremo a extremo, variación de retardo (jitter) y mayor PSNR (Relación Señal a Ruido de Pico). Además, se propone un protocolo de encaminamiento geográfico basado en el reenvío de salto-a-salto para VANETs llamado 3MRP (Multimedia multimetric Map-Aware Routing Protocol) [1] que incluye diversas métricas. Las métricas consideradas en 3MRP son la distancia al destino, la densidad de vehículos en el rango de transmisión, el ancho de banda disponible, la trayectoria futura de los nodos vecinos y la pérdida de paquetes en la capa MAC. Estas métricas se ponderan para obtener una puntuación multimetricanal. Así, un nodo puede seleccionar el mejor nodo de reenvío entre todos sus vecinos para aumentar la probabilidad de éxito de entrega de paquetes, minimizando el retardo medio de los paquetes y ofreciendo un cierto nivel de calidad de servicio. Por otra parte, se ha diseñado un nuevo algoritmo capaz de dar a cada métrica su correspondiente peso en función de las condiciones actuales de la red. De esta forma, los nodos se pueden clasificar de una mejor manera. Por último, se propone un nuevo protocolo de encaminamiento para VANETs llamado G-3MRP (Game Theoretical Multimedia Multimetric Map-aware Routing Protocol) [2] para enviar mensajes de video basado en teoría de juegos para N usuarios en escenarios urbanos. G-3MRP se basa en el protocolo de encaminamiento 3MRP. G-3MRP utiliza hasta tres nodos a través de los cuales los tres tipos de cuadros de video I, P y B serán enviados. Las métricas utilizadas son las mismas que en 3MRP. G-3MRP logra una mayor grado de satisfacción de los usuarios mediante la recepción de muchos más paquetes de video y con un mayor nivel de PSNR, que la anterior propuesta 3MRP+DSW. También hemos analizado el problema de detección de obstáculos en mapas reales para VANETs en escenarios urbanos. Para este propósito, hemos desarrollado nuestra herramienta REVsim [3] de tal forma que puede estar fácilmente integrada en nuestra propuesta de protocolo de encaminamiento para que las simulaciones sean más realistas.Postprint (published version

Optimized path selection in a game-theoretic routing protocol for video-streaming services over MANETs

Mobile ad hoc networks (MANETs) are infrastructureless networks formed by wireless mobile devices. Recently, the demand over multimedia services such as video streaming has increased specially since the number of mobile end users is growing as well. MPEG-2 VBR is one of the most fitting video coding techniques for MANETs which improves the distribution of video streams specially when it is used with a proper multipath routing scheme. In this article, we aimed to design a routing scheme to dynamically select the forwarding paths using a game-theoretic approach over a multipath routing protocol. Our proposal seeks to describe an equation of the probability p of sending video frames through the best available path. p depends on network parameters that vary throughout time. The aim is that the most important video frames (I+P) will be sent through the best path with a certain probability p and will be sent through the second best path with a probability 1-p. To achieve that, we carried out simulations done with fixed values of p and after that we applied a lineal regression method to obtain the coefficients of the equation for p. Simulations have been done to show the benefits of our proposal where interfering traffic and mobility of the nodes are present.Peer ReviewedPostprint (published version

Flexible cross layer design for improved quality of service in MANETs

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel UniversityMobile Ad hoc Networks (MANETs) are becoming increasingly important because of their unique characteristics of connectivity. Several delay sensitive applications are starting to appear in these kinds of networks. Therefore, an issue in concern is to guarantee Quality of Service (QoS) in such constantly changing communication environment. The classical QoS aware solutions that have been used till now in the wired and infrastructure wireless networks are unable to achieve the necessary performance in the MANETs. The specialized protocols designed for multihop ad hoc networks offer basic connectivity with limited delay awareness and the mobility factor in the MANETs makes them even more unsuitable for use. Several protocols and solutions have been emerging in almost every layer in the protocol stack. The majority of the research efforts agree on the fact that in such dynamic environment in order to optimize the performance of the protocols, there is the need for additional information about the status of the network to be available. Hence, many cross layer design approaches appeared in the scene. Cross layer design has major advantages and the necessity to utilize such a design is definite. However, cross layer design conceals risks like architecture instability and design inflexibility. The aggressive use of cross layer design results in excessive increase of the cost of deployment and complicates both maintenance and upgrade of the network. The use of autonomous protocols like bio-inspired mechanisms and algorithms that are resilient on cross layer information unavailability, are able to reduce the dependence on cross layer design. In addition, properties like the prediction of the dynamic conditions and the adaptation to them are quite important characteristics. The design of a routing decision algorithm based on Bayesian Inference for the prediction of the path quality is proposed here. The accurate prediction capabilities and the efficient use of the plethora of cross layer information are presented. Furthermore, an adaptive mechanism based on the Genetic Algorithm (GA) is used to control the flow of the data in the transport layer. The aforementioned flow control mechanism inherits GA’s optimization capabilities without the need of knowing any details about the network conditions, thus, reducing the cross layer information dependence. Finally, is illustrated how Bayesian Inference can be used to suggest configuration parameter values to the other protocols in different layers in order to improve their performance.National Foundation of Scholarships of Greece(I.K.Y.

Analysis, evaluation and improvement of RT-WMP for real-time and QoS wireless communication: Applications in confined environments

En los ultimos años, la innovación tecnológica, la característica de flexibilidad y el rápido despligue de las redes inalámbricas, han favorecido la difusión de la redes móviles ad-hoc (MANETs), capaces de ofrecer servicios para tareas específicas entre nodos móviles. Los aspectos relacionados al dinamismo de la topología móvil y el acceso a un medio compartido por naturaleza hacen que sea preciso enfrentarse a clases de problemas distintos de los relacionados con la redes cableadas, atrayendo de este modo el interés de la comunidad científica. Las redes ad-hoc suelen soportar tráfico con garantía de servicio mínimo y la mayor parte de las propuestas presentes en literatura tratan de dar garantías de ancho de banda o minimizar el retardo de los mensajes. Sin embargo hay situaciones en las que estas garantías no son suficientes. Este es el caso de los sistemas que requieren garantías mas fuertes en la entrega de los mensajes, como es el caso de los sistemas de tiempo real donde la pérdida o el retraso de un sólo mensaje puede provocar problemas graves. Otras aplicaciones como la videoconferencia, cada vez más extendidas, implican un tráfico de datos con requisitos diferentes, como la calidad de servicio (QoS). Los requisitos de tiempo real y de QoS añaden nuevos retos al ya exigente servicio de comunicación inalámbrica entre estaciones móviles de una MANET. Además, hay aplicaciones en las que hay que tener en cuenta algo más que el simple encaminamiento de los mensajes. Este es el caso de aplicaciones en entornos subterráneos, donde el conocimiento de la evolución de propagación de la señal entre los diferentes nodos puede ser útil para mejorar la calidad de servicio y mantener la conectividad en cada momento. A pesar de ésto, dentro del amplio abanicos de propuestas presente en la literatura, existen un conjunto de limitaciones que van de el mero uso de protocolos simulados a propuestas que no tienen en cuenta entornos no convencionales o que resultan aisladas desde el punto de vista de la integración en sistemas complejos. En esta tesis doctoral, se propone un estudio completo sobre un plataforma inalámbrica de tiempo real, utilizando el protocolo RT-WMP capaz de gestionar trafíco multimedia al mismo tiempo y adaptado al entorno de trabajo. Se propone una extensión para el soporte a los datos con calidad de servicio sin limitar las caractaristícas temporales del protocolo básico. Y con el fin de tener en cuenta el efecto de la propagación de la señal, se caracteriza el entorno por medio de un conjunto de restricciones de conectividad. La solución ha sido desarrollada y su validez ha sido demostrada extensamente en aplicaciones reales en entornos subterráneos, en redes malladas y aplicaciones robóticas