Improved learning automata applied to routing in multi-service networks

Multi-service communications networks are generally designed, provisioned and configured, based on source-destination user demands expected to occur over a recurring time period. However due to network users' actions being non-deterministic, actual user demands will vary from those expected, potentially causing some network resources to be under- provisioned, with others possibly over-provisioned. As actual user demands vary over the recurring time period from those expected, so the status of the various shared network resources may also vary. This high degree of uncertainty necessitates using adaptive resource allocation mechanisms to share the finite network resources more efficiently so that more of actual user demands may be accommodated onto the network. The overhead for these adaptive resource allocation mechanisms must be low in order to scale for use in large networks carrying many source-destination user demands. This thesis examines the use of stochastic learning automata for the adaptive routing problem (these being adaptive, distributed and simple in implementation and operation) and seeks to improve their weakness of slow convergence whilst maintaining their strength of subsequent near optimal performance. Firstly, current reinforcement algorithms (the part causing the automaton to learn) are examined for applicability, and contrary to the literature the discretised schemes are found in general to be unsuitable. Two algorithms are chosen (one with fast convergence, the other with good subsequent performance) and are improved through automatically adapting the learning rates and automatically switching between the two algorithms. Both novel methods use local entropy of action probabilities for determining convergence state. However when the convergence speed and blocking probability is compared to a bandwidth-based dynamic link-state shortest-path algorithm, the latter is found to be superior. A novel re-application of learning automata to the routing problem is therefore proposed: using link utilisation levels instead of call acceptance or packet delay. Learning automata now return a lower blocking probability than the dynamic shortest-path based scheme under realistic loading levels, but still suffer from a significant number of convergence iterations. Therefore the final improvement is to combine both learning automata and shortest-path concepts to form a hybrid algorithm. The resulting blocking probability of this novel routing algorithm is superior to either algorithm, even when using trend user demands

Dynamic bandwidth allocation in multi-class IP networks using utility functions.

PhDAbstact not availableFujitsu Telecommunications Europe Lt

Renegotiable VBR service

In this work we address the problem of supporting the QoS requirements for applications while efficiently allocating the network resources. We analyse this problem at the source node where the traffic profile is negotiated with the network and the traffic is shaped according to the contract. We advocate VBR renegotiation as an efficient mechanism to accommodate traffic fluctuations over the burst time-scale. This is in line with the Integrated Service of the IETF with the Resource reSerVation Protocol (RSVP), where the negotiated contract may be modified periodically. In this thesis, we analyse the fundamental elements needed for solving the VBR renegotiation. A source periodically estimates the needs based on: (1) its future traffic, (2) cost objective, (3) information from the past. The issues of this estimation are twofold: future traffic prediction given a prediction, the optimal change. In the case of a CBR specification the optimisation problem is trivial. But with a VBR specification this problem is complex because of the multidimensionality of the VBR traffic descriptor and the non zero condition of the system at the times where the parameter set is changed. We, therefore, focus on the problem of finding the optimal change for sources with pre-recorded or classified traffic. The prediction of the future traffic is out of the scope of this thesis. Traditional existing models are not suitable for modelling this dynamic situation because they do not take into account the non-zero conditions at the transient moments. To address the shortfalls of the traditional approaches, a new class of shapers, the time varying leaky bucket shaper class, has been introduced and characterised by network calculus. To our knowledge, this is the first model that takes into account non-zero conditions at the transient time. This innovative result forms the basis of Renegotiable VBR Service (RVBR). The application of our RVBR mathematical model to the initial problem of supporting the applications' QoS requirements while efficiently allocating the network resources results in simple, efficient algorithms. Through simulation, we first compare RVBR service versus VBR service and versus renegotiable CBR service. We show that RVBR service provides significant advantages in terms of resource costs and resource utilisation. Then, we illustrate that when the service assumes zero conditions at the transient time, the source could potentially experience losses in the case of policing because of the mismatch between the assumed bucket and buffer level and the policed bucket and buffer level. As an example of RVBR service usage, we describe the simulation of RVBR service in a scenario where a sender transmits a MPEG2 video over a network using RSVP reservation protocol with Controlled-Load service. We also describe the implementation design of a Video on Demand application, which is the first example of an RVBR-enabled application. The simulation and experimentation results lead us to believe that RVBR service provides an adequate service (in terms of QoS guaranteed and of efficient resource allocation) to sources with pre-recorded or classified traffic

A new approach to service provisioning in ATM networks

The authors formulate and solve a problem of allocating resources among competing services differentiated by user traffic characteristics and maximum end-to-end delay. The solution leads to an alternative approach to service provisioning in an ATM network, in which the network offers directly for rent its bandwidth and buffers and users purchase freely resources to meet their desired quality. Users make their decisions based on their own traffic parameters and delay requirements and the network sets prices for those resources. The procedure is iterative in that the network periodically adjusts prices based on monitored user demand, and is decentralized in that only local information is needed for individual users to determine resource requests. The authors derive the network's adjustment scheme and the users' decision rule and establish their optimality. Since the approach does not require the network to know user traffic and delay parameters, it does not require traffic policing on the part of the network

QoS-based multipath routing for the Internet

The new generation of network services is being developed for incorporation in communication infrastructure. These services, generally called Quality of Services (QoS), should accommodate data file, video, and audio applications. The different performance requirements of these applications necessitate a re-examination of the main architectural components of today\u27s networks, which were designed to support traditional data applications. Routing, which determines the sequence of network nodes a packet traverses between source and destination, is one such component. Here, we examine the potential routing problems in future Internet and discuss the advantages of class-based multi-path routing methods. The result is a new approach to routing in packet-switched networks, which is called Two-level Class-based Multipath routing with Prediction (TCMP). In TCMP, we compute multiple paths between each source and destination based on link propagation delay and bottleneck bandwidth. A leaky bucket is adopted in each router to monitor the bottleneck bandwidth on equal paths during the network\u27s stable period, and to guide its traffic forwarDing The TCMP can avoid frequent flooding of routing information in a dynamic routing method; therefore, it can be applied to large network topologies

Learning algorithms for the control of routing in integrated service communication networks

There is a high degree of uncertainty regarding the nature of traffic on future integrated service networks. This uncertainty motivates the use of adaptive resource allocation policies that can take advantage of the statistical fluctuations in the traffic demands. The adaptive control mechanisms must be 'lightweight', in terms of their overheads, and scale to potentially large networks with many traffic flows. Adaptive routing is one form of adaptive resource allocation, and this thesis considers the application of Stochastic Learning Automata (SLA) for distributed, lightweight adaptive routing in future integrated service communication networks. The thesis begins with a broad critical review of the use of Artificial Intelligence (AI) techniques applied to the control of communication networks. Detailed simulation models of integrated service networks are then constructed, and learning automata based routing is compared with traditional techniques on large scale networks. Learning automata are examined for the 'Quality-of-Service' (QoS) routing problem in realistic network topologies, where flows may be routed in the network subject to multiple QoS metrics, such as bandwidth and delay. It is found that learning automata based routing gives considerable blocking probability improvements over shortest path routing, despite only using local connectivity information and a simple probabilistic updating strategy. Furthermore, automata are considered for routing in more complex environments spanning issues such as multi-rate traffic, trunk reservation, routing over multiple domains, routing in high bandwidth-delay product networks and the use of learning automata as a background learning process. Automata are also examined for routing of both 'real-time' and 'non-real-time' traffics in an integrated traffic environment, where the non-real-time traffic has access to the bandwidth 'left over' by the real-time traffic. It is found that adopting learning automata for the routing of the real-time traffic may improve the performance to both real and non-real-time traffics under certain conditions. In addition, it is found that one set of learning automata may route both traffic types satisfactorily. Automata are considered for the routing of multicast connections in receiver-oriented, dynamic environments, where receivers may join and leave the multicast sessions dynamically. Automata are shown to be able to minimise the average delay or the total cost of the resulting trees using the appropriate feedback from the environment. Automata provide a distributed solution to the dynamic multicast problem, requiring purely local connectivity information and a simple updating strategy. Finally, automata are considered for the routing of multicast connections that require QoS guarantees, again in receiver-oriented dynamic environments. It is found that the distributed application of learning automata leads to considerably lower blocking probabilities than a shortest path tree approach, due to a combination of load balancing and minimum cost behaviour

Call admission and routing in telecommunication networks.

by Kit-man Chan.Thesis (M.Phil.)--Chinese University of Hong Kong, 1994.Includes bibliographical references (leaves 82-86).Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Overview of Integrated Service Digital Networks --- p.1Chapter 1.2 --- Multirate Loss Networks --- p.5Chapter 1.3 --- Previous Work --- p.7Chapter 1.4 --- Organization --- p.11Chapter 1.5 --- Publications --- p.12Chapter 2 --- Call Admission in Multirate Loss Networks --- p.13Chapter 2.1 --- Introduction --- p.13Chapter 2.2 --- Two Adaptive Routing Rules --- p.15Chapter 2.3 --- Call Admission Policies --- p.17Chapter 2.4 --- Analysis of Call Admission Policies --- p.25Chapter 2.4.1 --- "The CS, LO, GB and the EB Policies" --- p.25Chapter 2.4.2 --- The DP Policy --- p.29Chapter 2.5 --- Performance Comparisons --- p.32Chapter 2.6 --- Concluding Remarks --- p.35Chapter 3 --- Least Congestion Routing in Multirate Loss Networks --- p.41Chapter 3.1 --- Introduction --- p.41Chapter 3.2 --- The M2 and MTB Routings --- p.42Chapter 3.2.1 --- M2 Routing --- p.43Chapter 3.2.2 --- MTB Routing --- p.43Chapter 3.3 --- Bandwidth Sharing Policies and State Aggregation --- p.45Chapter 3.4 --- Analysis of M2 Routing --- p.47Chapter 3.5 --- Analysis of MTB Routing --- p.50Chapter 3.6 --- Numerical Results and Discussions --- p.53Chapter 3.7 --- Concluding Remarks --- p.56Chapter 4 --- The Least Congestion Routing in WDM Lightwave Networks --- p.60Chapter 4.1 --- Introduction --- p.60Chapter 4.2 --- Architecture and Some Design Issues --- p.62Chapter 4.3 --- The Routing Rule --- p.66Chapter 4.4 --- Analysis of the LC Routing Rule --- p.67Chapter 4.4.1 --- Fixed Point Model --- p.67Chapter 4.4.2 --- Without Direct-link Priority --- p.68Chapter 4.4.3 --- With Direct-link Priority --- p.72Chapter 4.5 --- Performance Comparisons --- p.73Chapter 4.6 --- Concluding Remarks --- p.75Chapter 5 --- Conclusions and Future Work --- p.79Chapter 5.1 --- Future Work --- p.8

Performance Analysis and Multi-Objective Design for Multirate Multihop Loss Networks

In this paper, we consider a class of loss networks where multipletraffic classes are present, each has different bandwidth requirement,and each traffic stream is routed according to an adaptive routingscheme.We propose a fixed-point method, a.k.a. reduced load approximation,to estimate the end-to-end blocking probability for such networks.The approximation scheme is shownto be asymptotically correct in a natural limiting regime, and it givesconservative estimates of blocking probabilities under heavy trafficload.Simulation results are provided to compare performance estimatesobtained from our analytical approximation scheme and discrete eventsimulations.We also show how this analytical approximation scheme can be linked withnumerical mathematical programming tools to help design a network,by selecting network design parameters via trade-off analysis, evenwith several design objectives.In one application we use the multi-objective optimization toolCONSOL-OPTCAD to design trunk reservation parameters and balance linkcapacity. In another application we use automatic differentiationto get sensitivities of blocking probabilities w.r.t. offered trafficload