Dynamic routing in circuit-switched non-hierarchical networks

This thesis studies dynamic routing in circuit-switched non-hierarchical networks based on learning automata algorithms. The application of a mathematical model for a linear reward penalty algorithm is explained. Theoretical results for this scheme verified by simulations shows the accuracy of the model. Using simulation and analysis, learning automata algorithms are compared to several other strategies on different networks. The implemented test networks may be classified into two groups. The first group are designed for fixed routing and in such networks fixed routing performs better than any dynamic routing scheme. It will be shown that dynamic routing strategies perform as well as fixed routing when trunk reservation is employed. The second group of networks are designed for dynamic routing and trunk reservation deteriorates the performance. Comparison of different routing algorithms on small networks designed to force dynamic routing demonstrates the superiority of automata under both normal and failure conditions. The thesis also considers the instability problem in non-hierarchical circuit-switched networks when dynamic routing is implemented. It is shown that trunk reservation prevents instability and increases the carried load at overloads. Finally a set of experiments are performed on large networks with realistic capacity and traffic matrices. Simulation and analytic results show that dynamic routing outperforms fixed routing and trunk reservation deteriorates the performance at low values of overload. At high overloads, optimization of trunk reservation is necessary for this class of networks. Comparison results show the improved performance with automata schemes under both normal and abnormal traffic conditions. The thesis concludes with a discussion of proposed further work including expected developments in Integrated Service Digital Networks

Learning algorithms for the control of routing in integrated service communication networks

There is a high degree of uncertainty regarding the nature of traffic on future integrated service networks. This uncertainty motivates the use of adaptive resource allocation policies that can take advantage of the statistical fluctuations in the traffic demands. The adaptive control mechanisms must be 'lightweight', in terms of their overheads, and scale to potentially large networks with many traffic flows. Adaptive routing is one form of adaptive resource allocation, and this thesis considers the application of Stochastic Learning Automata (SLA) for distributed, lightweight adaptive routing in future integrated service communication networks. The thesis begins with a broad critical review of the use of Artificial Intelligence (AI) techniques applied to the control of communication networks. Detailed simulation models of integrated service networks are then constructed, and learning automata based routing is compared with traditional techniques on large scale networks. Learning automata are examined for the 'Quality-of-Service' (QoS) routing problem in realistic network topologies, where flows may be routed in the network subject to multiple QoS metrics, such as bandwidth and delay. It is found that learning automata based routing gives considerable blocking probability improvements over shortest path routing, despite only using local connectivity information and a simple probabilistic updating strategy. Furthermore, automata are considered for routing in more complex environments spanning issues such as multi-rate traffic, trunk reservation, routing over multiple domains, routing in high bandwidth-delay product networks and the use of learning automata as a background learning process. Automata are also examined for routing of both 'real-time' and 'non-real-time' traffics in an integrated traffic environment, where the non-real-time traffic has access to the bandwidth 'left over' by the real-time traffic. It is found that adopting learning automata for the routing of the real-time traffic may improve the performance to both real and non-real-time traffics under certain conditions. In addition, it is found that one set of learning automata may route both traffic types satisfactorily. Automata are considered for the routing of multicast connections in receiver-oriented, dynamic environments, where receivers may join and leave the multicast sessions dynamically. Automata are shown to be able to minimise the average delay or the total cost of the resulting trees using the appropriate feedback from the environment. Automata provide a distributed solution to the dynamic multicast problem, requiring purely local connectivity information and a simple updating strategy. Finally, automata are considered for the routing of multicast connections that require QoS guarantees, again in receiver-oriented dynamic environments. It is found that the distributed application of learning automata leads to considerably lower blocking probabilities than a shortest path tree approach, due to a combination of load balancing and minimum cost behaviour

Simulation and analysis of adaptive routing and flow control in wide area communication networks

This thesis presents the development of new simulation and analytic models for the performance analysis of wide area communication networks. The models are used to analyse adaptive routing and flow control in fully connected circuit switched and sparsely connected packet switched networks. In particular the performance of routing algorithms derived from the L(_R-I) linear learning automata model are assessed for both types of network. A novel architecture using the INMOS Transputer is constructed for simulation of both circuit and packet switched networks in a loosely coupled multi- microprocessor environment. The network topology is mapped onto an identically configured array of processing centres to overcome the processing bottleneck of conventional Von Neumann architecture machines. Previous analytic work in circuit switched work is extended to include both asymmetrical networks and adaptive routing policies. In the analysis of packet switched networks analytic models of adaptive routing and flow control are integrated to produce a powerful, integrated environment for performance analysis The work concludes that routing algorithms based on linear learning automata have significant potential in both fully connected circuit switched networks and sparsely connected packet switched networks

Improved learning automata applied to routing in multi-service networks

Multi-service communications networks are generally designed, provisioned and configured, based on source-destination user demands expected to occur over a recurring time period. However due to network users' actions being non-deterministic, actual user demands will vary from those expected, potentially causing some network resources to be under- provisioned, with others possibly over-provisioned. As actual user demands vary over the recurring time period from those expected, so the status of the various shared network resources may also vary. This high degree of uncertainty necessitates using adaptive resource allocation mechanisms to share the finite network resources more efficiently so that more of actual user demands may be accommodated onto the network. The overhead for these adaptive resource allocation mechanisms must be low in order to scale for use in large networks carrying many source-destination user demands. This thesis examines the use of stochastic learning automata for the adaptive routing problem (these being adaptive, distributed and simple in implementation and operation) and seeks to improve their weakness of slow convergence whilst maintaining their strength of subsequent near optimal performance. Firstly, current reinforcement algorithms (the part causing the automaton to learn) are examined for applicability, and contrary to the literature the discretised schemes are found in general to be unsuitable. Two algorithms are chosen (one with fast convergence, the other with good subsequent performance) and are improved through automatically adapting the learning rates and automatically switching between the two algorithms. Both novel methods use local entropy of action probabilities for determining convergence state. However when the convergence speed and blocking probability is compared to a bandwidth-based dynamic link-state shortest-path algorithm, the latter is found to be superior. A novel re-application of learning automata to the routing problem is therefore proposed: using link utilisation levels instead of call acceptance or packet delay. Learning automata now return a lower blocking probability than the dynamic shortest-path based scheme under realistic loading levels, but still suffer from a significant number of convergence iterations. Therefore the final improvement is to combine both learning automata and shortest-path concepts to form a hybrid algorithm. The resulting blocking probability of this novel routing algorithm is superior to either algorithm, even when using trend user demands

Adaptive control of communication networks using learning automata.

This research investigates communications network routing procedures, based on distributed learning automata concepts for circuit and packet switched networks. For this application, the learning automaton is shown to be an ideal adaptive control mechanism, with simple feedback and updating strategies which allow extremely practical implementations and perform very close to the desired optimum. In this thesis, the nature of learning automata routing schemes are explored by analytical and computer simulation techniques, primarily developing an elementary understanding of the automata routing and adaption process. Using simple circuit and message switched networks the conditions for minimum blocking probability and average delay are established and compared with the equilibrium behaviour of learning automata operating under alternative reinforcement algorithms. Later, large scale simulations of real networks are used to demonstrate and relate the learning automata scheme to existing routing techniques. These experiments, which are performed on sophisticated simulation packages produced for this study, take as examples hierarchical and general structured telephone networks and packet switched communications networks configured with both virtual call and datagram protocols. In addition, studies under failure mode conditions, including link, node and focussed overloads, conclusively demonstrate the superior performance afforded by the learning automata routing approach

Theory and application of learning automata.

Although the theoretical performance of many learning automata has been considered, the practical operation of these automata has received far less attention. This work starts with the construction of two action Tsetlin and Krylov automata. The performance of these automata has been measured in stationary and non-stationary environments. The operation of a hierarchical automaton controlling the memory size of a Tsetlin automaton is also investigated. Two new automata are proposed with the aim of avoiding the operational disadvantages of the Tsetlin automaton. These automata have been tested using a computer simulation and, in addition, theoretical performance results have been calculated and compared with results for Tsetlin, Krylov and Lri automata. A model of a non-autonomous environment is simulated and its operation analysed theoretically. A more accurate model is analysed, and its operation with a Lri automaton examined and compared to theoretical predictions. The requirements for learning automata to operate successfully in non-autonomous environments are considered and it is shown that the Lrp and Lri automata do not converge to the optimum for a non-autonomous environment. Three automata are proposed, which are designed to operate in non-autonomous environments. Their performances are compared to those of the Lrp and Lri automata. The operation of automata in a hierarchical learning system and in cooperative and competitive games is considered. In these situations the performance of the new automata is compared to that of the Lrp and Lri automata. Finally, two applications of learning automata are investigated. The first considers the Tsetlin allocation scheme, gives a modification that increases the performance and makes a comparison with a scheme using other learning automata. The second involves the selection of a processor in a multiprocessor computer system and compares a scheme using learning automata with a fixed scheduling discipline