Mobile Networks

The growth in the use of mobile networks has come mainly with the third generation systems and voice traffic. With the current third generation and the arrival of the 4G, the number of mobile users in the world will exceed the number of landlines users. Audio and video streaming have had a significant increase, parallel to the requirements of bandwidth and quality of service demanded by those applications. Mobile networks require that the applications and protocols that have worked successfully in fixed networks can be used with the same level of quality in mobile scenarios. Until the third generation of mobile networks, the need to ensure reliable handovers was still an important issue. On the eve of a new generation of access networks (4G) and increased connectivity between networks of different characteristics commonly called hybrid (satellite, ad-hoc, sensors, wired, WIMAX, LAN, etc.), it is necessary to transfer mechanisms of mobility to future generations of networks. In order to achieve this, it is essential to carry out a comprehensive evaluation of the performance of current protocols and the diverse topologies to suit the new mobility conditions

Saturation routing for asynchronous transfer mode (ATM) networks

The main objective of this thesis is to show that saturation routing, often in the past considered inefficient, can in fact be a viable approach to use in many important applications and services over an Asynchronous Transfer Mode (ATM) network. For other applications and services, a hybrid approach (one that partially uses saturation routing) is presented. First, the minimum effects of saturation routing are demonstrated by showing that the ratio, defined as f, of routing overhead cells over information cells is small even for large networks. Second, modeling and simulation and M/D/l queuing analysis techniques are used to show that the overall effect on performance when using saturation routing is not significant over ATM networks. Then saturation routing ATM implementation is also provided, with important extensions to services such as multicast routing. After an analytical comparison, in terms of routing overhead, is made between Saturation Routing and the currently proposed Private Network-Network Interface (PNNI) procedure for ATM routing made by the ATM forum. This comparison is made for networks of different sizes (343node and 2401 -node networks) and different number of hierarchical levels (3 and 4 levels of hierarchy). The results show that the higher the number of levels of hierarchy and the farthest (in terms of hierarchical levels) the source and the destination nodes are from each other, the more advantageous saturation routing becomes. Finally, a set of measures of performance for use by saturation routing (or any routing algorithm), as metrics for routing path selection, is proposed. Among these measures, an innovative new measure of performance derived for measuring quality of service provided to Constant Bit Rate (CBR) users (e.g., such as voice and video users) called the Burst Voice Arrival Lag (BVAL) is described and derived

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

Distributed Optimisation in Wireless Sensor Networks: A Hierarchical Learning Approachs

Ph.DDOCTOR OF PHILOSOPH

A Scalable Virtual Circuit Routing Scheme for ATM Networks

High-speed networks, such as ATM networks, are expected to support diverse quality-of-service (QoS) requirements, including real-time QoS. Real-time QoS is required by many applications such as voice and video. To support such service, routing protocols based on the Virtual Circuit (VC) model have been proposed. However, these protocols do not scale well to large networks in terms of storage and communication overhead. In this paper, we present a scalable VC routing protocol. It is based on the recently proposed viewserver hierarchy, where each viewserver maintains a partial view of the network. By querying these viewservers, a source can obtain a merged view that contains a path to the destination. The source then sends a request packet over this path to setup a real-time VC through resource reservations. The request is blocked if the setup fails. We compare our protocol to a simple approach using simulation. Under this simple approach, a source maintains a full view of the network. In addition to the savings in storage, our results indicate that our protocol performs close to or better than the simple approach in terms of VC carried load and blocking probability over a wide range of real-time workload. (Also cross-referenced as UMIACS-TR-94-115

Video streaming over the internet using application layer multicast

Multicast is a very important communication paradigm. However, the deployment of multicast at IP layer is very slow, due to development and deployment issues such as ISPs' lack of incentives to update routers and inter-operability among multicast routing protocols. Application Layer Multicast (ALM) is a good alternative, where participating peers organize themselves into a logical overlay network atop the physical links and data is \tunneled" to each other via unicast links. The distinctive feature between IP multicast and ALM is that in ALM, data replication and forwarding functionalities are performed by participating peers (a.k.a. end systems), rather than the routers in Internet Protocol (IP) multicast. This fundamental difference enables ALM to be able to circumvent the development and deployment issues of IP multicast, by exploiting the resources (e.g., CPU cycles, storage, and access bandwidth) at the edge of the network. Nevertheless, it also raises other challenges, as peers are not as stable as routers since they may join and depart the on-going session at will. In this thesis, we address some of the challenges and they are summarized as follows: First, most current P2P or ALM streaming systems are equipped with a non-scalable membership management algorithm, greatly hindering their applicability to large-scale implementations over the Internet: they either rely on a central entity to handle group membership, or simply assume that all group members are visible to each other and flooding is the main mechanism used to disseminate membership-related updates to all participating group members. This implies that they are only applicable to small groups. Second, one of ALM's prominent features, flexility, has not been fully exploited: moving the multicast functionalities from lower layer (IP layer) to higher layer (Application layer) can greatly facilitate the integration of Quality-of-Service (QoS) support. The end-to-end philosophy states that it is better to leave those functionalities to higher layers because the heterogeneity among users' requirements can be handled much better by end users, rather than the network. However, QoS, and in particular, reliability has not been thoroughly addressed in existing ALM schemes. Third, admission control algorithms are essential to the success of any ALM system, due to the fact that in ALM, each peer acts as both a client as well as a server. On the other hand, the heterogeneity among peers, in terms of their computational power, storage capacity, and access bandwidth, further complicates the design of a good admission control. Several contributions are made to address the aforementioned research challenges, and they are outlined as follows: The first contribution is a devised gossip-based membership management algorithm that is able to collect and disseminate membership-related information under high rate of churn, using relatively low communication overheads. The second contribution is a reliability-centric multicast tree construction algorithm that greatly enhance peers' perceived reliability. The third contribution is a QoS-aware tree construction algorithm that accommodates the heterogeneity among peers, such as access bandwidth, network distance, and reliability. The last contribution is the identification of the admission control problem in this overlay video streaming

Integrating LEO Satellite Constellations into Internet Backbone

Low Earth Orbit (LEO) satellite constellations have been used for ubiquitous and flexible Internet access services. However, a number of problems related to the integration of terrestrial with satellite hosts should be resolved for the effective exploitation of LEO constellations. LEO constellations are different from terrestrial Internet because of its special properties, which result in a lot of problems. A key issue is how to route Internet packets to the LEO constellation. In the thesis (1) the background of LEO constellations was introduced; (2) the obstacles of routing between the satellites and Internet were outlined; (3) The particular problem, which must be solved, is the routing burst stream traffic in LEO satellite constellations. Two novel routing algorithmsCControl Route Transmission (CRT) and CRT with bandwidth allocation (BCRT)Cwere utilized to address the bursts routing problem. CRT is an adaptive protocol which is able to minimize the congestion in the constellations. BCRT is a CRT extension which is allowed to class the traffic (e.g. video) with different QoS requirements and guarantees. Both of CRT and BCRT work in time epochs. Routes are computed on the basis of a directed weighted graph representing the global traffic traveling in the constellations. Both CRT and BCRT were evaluated via simulation and compared with other proposals in the literatures. The results showed that CRT is a simple algorithm, but the strategy produced by CRT could avoid the congestion and enhance the global resource usage in different traffic conditions. Moreover, the explicit reservation and reroute of BCRT greatly improve the performance of CRT. In particular, the dropping rate of BCRT is very low and the average delivery time is comparable with other proposals in the literatures.Low Earth Orbit (LEO) satellite constellations have been used for ubiquitous and flexible Internet access services. However, a number of problems related to the integration of terrestrial with satellite hosts should be resolved for the effective exploitation of LEO constellations. LEO constellations are different from terrestrial Internet because of its special properties, which result in a lot of problems. A key issue is how to route Internet packets to the LEO constellation. In the thesis (1) the background of LEO constellations was introduced; (2) the obstacles of routing between the satellites and Internet were outlined; (3) The particular problem, which must be solved, is the routing burst stream traffic in LEO satellite constellations. Two novel routing algorithmsCControl Route Transmission (CRT) and CRT with bandwidth allocation (BCRT)Cwere utilized to address the bursts routing problem. CRT is an adaptive protocol which is able to minimize the congestion in the constellations. BCRT is a CRT extension which is allowed to class the traffic (e.g. video) with different QoS requirements and guarantees. Both of CRT and BCRT work in time epochs. Routes are computed on the basis of a directed weighted graph representing the global traffic traveling in the constellations. Both CRT and BCRT were evaluated via simulation and compared with other proposals in the literatures. The results showed that CRT is a simple algorithm, but the strategy produced by CRT could avoid the congestion and enhance the global resource usage in different traffic conditions. Moreover, the explicit reservation and reroute of BCRT greatly improve the performance of CRT. In particular, the dropping rate of BCRT is very low and the average delivery time is comparable with other proposals in the literatures

Using POMDP as Modeling Framework for Network Fault Management

For highדּpeed networks, it is important that fault management be proactive--i.e., detect, diagnose, and mitigate problems before they result in severe degradation of network performance. Proactive fault manageשּׂent depends on monitoring the network to obtain the data on which to base manager decisions. However, monitoring introduces additional overhead that may itself degrade network performance especially when the network is in a stressed state. Thus, a tradeoff must be made be﫠tween the amount of data collected and transferred on one hand, and the speed and accuracy of fault detection and diagnosis on the other hand. Such a tradeoff can be naturally formulated as a Partially Observable Markov decision process (POMDP).Since exact solution of POMDPs for a realistic number of states is computationally prohibitive, we develop a reinforcementשּׁearningﬢased fast algorithm which learns the decisionגּule in an approximate network simulator and makes it fast deployable to the real network. Simulation results are given to diagnose a switch fault in an ATM network. This approach can be applied to centralized fault management or to construct intelligent agents for distributed fault management

Resource allocation and performance analysis problems in optical networks

Optical networks pose a rich variety of new design and performance analysis problems. Typically, the static design problems belong to the field of combinatorial optimisation, whereas decision-making and performance analysis problems are best treated using appropriate stochastic models. This dissertation focuses on certain issues in resource allocation and performance evaluation of backbone wavelength-routed (WR) networks and metropolitan area optical burst switching (OBS) networks. The first two parts of the thesis consider heuristic algorithms for the static routing and wavelength assignment (RWA) and logical topology design (LTD) problems that arise in the context of WR networks. In a static RWA problem, one is asked to establish a given set of lightpaths (or light trees) in an optical WR network with given constraints, where the objective often is to minimise the number of wavelength channels required. In LTD problem, the number of wavelength channels is given and one is asked to decide on the set of lightpaths so that, for instance, the mean sojourn time of packets travelling at the logical layer is minimised. In the thesis, several heuristic algorithms for both the RWA and LTD problems are described and numerical results are presented. The third part of the thesis studies the dynamic control problem where connection requests, i.e. lightpath requests, arrive according to a certain traffic pattern and the task is to establish one lightpath at a time in the WR optical network so that the expected revenue is maximised or the expected cost is minimised. Typically, the goal of optimisation is to minimise some infinite time horizon cost function, such as the blocking probability. In this thesis, the dynamic RWA problem is studied in the framework of Markov decision processes (MDP). An algorithmic approach is proposed by which any given heuristic algorithm can be improved by applying the so-called first policy iteration (FPI) step of the MDP theory. Relative costs of states needed in FPI are estimated by on-the-fly simulations. The computational burden of the approach is alleviated by introducing the importance sampling (IS) technique with FPI, for which an adaptive algorithm is proposed for adjusting the optimal IS parameters at the same time as data are collected for the decision-making analysis. The last part of the thesis considers OBS networks, which represent an intermediate step towards full optical packet switching networks. In OBS networks, the data are transferred using optical bursts consisting of several IP packets going to the same destination. On the route of the burst, temporary reservations are made only for the time during which the burst is transmitted. This thesis focuses on fairness issues in OBS networks. It is demonstrated that fairness can be improved by using fibre delay lines together with Just-Enough-Time protocol (JET). Furthermore, by choosing the routes in an appropriate way one can also reach a satisfactory level of fairness and, at the same time, lower the overall blocking probability. Possible scheduling policies for metropolitan area OBS ring networks are also studied.reviewe