Control of transport dynamics in overlay networks

Transport control is an important factor in the performance of Internet protocols, particularly in the next generation network applications involving computational steering, interactive visualization, instrument control, and transfer of large data sets. The widely deployed Transport Control Protocol is inadequate for these tasks due to its performance drawbacks. The purpose of this dissertation is to conduct a rigorous analytical study on the design and performance of transport protocols, and systematically develop a new class of protocols to overcome the limitations of current methods. Various sources of randomness exist in network performance measurements due to the stochastic nature of network traffic. We propose a new class of transport protocols that explicitly accounts for the randomness based on dynamic stochastic approximation methods. These protocols use congestion window and idle time to dynamically control the source rate to achieve transport objectives. We conduct statistical analyses to determine the main effects of these two control parameters and their interaction effects. The application of stochastic approximation methods enables us to show the analytical stability of the transport protocols and avoid pre-selecting the flow and congestion control parameters. These new protocols are successfully applied to transport control for both goodput stabilization and maximization. The experimental results show the superior performance compared to current methods particularly for Internet applications. To effectively deploy these protocols over the Internet, we develop an overlay network, which resides at the application level to provide data transmission service using User Datagram Protocol. The overlay network, together with the new protocols based on User Datagram Protocol, provides an effective environment for implementing transport control using application-level modules. We also study problems in overlay networks such as path bandwidth estimation and multiple quickest path computation. In wireless networks, most packet losses are caused by physical signal losses and do not necessarily indicate network congestion. Furthermore, the physical link connectivity in ad-hoc networks deployed in unstructured areas is unpredictable. We develop the Connectivity-Through-Time protocols that exploit the node movements to deliver data under dynamic connectivity. We integrate this protocol into overlay networks and present experimental results using network to support a team of mobile robots

Novel applications and contexts for the cognitive packet network

Autonomic communication, which is the development of self-configuring, self-adapting, self-optimising and self-healing communication systems, has gained much attention in the network research community. This can be explained by the increasing demand for more sophisticated networking technologies with physical realities that possess computation capabilities and can operate successfully with minimum human intervention. Such systems are driving innovative applications and services that improve the quality of life of citizens both socially and economically. Furthermore, autonomic communication, because of its decentralised approach to communication, is also being explored by the research community as an alternative to centralised control infrastructures for efficient management of large networks. This thesis studies one of the successful contributions in the autonomic communication research, the Cognitive Packet Network (CPN). CPN is a highly scalable adaptive routing protocol that allows for decentralised control in communication. Consequently, CPN has achieved significant successes, and because of the direction of research, we expect it to continue to find relevance. To investigate this hypothesis, we research new applications and contexts for CPN. This thesis first studies Information-Centric Networking (ICN), a future Internet architecture proposal. ICN adopts a data-centric approach such that contents are directly addressable at the network level and in-network caching is easily supported. An optimal caching strategy for an information-centric network is first analysed, and approximate solutions are developed and evaluated. Furthermore, a CPN inspired forwarding strategy for directing requests in such a way that exploits the in-network caching capability of ICN is proposed. The proposed strategy is evaluated via discrete event simulations and shown to be more effective in its search for local cache hits compared to the conventional methods. Finally, CPN is proposed to implement the routing system of an Emergency Cyber-Physical System for guiding evacuees in confined spaces in emergency situations. By exploiting CPN’s QoS capabilities, different paths are assigned to evacuees based on their ongoing health conditions using well-defined path metrics. The proposed system is evaluated via discrete-event simulations and shown to improve survival chances compared to a static system that treats evacuees in the same way.Open Acces

A hybrid load flow and event driven simulation approach to multi-state system reliability evaluation

Structural complexity of systems, coupled with their multi-state characteristics, renders their reliability and availability evaluation difficult. Notwithstanding the emergence of various techniques dedicated to complex multi-state system analysis, simulation remains the only approach applicable to realistic systems. However, most simulation algorithms are either system specific or limited to simple systems since they require enumerating all possible system states, defining the cut-sets associated with each state and monitoring their occurrence. In addition to being extremely tedious for large complex systems, state enumeration and cut-set definition require a detailed understanding of the system׳s failure mechanism. In this paper, a simple and generally applicable simulation approach, enhanced for multi-state systems of any topology is presented. Here, each component is defined as a Semi-Markov stochastic process and via discrete-event simulation, the operation of the system is mimicked. The principles of flow conservation are invoked to determine flow across the system for every performance level change of its components using the interior-point algorithm. This eliminates the need for cut-set definition and overcomes the limitations of existing techniques. The methodology can also be exploited to account for effects of transmission efficiency and loading restrictions of components on system reliability and performance. The principles and algorithms developed are applied to two numerical examples to demonstrate their applicability

Route Planning in Transportation Networks

We survey recent advances in algorithms for route planning in transportation networks. For road networks, we show that one can compute driving directions in milliseconds or less even at continental scale. A variety of techniques provide different trade-offs between preprocessing effort, space requirements, and query time. Some algorithms can answer queries in a fraction of a microsecond, while others can deal efficiently with real-time traffic. Journey planning on public transportation systems, although conceptually similar, is a significantly harder problem due to its inherent time-dependent and multicriteria nature. Although exact algorithms are fast enough for interactive queries on metropolitan transit systems, dealing with continent-sized instances requires simplifications or heavy preprocessing. The multimodal route planning problem, which seeks journeys combining schedule-based transportation (buses, trains) with unrestricted modes (walking, driving), is even harder, relying on approximate solutions even for metropolitan inputs.Comment: This is an updated version of the technical report MSR-TR-2014-4, previously published by Microsoft Research. This work was mostly done while the authors Daniel Delling, Andrew Goldberg, and Renato F. Werneck were at Microsoft Research Silicon Valle

Evolutionary optimisation of network flow plans for emergency movement in the built environment

Planning for emergency evacuation, and, more generally, for emergency movement involving both evacuation (egress) of occupants and ingress of first responders, presents important and challenging problems. A number of the current issues that arise during emergency incidents are due to the uncertainty and transiency of environmental conditions. In general, movement plans are formulated at building design-time, and those involved, such as building occupants and emergency responders, are left to adapt routing plans to actual events as they unfold. In the context of next-generation emergency response systems, it has been proposed to dynamically plan and route individuals during an emergency event, replanning to take account of changes in the environment. In this work, an emergency movement problem, the Maximal Safest Escape (MSE) problem, is formulated in terms that model the uncertain and transient environmental conditions as a flow problem in time-dependent networks with time-varying and stochastic edge travel-times and capacities (STV Networks). The objective of the MSE problem is to find flow patterns with the a priori maximal probability of successfully conveying all supply from the source to the sink in some given STV Network. The MSE and its deterministic counterpart are proved to be NP-hard. Furthermore, due to inherent complexity in evaluating the exact quality of candidate solutions, a simulation approximation method is presented based on well-known Monte-Carlo sampling methods. Given the complexity of the problem, and using the approximation method for evaluating solutions, it is proposed to tackle the MSE problem using a metaheuristic approach based on an existing framework that integrates Evolutionary Algorithms (EA) with a state-of-the-art statistical ranking and selection method, the Optimal Computing Budget Allocation (OCBA). Several improvements are proposed for the framework to reduce the computational demand of the ranking method. Empirically, the approach is compared with a simple fitness averaging approach and conditions under which the integrated framework is more efficient are investigated. The performance of the EA is compared against upper and lower bounds on optimal solutions. An upper bound is established through the “wait-and-see” bound, and a lower bound by a naıve random search algorithm (RSA). An experimental design is presented that allows for a fair comparison between the EA and the RSA. While there is no guarantee that the EA will find optimal solutions, this work demonstrates that the EA can still find useful solutions; useful solutions are those that are at least better than some baseline, here the lower bound, in terms of solution quality and computational effort. Experimentally, it is demonstrated that the EA performs significantly better than the baseline. Also, the EA finds solutions relatively close to the upper bound; however, it is difficult to establish how optimistic the upper bounds. The main approach is also compared against an existing approach developed for solving a related problem wrapped in a heuristic procedure in order to apply the approach to the MSE. Empirical results show that the heuristic approach requires significantly less computation time, but finds solutions of significantly lower quality. Overall, this work introduces and empirically verifies the efficacy of a metaheuristic based on a framework integrating EAs with a state-of-the-art statistical ranking and selection technique, the OCBA, for a novel flow problem in STV Networks. It is suggested that the lessons learned during the course of this work, along with the specific techniques developed, may be relevant for addressing other flow problems of similar complexity

Maximum entropy based evolutionary optimization of water distribution networks under multiple operating conditions and self-adaptive search space reduction method

Previously held under moratorium from 1st December 2016 until 1st December 2021.One of the complexities in designing WDN is evaluation of network performance. The accurate network performance measures such as reliability or failure tolerance are very time consuming calculations, thus surrogate measures are used for water distribution network (WDN) design optimization. Entropy is particularly advantageous since it involves only the flow in the pipe and the demands at the nodes. This thesis developed efficient new computational methods based on the maximum entropy formalism for the optimization of water distribution systems. Thus the maximum entropy based design approach has been extended here to include multiple operation conditions. Also, the path-related properties of the flow entropy have been exploited to develop a new self-adaptive approach for solution space reduction in multiobjective evolutionary optimization of water distribution systems that resulted in a significant reduction in the number of function evaluations required to find optimal and near optimal solutions. The novelty and originality of the current research are presented next. A new penalty-free multi-objective evolutionary optimization approach for the design of WDNs has been developed. It combines genetic algorithm with least cost design and maximum entropy. The approach can handle single operating conditions (SOC) as well as multiple operating conditions (MOC) for any given network. Previously, most of the work has been done for single loading patterns and it was assumed that nodal demands are constant. In reality nodal demand vary over the time so network designed to satisfy one operating condition might not be able to satisfy other loading patterns (i.e. pressure constraints might not be meet). The model has been applied to three well known water distribution networks. The approach has also been implemented on a large real-world network in the literature. Three different methods of designing for multiple loading patterns were investigated. Extensive testing proved that MOC outperform SOC in terms of hydraulic feasibility, pipe size distribution and reliability. The approach is computationally efficient and robust. The above mentioned penalty-free approach has been extended to form a module that would improve the convergence criteria of the GA by reducing its search space. For large real-world network GA might require extremely large number of function evaluations which could lead to delayed convergence. By reducing the search space, the GA’s effectiveness and efficiency will increase as the algorithm will identify the solutions in smaller number of function evaluations. The search space reduction method presented herein is based on entropy and uses the importance of every path through network, which is an inherent property of the entropy function. The developed algorithm is dynamic, self-adaptive and does not require pre-defining the reduced sets of candidate diameters for each pipe. The method has been applied to a large network from the literature. Two cases were studied, one based on full search space and one for reduce search space (RSS) approach. Rapid stabilization was observed for the results obtained using RSS.One of the complexities in designing WDN is evaluation of network performance. The accurate network performance measures such as reliability or failure tolerance are very time consuming calculations, thus surrogate measures are used for water distribution network (WDN) design optimization. Entropy is particularly advantageous since it involves only the flow in the pipe and the demands at the nodes. This thesis developed efficient new computational methods based on the maximum entropy formalism for the optimization of water distribution systems. Thus the maximum entropy based design approach has been extended here to include multiple operation conditions. Also, the path-related properties of the flow entropy have been exploited to develop a new self-adaptive approach for solution space reduction in multiobjective evolutionary optimization of water distribution systems that resulted in a significant reduction in the number of function evaluations required to find optimal and near optimal solutions. The novelty and originality of the current research are presented next. A new penalty-free multi-objective evolutionary optimization approach for the design of WDNs has been developed. It combines genetic algorithm with least cost design and maximum entropy. The approach can handle single operating conditions (SOC) as well as multiple operating conditions (MOC) for any given network. Previously, most of the work has been done for single loading patterns and it was assumed that nodal demands are constant. In reality nodal demand vary over the time so network designed to satisfy one operating condition might not be able to satisfy other loading patterns (i.e. pressure constraints might not be meet). The model has been applied to three well known water distribution networks. The approach has also been implemented on a large real-world network in the literature. Three different methods of designing for multiple loading patterns were investigated. Extensive testing proved that MOC outperform SOC in terms of hydraulic feasibility, pipe size distribution and reliability. The approach is computationally efficient and robust. The above mentioned penalty-free approach has been extended to form a module that would improve the convergence criteria of the GA by reducing its search space. For large real-world network GA might require extremely large number of function evaluations which could lead to delayed convergence. By reducing the search space, the GA’s effectiveness and efficiency will increase as the algorithm will identify the solutions in smaller number of function evaluations. The search space reduction method presented herein is based on entropy and uses the importance of every path through network, which is an inherent property of the entropy function. The developed algorithm is dynamic, self-adaptive and does not require pre-defining the reduced sets of candidate diameters for each pipe. The method has been applied to a large network from the literature. Two cases were studied, one based on full search space and one for reduce search space (RSS) approach. Rapid stabilization was observed for the results obtained using RSS

Simulation Methods for the Analysis of Complex Systems

open access bookEveryday systems like communication, transportation, energy and industrial systems are an indispensable part of our daily lives. Several methods have been developed for their reliability assessment—while analytical methods are computationally more efficient and often yield exact solutions, they are unable to account for the structural and functional complexities of these systems. These complexities often require the analyst to make unrealistic assumptions, sometimes at the expense of accuracy. Simulation-based methods, on the other hand, can account for these realistic operational attributes but are computationally intensive and usually system-specific. This chapter introduces two novel simulation methods: load flow simulation and survival signature simulation which together address the limitations of the existing analytical and simulation methods for the reliability analysis of large systems

In-situ Data Analytics In Cyber-Physical Systems

Cyber-Physical System (CPS) is an engineered system in which sensing, networking, and computing are tightly coupled with the control of the physical entities. To enable security, scalability and resiliency, new data analytics methodologies are required for computing, monitoring and optimization in CPS. This work investigates the data analytics related challenges in CPS through two study cases: Smart Grid and Seismic Imaging System. For smart grid, this work provides a complete solution for system management based on novel in-situ data analytics designs. We first propose methodologies for two important tasks of power system monitoring: grid topology change and power-line outage detection. To address the issue of low measurement redundancy in topology identification, particularly in the low-level distribution network, we develop a maximum a posterior based mechanism, which is capable of embedding prior information on the breakers status to enhance the identification accuracy. In power-line outage detection, existing approaches suer from high computational complexity and security issues raised from centralized implementation. Instead, this work presents a distributed data analytics framework, which carries out in-network processing and invokes low computational complexity, requiring only simple matrix-vector multiplications. To complete the system functionality, we also propose a new power grid restoration strategy involving data analytics for topology reconfiguration and resource planning after faults or changes. In seismic imaging system, we develop several innovative in-situ seismic imaging schemes in which each sensor node computes the tomography based on its partial information and through gossip with local neighbors. The seismic data are generated in a distributed fashion originally. Dierent from the conventional approach involving data collection and then processing in order, our proposed in-situ data computing methodology is much more ecient. The underlying mechanisms avoid the bottleneck problem on bandwidth since all the data are processed distributed in nature and only limited decisional information is communicated. Furthermore, the proposed algorithms can deliver quicker insights than the state-of-arts in seismic imaging. Hence they are more promising solutions for real-time in-situ data analytics, which is highly demanded in disaster monitoring related applications. Through extensive experiments, we demonstrate that the proposed data computing methods are able to achieve near-optimal high quality seismic tomography, retain low communication cost, and provide real-time seismic data analytics