Reliable Messaging to Millions of Users with MigratoryData

Web-based notification services are used by a large range of businesses to selectively distribute live updates to customers, following the publish/subscribe (pub/sub) model. Typical deployments can involve millions of subscribers expecting ordering and delivery guarantees together with low latencies. Notification services must be vertically and horizontally scalable, and adopt replication to provide a reliable service. We report our experience building and operating MigratoryData, a highly-scalable notification service. We discuss the typical requirements of MigratoryData customers, and describe the architecture and design of the service, focusing on scalability and fault tolerance. Our evaluation demonstrates the ability of MigratoryData to handle millions of concurrent connections and support a reliable notification service despite server failures and network disconnections

Mitigating the Performance Impact of Network Failures in Public Clouds

Some faults in data center networks require hours to days to repair because they may need reboots, re-imaging, or manual work by technicians. To reduce traffic impact, cloud providers \textit{mitigate} the effect of faults, for example, by steering traffic to alternate paths. The state-of-art in automatic network mitigations uses simple safety checks and proxy metrics to determine mitigations. SWARM, the approach described in this paper, can pick orders of magnitude better mitigations by estimating end-to-end connection-level performance (CLP) metrics. At its core is a scalable CLP estimator that quickly ranks mitigations with high fidelity and, on failures observed at a large cloud provider, outperforms the state-of-the-art by over 700$\times$ in some cases

Providing Predictable Performance during Network Contingencies

In IP backbone networks, packets may get dropped due to: i) lack of viable next hops when a link/router fails, ii) forwarding loops during network convergence, and iii) buffer overflows in case of congestion. Similarly, packets may be lost in wireless networks due to variations in signal strength between a pair of mobile nodes. This dissertation explores the possibility of providing a predictable performance during such network contingencies in wired backbone networks and robotic wireless networks. First, we study the feasibility of developing a combination of local reroute and global update mechanisms that can achieve loop-free convergence, while performing disruption-free forwarding around a failed link/router, without carrying any additional information in the IP datagrams and with out needing any coordination between routers. We show that order of updates rarely matters for loop-free convergence when failure inference based fast reroute (FIFR) scheme with interface-specific forwarding is employed for dealing with link or router failures. In the rare cases where order matters, it can be coupled with progressive link metric increments to ensure loop-freedom with unordered updates of forwarding tables. We also demonstrate that, apart from providing protection against failures, FIFR can also be utilized to mitigate packet drops due to network congestion caused by micro traffic bursts. Second, we address the problem of constructing a communication map, which encodes information on whether two robots at given locations can communicate using a wireless network. Unlike previous offline approaches that do not utilize data measured by robots, we propose an online method, utilizing Gaussian Processes, to efficiently build a communication map with multiple robots, by exploiting prior communication models that can be derived from the physical map of the environment. Our evaluation, using a team of TurtleBot 2 platforms, confirms that the proposed method requires robots to take fewer signal strength measurements and travel less distance, and yet obtain similar accuracy as methods that consider all the locations in the environment

Load Balance and Resource Efficiency in Communication Networks

Network management is critical for today’s network. This study investigates both load balancing and resource efficiency in network management. For load balancing, one unfavorable situation is that the active traffic uses a portion of the equal-cost paths instead of all. The root causes of load imbalance are not easily identified and located by network operators. Most research work related in this area concerns the design of load balancing mechanisms or network-wide troubleshooting that does not specify the causes of load imbalance. In this study, we describe a computational framework based on network measurements to identify the correlation mechanism causing the load imbalance. We also describe a novel framework based on Coprime to mitigate the load imbalance brought by hash correlations. In evaluation based on real network trace data and topologies, we have proved that we can reduces the error (CV or K-S statistic) by at least one magnitude. For resource efficiency, today’s network demands an increasing switch memory to support the essential functions, such as forwarding, monitoring, etc. However, the cache memory is restricted when processing data streams in which the input is presented as a sequence of items and can be examined in only a few passes (typically just one). This study introduces a new single-pass reservoir weighted-sampling stream aggregation algorithm, Priority-Based Aggregation (PBA). A naive approach to order sample regardless of key then aggregate the results is hopelessly inefficient. In distinction, our proposed algorithm uses a single persistent random variable across the lifetime of each key in the cache and maintains unbiased estimates of the key aggregates that can be queried at any point in the stream. Concerning statistical properties, we prove that PBA provides unbiased estimates of the true aggregates. We analyze the computational complexity of PBA and its variants and provide a detailed evaluation of its accuracy on synthetic and trace data. In addition to sampling, this study also considers placing classification rules into switches from various network functions. While much work has focused on compressing the rules, most of this work proposes solutions operating in the memory of a single switch. Instead, this study proposed a collaborative approach encompassing switches and network functions. This architecture enables trade-off between usage of (expensive) switch memory and (cheaper) downstream network bandwidth and network function resources. Our system can reduce memory usage significantly compared to strawman approaches as demonstrated with extensive simulations and prototype evaluation with real traffic traces and rules

Resultados y ajustes del Transmilenio

En el momento, Transmilenio constituye una de las prioridades nacional y local. En el presupuesto para el 2005 se asignan cuantiosos recursos de la Nación para apoyar la construcción en diversas ciudades del país, y éstas destinan la mayor parte de su margen presupuestal para realizar contrapartidas. La experiencia que ha tenido Bogotá en los cuatro años de operación ha generado un debate en cuanto a la viabilidad del sistema a largo plazo. En las intervenciones salieron a relucir tres aspectos centrales: la rentabilidad económica del proyecto, la estructura tarifaria y la generación de empleo. El presente artículo resume los resultados de un trabajo que sobre el tema realizó el Centro de Estudios Económicos de la Escuela durante el primer semestre de 2004 .At the moment, Transmilenio is one of the national and local priorities. In the budget for 2005, large national resources are assigned to support construction in various cities of the country, and these allocate most of their budget margin to make counterpart contributions. The experience that Bogotá has had in the four years of operation has generated a debate regarding the viability of the system in the long term. In the interventions, three central aspects came to light: the economic profitability of the project, the tariff structure and the generation of employment. This article summarizes the results of a study on the subject carried out by the School 's Center for Economic Studies during the first semester of 200

Proposal for a wastewater treatment system for livestock activities

En este artículo se presenta el diseño de un sistema de tratamiento de las aguas residuales de una finca dedicada a la producción lechera en el municipio de Une (Cundinamarca) (1).This article presents the design of a wastewater treatment system resulting from milk production of a farm in the municipality of Une (Cundinamarca)