Performance and policy dimensions in internet routing

The Internet Routing Project, referred to in this report as the 'Highball Project', has been investigating architectures suitable for networks spanning large geographic areas and capable of very high data rates. The Highball network architecture is based on a high speed crossbar switch and an adaptive, distributed, TDMA scheduling algorithm. The scheduling algorithm controls the instantaneous configuration and swell time of the switch, one of which is attached to each node. In order to send a single burst or a multi-burst packet, a reservation request is sent to all nodes. The scheduling algorithm then configures the switches immediately prior to the arrival of each burst, so it can be relayed immediately without requiring local storage. Reservations and housekeeping information are sent using a special broadcast-spanning-tree schedule. Progress to date in the Highball Project includes the design and testing of a suite of scheduling algorithms, construction of software reservation/scheduling simulators, and construction of a strawman hardware and software implementation. A prototype switch controller and timestamp generator have been completed and are in test. Detailed documentation on the algorithms, protocols and experiments conducted are given in various reports and papers published. Abstracts of this literature are included in the bibliography at the end of this report, which serves as an extended executive summary

Rethinking Timestamping: Time Stamp Counter Design for Virtualized Environment

Almost every processor supports Time Stamp Counter (TSC), which is a hardware register that increments its value every clock cycle. Due to its high resolution and accessibility, TSC is now widely used for a variety tasks that need time measurements such as wall clock, code benchmarking, or metering hardware usage for account billing. However, if not carefully configured and interpreted, TSC-based time measurements can yield inaccurate readings. For instance, modern CPU may dynamically change its frequency or enter into low-power states. Also, time spent on scheduling events, system calls, page faults, etc. should be correctly accounted for. Even more complications arise when TSC measurements are done in virtual environments; virtual machines, on which TSC readings are taken, can be suspended, migrated, and scheduled on a machine with different clock rate and performance. In production virtualization systems, some management tasks are executed inside guests on behalf of the management system, effectively consuming end-user’s CPU time, which we believe should be excluded from end-user billing. We argue that the main problem with current TSC is that its hardware semantic is too vague to serve as a multi-purpose time source. In this thesis, we propose an improved TSC design, called Caviar, to address most of the issues. Caviar extends existing TSC hardware interface by adding a control-register based configuration interface through which a system can set up secondary TSCs whose behavior should be correct when accessed in a localized execution context including virtualized environment. We experimentally confirmed inaccurate readings with current TSC by conducting a series of TSC measurements on various x86 platforms, including virtualized cloud computing servers. We analyzed some of the results and argue that how our proposed solution can fix the problems. In conclusion, we believe that the simple interface of Caviar can solve most of current TSC complications, be implemented with minimal hardware cost, and be adopted easily by system software

Fine-grained preemption analysis for latency investigation across virtual machines

This paper studies the preemption between programs running in different virtual machines on the same computer. One of the current monitoring methods consist of updating the average steal time through collaboration with the hypervisor. However, the average is insufficient to diagnose abnormal latencies in time-sensitive applications. Moreover, the added latency is not directly visible from the virtual machine point of view. The main challenge is to recover the cause of preemption of a task running in a virtual machine, whether it is a task on the host computer or in another virtual machine. We propose a new method to study thread preemption crossing virtual machines boundaries using kernel tracing. The host computer and each monitored virtual machine are traced simultaneously. We developed an efficient and portable trace synchronization method, which is required to account for time offset and drift that occur within each virtual machine. We then devised an algorithm to recover the root cause of preemption between threads at every level. The algorithm successfully detected interactions between multiple competing threads in distinct virtual machines on a multi-core machine

Freezing time emulating new and faster devices with virtual machines

Recent proposals of emerging data storage devices make it necessary to reevaluate all levels of the storage hierarchy to optimize the software stack performance. However, these new devices are not always widely available and therefore early experiments may be impossible. Emulators aim at mimicking as close as possible the behavior of a component, nonetheless, emulating new and fast storage devices is a challenging task due to time perception. In this work, we propose an approach to emulate storage devices using virtual machines (VMs) allowing the evaluation of a new device within a real system. We use a technique called freezing time, which pauses a VM to manipulate its clock and hide the real I/O completion time. Our approach is implemented at the hypervisor level and it is transparent to the guest operating system or application. We evaluate the technique under a real system using regular magnetic disks to emulate faster storage devices. Our method presented a latency error of 6.5% compared to a real device. Moreover, decoupled experiment between two laboratories, at the Barcelona Super Computing Center (BSC) in Spain, and the Center of Computer Science and Free Software (C3SL) in Brazil, demonstrated that our approach is reproducible and promising to allow the virtual evaluation of next-gen storage devices.This work was partially supported by the Spanish Ministry of Science and Innovation under the TIN2015-65316 Grant, the Generalitat de Catalunya under contract 2014-SGR-1051, the Serrapilheira Institute (Grant number Serra-1709-16621), as well as the European Union’s Horizon 2020 Research and Innovation Programme, under Grant Agreement no. 671951 (NEXTGenIO) for the extensions added after the MASCOTS paper.Peer ReviewedPostprint (author's final draft

High-Precision, High-Accuracy Timekeeping in Distributed Survey Systems

Accurate and precise timekeeping between computers in a distributed survey system is essential to ensure adequate data quality, especially with Multibeam Echosounders (MBES) which can otherwise suffer from significant motion artefacts. We show that clock synchronisation on the order of 100-150ns (rms) is readily achievable in a modern MBES-based survey system utilising an Ethernet-based time synchronisation mechanism and some custom timekeeping software. We also show that improving the timekeeping eliminates motion artefacts in the observed bathymetry, and simplifies the patch-test procedure.Para asegurar una calidad adecuada de los datos es esencial una exacta y precisa mantencion de la hora entre ordenadores en un sistema de levantamiento distribuido, especialmente con Sondadores Acusticos Multihaz (MBES), que sino pueden experimentar las consecuencias de un significativo movimiento de sus artefactos. Mostramos que la sincronizaci6n de un reloj en el orden de 100-150ns (rms) puede llevarse a cabo facilmente en un sistema moderno de levantamientos efectuados mediante MBES, utilizando un mecanismo Ethernet basado en la sincronizacion del tiempo y en algun programa de mantencion de la hora adaptado. Tambien mostramos que mejorando la mantencion de la hora se elimina el movimiento de los artefactos en la batimetria observada, y simplifica el procedimiento de pruebas en parches.Il est essentiel d'avoir un chronometrage exact et precis entre ordinateurs dans un systeme integre pour les leves hydrographiques afin d'assurer une qualite des donnees appropriee notamment pour les sondeurs multifaisceaux (MBES) qui, dans le cas contraire, peuvent patir de perturbations significatives liees au mouvement. Nous montrons qu'une synchronisation des horloges de l'ordre de 100 a 150ns (valeur quadratique moyenne) est facilement obtenue dans un systeme de leves moderne reposant sur les MBES et utilisant un mecanisme de synchronisation du temps dans un reseau Ethernet et un logiciel specifique de chronometrage. Nous demontrons egalement que l'amelioration du chronometrage permet de supprimer certaines perturbations liees au mouvement dans la bathymetrie observee et de simplifier la procedure d'essai par plage

High-Precision, High-Accuracy Timekeeping in Distributed Survey Systems

Accurate and precise timekeeping between computers in a distributed survey system is essential to ensure adequate data quality, especially with Multibeam Echosounders (MBES) which can otherwise suffer from significant motion artefacts. We show that clock synchronisation on the order of 100-150ns (rms) is readily achievable in a modern MBES-based survey system utilising an Ethernet-based time synchronisation mechanism and some custom timekeeping software. We also show that improving the timekeeping eliminates motion artefacts in the observed bathymetry, and simplifies the patch-test procedure.Para asegurar una calidad adecuada de los datos es esencial una exacta y precisa mantencion de la hora entre ordenadores en un sistema de levantamiento distribuido, especialmente con Sondadores Acusticos Multihaz (MBES), que sino pueden experimentar las consecuencias de un significativo movimiento de sus artefactos. Mostramos que la sincronizaci6n de un reloj en el orden de 100-150ns (rms) puede llevarse a cabo facilmente en un sistema moderno de levantamientos efectuados mediante MBES, utilizando un mecanismo Ethernet basado en la sincronizacion del tiempo y en algun programa de mantencion de la hora adaptado. Tambien mostramos que mejorando la mantencion de la hora se elimina el movimiento de los artefactos en la batimetria observada, y simplifica el procedimiento de pruebas en parches.Il est essentiel d'avoir un chronometrage exact et precis entre ordinateurs dans un systeme integre pour les leves hydrographiques afin d'assurer une qualite des donnees appropriee notamment pour les sondeurs multifaisceaux (MBES) qui, dans le cas contraire, peuvent patir de perturbations significatives liees au mouvement. Nous montrons qu'une synchronisation des horloges de l'ordre de 100 a 150ns (valeur quadratique moyenne) est facilement obtenue dans un systeme de leves moderne reposant sur les MBES et utilisant un mecanisme de synchronisation du temps dans un reseau Ethernet et un logiciel specifique de chronometrage. Nous demontrons egalement que l'amelioration du chronometrage permet de supprimer certaines perturbations liees au mouvement dans la bathymetrie observee et de simplifier la procedure d'essai par plage

Faithful reproduction of network experiments

The proliferation of cloud computing has compelled the research community to rethink fundamental aspects of network systems and architectures. However, the tools commonly used to evaluate new ideas have not kept abreast of the latest developments. Common simulation and emulation frameworks fail to provide scalability, fidelity, reproducibility and execute unmodified code, all at the same time. We present SELENA, a Xen-based network emulation framework that offers fully reproducible experiments via its automation interface and supports the use of unmodified guest operating systems. This allows out-of-the-box compatibility with common applications and OS components, such as network stacks and filesystems. In order to faithfully emulate faster and larger networks, SELENA adopts the technique of time-dilation and transparently slows down the passage of time for guest operating systems. This technique effectively virtualizes the availability of host’s hardware resources and allows the replication of scenarios with increased I/O and computational demands. Users can directly control the tradeoff between fidelity and running-times via intuitive tuning knobs. We evaluate the ability of SELENA to faithfully replicate the behaviour of real systems and compare it against existing popular experimentation platforms. Our results suggest that SELENA can accurately model networks with aggregate link speeds of 44 Gbps or more, while improving by four times the execution time in comparison to ns3 and exhibits near-linear scaling properties.This is the author accepted manuscript. The final version is available from ACM via http://dx.doi.org/10.1145/2658260.265827

Freezing Time: a new approach for emulating fast storage devices using VM

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Recently we are seeing a considerable effort from both academy and industry in proposing new technologies for storage devices. Often these devices are not readily available for evaluation and methods to allow performing their tests just from their performance parameters are an important tool for system administrators. Simulators are a traditional approach for carrying out such evaluations, however, they are more suitable for evaluating the storage device as an isolate component, mostly due to time constraints. In this paper, we propose an approach based on virtual machine technology that is capable of emulate storage devices transparently for the operating system allowing evaluation of simulating devices within a real system using any synthetic or real workload. To emulate devices in real environments it is necessary to use the currently available devices as a storage medium which creates a difficulty when the device to be emulated is faster than this storage medium. To circumvent this limitation we introduce a new technique called Freezing Time, which takes advantage of virtual machine pausing mechanism to manipulate the virtual machine clock and hide the real I/O completion time. Our approach can be implemented just requiring the hypervisor to be modified, providing a high degree of compatibility and flexibility since it is not necessary to modify neither the operating system nor the application. We evaluate our tool under a real system using old magnetic disks to emulate faster storage devices. Experiments using our technique presented an average latency error of 6.08% for read operations and 6.78% for write operations when comparing a real to device.This work was partially supported by the Spanish Ministry of Science and Innovation under the TIN2015–65316 grant, the Generalitat de Catalunya under contract 2014–SGR–1051.Peer ReviewedPostprint (author's final draft