Rethinking Timestamping: Time Stamp Counter Design for Virtualized Environment

Almost every processor supports Time Stamp Counter (TSC), which is a hardware register that increments its value every clock cycle. Due to its high resolution and accessibility, TSC is now widely used for a variety tasks that need time measurements such as wall clock, code benchmarking, or metering hardware usage for account billing. However, if not carefully configured and interpreted, TSC-based time measurements can yield inaccurate readings. For instance, modern CPU may dynamically change its frequency or enter into low-power states. Also, time spent on scheduling events, system calls, page faults, etc. should be correctly accounted for. Even more complications arise when TSC measurements are done in virtual environments; virtual machines, on which TSC readings are taken, can be suspended, migrated, and scheduled on a machine with different clock rate and performance. In production virtualization systems, some management tasks are executed inside guests on behalf of the management system, effectively consuming end-user’s CPU time, which we believe should be excluded from end-user billing. We argue that the main problem with current TSC is that its hardware semantic is too vague to serve as a multi-purpose time source. In this thesis, we propose an improved TSC design, called Caviar, to address most of the issues. Caviar extends existing TSC hardware interface by adding a control-register based configuration interface through which a system can set up secondary TSCs whose behavior should be correct when accessed in a localized execution context including virtualized environment. We experimentally confirmed inaccurate readings with current TSC by conducting a series of TSC measurements on various x86 platforms, including virtualized cloud computing servers. We analyzed some of the results and argue that how our proposed solution can fix the problems. In conclusion, we believe that the simple interface of Caviar can solve most of current TSC complications, be implemented with minimal hardware cost, and be adopted easily by system software

Global Synchronization of Asynchronous Computing Systems

The MSU ERC UltraScope system consists of a distributed computing system, custom PCI cards, GPS receivers, and a re-radiation system. The UltraScope system allows precision timestamping of events in a distributed application on a system where the CPU and PCI clocks are phase-locked. The goal of this research is to expand the UltraScope system, using software routines and minimal hardware modifications, to allow precision timestamping of events on an asynchronous distributed system. The timestamp process is similar to the Network Time Protocol (NTP) in that it uses a series of timestamps to improve precision. As expected, the precision is less accurate on an asynchronous system than on a synchronous system. Results show that the precision is improved using this sequence of timestamps, and the major error component is due to operating system delays. The errors associated with this timestamping process are characterized using a synchronous system as a baseline

Fine-grained preemption analysis for latency investigation across virtual machines

This paper studies the preemption between programs running in different virtual machines on the same computer. One of the current monitoring methods consist of updating the average steal time through collaboration with the hypervisor. However, the average is insufficient to diagnose abnormal latencies in time-sensitive applications. Moreover, the added latency is not directly visible from the virtual machine point of view. The main challenge is to recover the cause of preemption of a task running in a virtual machine, whether it is a task on the host computer or in another virtual machine. We propose a new method to study thread preemption crossing virtual machines boundaries using kernel tracing. The host computer and each monitored virtual machine are traced simultaneously. We developed an efficient and portable trace synchronization method, which is required to account for time offset and drift that occur within each virtual machine. We then devised an algorithm to recover the root cause of preemption between threads at every level. The algorithm successfully detected interactions between multiple competing threads in distinct virtual machines on a multi-core machine