Heiser, Gernot

LeVasseur, Joshua

Uhlig, Volkmar

KITopen

Are Virtual-Machine Monitors Microkernels Done Right?Gernot HeiserNational ICT Australia∗ and University of New South WalesSydney, Australiagernot@nicta.com.auVolkmar UhligIBM T.J. Watson Research Center, Yorktown Heights, NYvuhlig@us.ibm.comJoshua LeVasseurUniversity of Karlsruhe, Germanyjtl@ira.uka.deAbstractA paper by Hand et al. at the recent HotOS work-shop re-examined microkernels and contrasted themto virtual-machine monitors (VMMs). It found thatthe two kinds of systems share architectural com-monalities but also have a number of technical dif-ferences which the paper examined. It concluded thatVMMs are a special case of microkernels, “microker-nels done right”.A closer examination of that paper shows that itcontains a number of statements which are poorlyjustified or even refuted by the literature. While webelieve that it is indeed timely to reexamine the mer-its and issues of microkernels, such an examinationneeds to be based on facts.1 IntroductionAt the HotOS workshop in June this year, Hand andcoauthors presented a paper [HWF+05] titled “Arevirtual machine monitors microkernels done right?”∗National ICT Australia is funded by the Australian Govern-ment’s Department of Communications, Information Technol-ogy, and the Arts and the Australian Research Council throughBacking Australia’s Ability and the ICT Research Centre of Ex-cellence programs.The paper compares and contrasts microkernels andvirtual-machine monitors (VMMs) as platforms forsystems design and implementation. While identify-ing architectural similarities, it examines the differ-ence in the approaches, and concludes that VMMsare one specific point in the microkernel designspace, the “right” one. Unstated but implied is theassertion that VMMs such as Xen [BDF+03] are the(to date) only “right” approach to building microker-nels.Taking a closer look at the main assertions madeby Hand et al, we find that they are hard to justify,or even squarely at odds with the literature. Whilewe think that reexamining the merits and failures ofmicrokernels is a potentially valuable exercise, westrongly believe that such a discussion must be per-formed in accordance with established scientific prin-ciples, and most of all, be grounded in facts. As acontribution to an informed discussion, we examinethe assertions made by Hand et al in the light of thepublic record.2 BackgroundBefore addressing the specific assertions made inHand et al’s paper, we provide some (we hope) usefulbackground for the discussion.952.1 History RevisitedMicrokernels and virtual machine monitors both havea long history, dating back to the early 1970’s [BH70,Gol74]. Given that there are significant similarities,it is useful to look at a somewhat narrow definition ofboth.Goldberg [Gol74] defines a virtual machine mon-itor as “[...] software which transforms the singlemachine interface into the illusion of many. Eachof these interfaces (virtual machines) is an efficientreplica of the original computer system, completewith all of the processor instructions [...]”.Liedtke [Lie96] describes the microkernel ap-proach as “... to minimize the kernel and to imple-ment whatever possible outside of the kernel”.Both definitions appear sufficiently distinct to raisethe question of how much commonality there can be.Examining the goals of the two approaches showsthat there is more similarity than is evident from thedefinitions: Goldberg lists software reliability, datasecurity, alternative system APIs, and improved andnew mechanisms as benefits; Liedtke lists flexibilityand extensibility, fault isolation, maintainability, andrestricted interdependencies.It seems that while VMMs and microkernels sharea common set of goals, they take a different approachtowards the solution. Yet both approaches considerminimality important. While for microkernels it is akey objective, Goldberg reports it as a result of thesystem structure: “A key principle in the analysis ofsoftware reliability is that the VMM is likely to becorrect—i.e., the probability of failure is near zero.This assumption is reasonable because the VMM islikely to be a very small program [...]”.2.2 Core primitivesIn the effort to minimise kernel functionality, micro-kernels offer a minimal set of abstractions with a cen-tral primitive for extensibility: inter-process commu-nication (IPC). In a microkernel, IPC serves three pri-mary purposes:1. IPC is the mechanism for kernel-controlledchange of execution flow between protectiondomains;2. IPC is the mechanism for kernel-controlled datatransfer between protection domains;3. IPC is the mechanism for resource delegationbetween protection domains that requires mu-tual agreement between multiple (potentiallydistrusting) parties.Combining these three orthogonal operations intoa single primitive reduces the number of securitymechanisms, reduces the code complexity, and re-duces the code size. A smaller code base reducesthe number of errors in the privileged kernel, as wellas reducing the cache footprint. An obvious key re-quirement for any microkernel is thus a low-overheadIPC primitive. All other operations that require acombination of the three mechanisms can be imple-mented via the single IPC primitive.VMMs in comparison, closely resemble processorhardware and offer a rich variety of primitives. Eachprimitive requires a dedicated set of security mecha-nisms, resources, and kernel code. A comprehensivelist is beyond the scope of this paper, thus we only listthe common subset of primitives that can be found inmost VMMs:1. synchronous switch of protection domain fromguest user to guest kernel;2. synchronous switch of protection domain fromguest kernel to guest user;3. asynchronous communication channels acrossdomains (virtual machine (VM) to virtual ma-chine);4. resource allocation per VM via VMM hyper-call interface;5. resource allocation within the VM (e.g., viahardware page-table virtualisation);6. resource re-allocation (e.g., via page flipping);7. page-fault and exception handling via exceptionvirtualisation;968. asynchronous event notification across domainsvia virtual-interrupt signalling mechanism;9. hardware interrupt notification via virtualizedinterrupt controller;10. a set of common devices, such as NIC and disk.The interfaces provided by the VMM have an in-triguing benefit for an important class of highly com-plex software: existing operating systems. Avail-able operating systems already program to the inter-face provided by the hardware and resembled by theVMM. Thus existing operating systems require no oronly minimal changes to run on a VMM, whereasadaptation to the microkernel primitives often re-quires significant modifications. However, this ben-efit is being eroded by the increasing divergence ofVMMs from pure virtualisation (faithful representa-tion of the underlying hardware) to paravirtualisation(representation of modified hardware that lends itselfbetter to efficient support of legacy OSen).The diversity of interfaces also leads to struc-tural compromises, such as centralized super-VMsthat combine and colocate significant critical systemfunctionality. Such a structure potentially decreasesoverall reliability and poses the risk of a single pointof failure. This problem becomes even more inher-ent if this super-VM runs a legacy operating systemand thus re-introduces a large number of softwarebugs [CYC+01].For extensions that are not an existing operatingsystem, the VMM’s interfaces significantly increasethe complexity of software design. As, per defini-tion, a VMM presents an interface that is close to theunderlying architecture, software developed for oneVMM is inherently unportable across architectures.In contrast, a microkernel abstracts and hides the pe-culiarities of the hardware platform behind its com-mon set of abstractions. For example, software thatis written for an L4 microkernel [Lie95] naturallyruns on nine different processor platforms, from em-bedded devices such as ARM, to desktop and smallservers such as x86, up to large multiprocessor Pow-erPC and Itanium machines. Hence, it is possible toleverage and reuse system components across a widevariety of hardware platforms, thereby minimisingporting and maintenance overhead.3 Architectural LessonsNow we reexamine the architectural lessons pre-sented by Hand et al in detail, following the headingsof their paper, and clarify the role of microkernels.3.1 Avoid Liability InversionThe paper states that moving system services outof the kernel relaxes the dependability boundarieswithin the system. Applications and even the ker-nel depend on user-level code. This situation iscalled liability inversion and an example from Mach[YTR+87] is used to argue that “inelegant” mecha-nisms are required to ensure correct system opera-tion as a consequence of the “kernel abdicating itsliability”. It is further argued that one of the princi-pal design guidelines of Xen were to avoid liabilityinversion.At the workshop, Butler Lampson was quick topoint out that this liability inversion is in fact an is-sue in Xen as well. An example for this is actu-ally given in another paper at the same workshopby some of the same authors: the Parallax storagesystem [WRF+05] essentially uses external pagersto provide file service. While that paper argues thatthe design avoids liability inversion, Parallax is “pro-viding a critical system service for a set of VMMs”.This is exactly what a user-level server does in amicrokernel-based system. The argument is madethat a failure of the Parallax server only affects itsclients — exactly the same situation as if a serverfails in an L4-based system. Hence, we fail to see thedifference between a VMM and a microkernel in thisrespect.Possibly this apparent conflict is a result of alack of understanding of microkernels (even thoughthis has been thoroughly explained in the litera-ture [Lie96]). The confusion might in fact be the re-sult of an invalid generalisation of a specific example(a particular design fault of Mach) onto a whole classof systems (microkernels).973.2 Make IPC performance irrelevantHere Hand et al. argue that, while microkernel de-signers have spent considerable effort on optimisinginter-process communication (IPC) mechanisms, thisis irrelevant as it is “not a critical design concern inthe construction of high-performance VMMs.”They further argue that IPC between virtual ma-chines is much less frequent and thus not per-formance critical, as a consequence of the VMMscheduling and protecting complete operating sys-tems.This is an interesting line of argument, as it is atodds with the reality of Xen-based systems in at leasttwo respects:• Xen uses a separate virtual machine (calledDom0) to encapsulate legacy device drivers[FHN+04]. Hence, any I/O operation impliesat least one round-trip communication betweenthe guest VM and Dom0. The authors call thisa “simple asynchronous unidirectional eventmechanism” — it is nothing else than a formof asynchronous IPC.And performance-critical it is indeed. A recentpaper [CG05] examines the CPU overhead ofDom0 drivers under high load, and finds that theCPU load generated by Dom0 accounts for al-most all of the CPU load of the system undertest! They also find that the Dom0 CPU timeis proportional to the number of Xen’s page-flipping operations, that is, message transfers,irrespective of the message size. The clear im-plication of this data is that IPC costs dominatethe driver overhead in Xen systems under highI/O load.• While it is true that Xen schedules complete op-erating systems, this does not mean that thereis no other interaction with the VMM. In fact,each guest-application exception and systemcall causes a trap into the VMM, which then in-vokes corresponding functionality in the guestOS. This is nothing but an IPC operation be-tween the guest application and the guest OS.Xen provides a shortcut based on x86’s trapgates that avoids invoking the VMM on guestsystemcalls. However, this shortcut is specifi-cally targeted and limited to Linux’s int 0x80system-call variant and restricts the use of seg-ments. Protection can only be preserved if allactive segment configurations explicitly excludethe VMM kernel. Since x86’s trap mechanismonly reloads two of the six segment selectors,the solution is limited; Linux’s latest glibc vi-olates the assumption and renders the shortcutuseless.A Xen-based system performs essentially thesame number of IPC operations as a compara-ble microkernel-based system (such as L4Linux[HHL+97]).3.3 Treat the OS as a componentUnder this heading, Hand et al. argue that a benefitof VMMs is that they are designed to run completelegacy systems, with familiar programming and de-velopment environments, and lending themselves toextensions such as Parallax. The (unstated) implica-tion of such statements has to be that microkernelsare somehow not suitable for such use.This is a really surprising notion, as L4 has demon-strated many years ago that it is perfectly suitable asa VMM supporting a paravirtualised Linux systemwith excellent performance [HHL+97], and the Dres-den DROPS system [HBB+98] is built specificallyon extending a paravirtualised Linux system runningon a microkernel with real-time services and is in in-dustrial use.Again, we fail to see the claimed “significant dif-ference” between VMMs and microkernels.4 ConclusionsIn summary, the “important differences” between mi-crokernels and VMMs identified by Hand et al. donot seem to hold up to scrutiny. As a consequence,their conclusion “that VMMs are microkernels doneright” cannot be inferred from the arguments they98presented. Yet, the observation, also made by others[HPHS04], that VMMs and microkernels are closelyrelated, deserves further attention. We believe that asystematic and objective examination of the similar-ities and differences of microkernels and VMMs isstill outstanding, and would make a valuable contri-bution to OS theory and practice.References[BDF+03] Paul Barham, Boris Dragovic, Keir Fraser,Steven Hand, Tim Harris, Alex Ho, RolfNeugebauer, Ian Pratt, and Andrew Warfi eld.Xen and the art of virtualization. In Proceed-ings of the 19th ACM Symposium on OS Prin-ciples, pages 164–177, Bolton Landing, NY,USA, October 2003.[BH70] Per Brinch Hansen. The nucleus of a mul-tiprogramming operating system. Communi-cations of the ACM, 13:238–250, 1970.[CG05] Ludmila Cherkasova and Rob Gardner. Mea-suring CPU overhead ofr I/O processing inthe Xen virtual machine monitor. In Proceed-ings of the 2005 USENIX Technical Confer-ence, pages 387–390, Annaheim, CA, USA,April 2005.[CYC+01] Andy Chou, Jun-Feng Yang, BenjaminChelf, Seth Hallem, and Dawson Engler. Anempirical study of operating systems errors.In Proceedings of the 18th ACM Symposiumon OS Principles, pages 73–88, Lake Louise,Alta, Canada, October 2001.[FHN+04] Keir Fraser, Steven Hand, Rolf Neuge-bauer, Ian Pratt, Andrew Warfi eld, and MarkWilliamson. Reconstructing I/O. Techni-cal Report UCAM-CL-TR-596, University ofCambridge, August 2004.[Gol74] Robert P. Goldberg. Survey of virtual ma-chine research. IEEE Computer, 7(6):34–45,June 1974.[HBB+98] Hermann Ha¨rtig, Robert Baumgartl, MartinBorriss, Claude-Joachim Hamann, MichaelHohmuth, Frank Mehnert, Lars Reuther, Se-bastian Schnberg, and Jean Wolter. Drops— OS support for distributed multimediaapplications. In Proceedings of the 8thSIGOPS European Workshop, Sintra, Portu-gal, September 1998.[HHL+97] Hermann Ha¨rtig, Michael Hohmuth, JochenLiedtke, Sebastian Scho¨nberg, and JeanWolter. The performance of µ-kernel-basedsystems. In Proceedings of the 16th ACMSymposium on OS Principles, pages 66–77,St. Malo, France, October 1997.[HPHS04] Michael Hohmuth, Michael Peter, HermannHa¨rtig, and Jonathan S. Shapiro. Reduc-ing TCB size by using untrusted compo-nents —small kernels versus virtual-machinemonitors. In Proceedings of the 11thSIGOPS European Workshop, Leuven, Bel-gium, September 2004.[HWF+05] Steven Hand, Andrew Warfi eld, Keir Fraser,Evangelos Kottsovinos, and Dan Magen-heimer. Are virtual machine monitors mi-crokernels done right? In Proceedings ofthe 10th Workshop on Hot Topics in Operat-ing Systems, Sante Fe, NM, USA, June 2005.USENIX.[Lie95] Jochen Liedtke. Improved address-spaceswitching on Pentium processors by transpar-ently multiplexing user address spaces. Tech-nical Report 933, GMD SET-RS, Schloß Bir-linghoven, 53754 Sankt Augustin, Germany,November 1995.[Lie96] Jochen Liedtke. Towards real microkernels.Communications of the ACM, 39(9):70–77,September 1996.[WRF+05] Andrew Warfi eld, Russ Ross, Keir Fraser,Christian Limpach, and Steven Hand. Par-allax: Managing storage for a million ma-chines. In Proceedings of the 10th Workshopon Hot Topics in Operating Systems, SantaFe, NM, USA, June 2005. USENIX.[YTR+87] Michael Young, Avadis Tevanian, RichardRashid, David Golub, Jeffrey Eppinger,Jonathan Chew, William Bolosky, DavidBlack, and Robert Baron. The duality ofmemory and communication in the imple-mentation of a multiprocessor operating sys-tem. In Proceedings of the 11th ACM Sympo-sium on OS Principles, pages 63–76, 1987.99

Are virtual-machine monitors microkernels done right?

https://publikationen.bibliothek.kit.edu/1000027247/2059152

Are virtual-machine monitors microkernels done right?

Abstract

Similar works

Full text

Available Versions

KITopen