Comer, Douglas E.

Munson, Steven B.

English

Purdue E-Pubs

Purdue University Purdue e-Pubs Department of Computer Science Technical Reports Department of Computer Science 1988 Efficient Interprocess Communication Using Shared Memory Douglas E. Comer Purdue University, comer@cs.purdue.edu Steven B. Munson Report Number: 88-744 Comer, Douglas E. and Munson, Steven B., "Efficient Interprocess Communication Using Shared Memory" (1988). Department of Computer Science Technical Reports. Paper 640. https://docs.lib.purdue.edu/cstech/640 This document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries. Please contact epubs@purdue.edu for additional information. EFFICIENT INTERPROCESS COMMUNICAnONUSING SHARED MEMORYDouglas E. ComerSteven B. MunsonCSD·1R·744February 1988Efficient Interprocess Communication Using Shared MemoryDouglas E. ComerSteven B. MunsonComputer Science DepartmentPurdue UniversityWest Lafayette, IN 47907CSD-TR-744January 1988AbstractMultitasking operating systems allow multiple application programs to execute con-currently without interference. Each task generates addresses starting at zero and the operatingsystem uses memory management hardware to map each task's addresses into a unique part ofphysical memory. The memory management unit maps all the addresses a task generates intoan area of physical memory reserved for that task; one task carmot accidentally read or alter thememory assigned to another wk. Only the operating system itself can access unmapped physi-cal memory. Isolation of tasks makes it easy to write or debug a given application program butit makes sharing memory among multiple tasks impossible. Thus, when tasks need to commun-icate, one task must ask the operating system to copy the data from its address space to anothertask's address space.This article reports the results of a research project that is exploring how to supportmemory sharing among multiple tasks. The scheme described here uses convenlional memorymanagement hardware in a new way. It has all the advantages of task isolation, but allowscooperating sets of tasks to share parts of their address spaces. The chief advantage of ourscheme is that it pennits tasks to share objects that contain pointers (addresses) like linked lists,even though each task usually interprets memory addresses privately. In our scheme, a taskdynamically establishes an area of memory to be shared and grants other tasks access to thatarea. Access to objects in the shared area is no more expensive than access to objects in anyother pan: of memory.• This work was supported in part by granlS from the National Science FOWldation's Office of Ad-vanced Scientific Computing (NSF-851·8369), with additional support from Digital Equipment Corpora-tion, CSNET, AIT Foundation and Purdue Univen;ity-2-1. IntroductionIn any multitasking computer system, the part of the operating system known as the taskmanager or process manager switches the CPU among multiple tasks. It meticulously savesthe registers and machine state for one task and loads the saved machine state for another.When a multitasking system suppons virtual memory, each task, sometimes called a process Iexecutes in its own private memory address space without knowledge of the address spaces ofother tasks that execute concurrently. That is, each task uses memory addresses starting at zeroand extending upward. and the operating system arranges for the memory management hardwareto map each task's addresses into a different region of physical memory. The operating systemmust save and restore the mappings from a task's address space to physical memory, just as itsaves other machine registers, when it switches the CPU from that task to another.In a vinual memory system, each task imagines that it has all of memory to use, starting atlocation zero. As we have said, memory management hardware maps the addresses a task gen-erates into an area of physical memory reserved for that task. We say that the task generates avirtual address to distinguish those addresses from the physical or real addresses that theunderlying hardware uses. Later sections describe the mechanism most commonly used to per-fonn the mapping. For now, it is only important to understand that each application program iswritten to use addresses starting at zero, and that conventional operating systems arrange to keeptasks completely isolatcd when they execute.The key concepts of virtual memory that we will discuss in this paper are protection andcommunication. Clearly, isolating each task in a separatc address space protects the task'smemory, because other tasks cannot read or change data values. On some hardware, the operat-ing system, or kernel, executes in its own vinual address space, called kernel space, which isdistinct from that of any task. In such cases, the operating system must execute special instruc-tions to address physical memory directly.The consequence of using vinual memory for protection is that tasks cannot easily movedata among themselves. It is impossible, for example, to have one task assign a value to aninteger variable that belongs to another task. All communication between tasks must go throughthe operating system. However, placing a request with the operating system, or moving dalabetween the task's address space and the operating system's address space can be expensive.On most processors, a task must execute a special machine instruction known as a trap or sys-tem call to make such transfers. The system call instruction transfers control to the operatingsystem and makes it possible for the system to access data in the task's address space. Theoperating system copies the data into its address space and changes the memory mapping.Later, when some other task places a system call to extract the data, the operating system mustcopy the data into that task's address space.To understand how inefficient communication can be, consider an example from UNIX.The commandwho I wc-lcreates two tasks (processes), one to execuLe the "who" command and one to execute the "we"command, and connects the output of one task to Lbe input of the other. The "who" commandproduces a list of users who are logged into the computer, with one user per line. The "wc"command counts lines in its input. Thus, the effect is to count the number of users logged intothe computer. Because the two tasks each execute in their own address space, communication- 3 -between them must go through the operating system. As a result, the operating system mustcopy the data from the address space of the "who" command into the address space of theoperating system, and from there into the address space of the "we" command. In this exam-ple, the amount of data being transferred is so small iliat communication costs are unnoticeablecompared to processing costs. However, if we consider tasks like print spooters that transferlarge volumes of data. the communication costs can dominate ~e cost of the computation.Memory sharing is not new to operating systems. In the xmu operating system[Com84], the operating system and all tasks share the same address space. and they communi-cate by sending pointers to global data. V [Che83] and THOTH [Che79] allow tasks to begrouped into teams, and, within a team, all tasks share their whole address space. TENEX[Bob72] has system calls that change a task's address mapping so that different tasks can sharememory. DEMOS [Bas77] uses links for communication which allow the communicating tasksto read or write sections of each other's memory. Accent [Fit86] has a system call called Move-Words, which moves data between tasks or between a task and the operating system by sharingthe memory the data occupies until one of the tasks attempts to write in the shared memory, inwhich case the operating system traps the attempt and makes a copy first.2. Hardware Page TablesThe hardware memory management unit (MMU) perfonns two functions that are essentialto memory sharing; it maps addresses used by a program into addresses in physical memory,and it provides access protection. It perfonns both of these tasks using page tables. Each taskhas its own set of page tables, each of which describes a segment of its address space. A pagetable is simply an array of page table entries. each of which describes how to map addresseswithin a fixed range, called a page, onto a section of real memory, called a frame. Hardwaredesigners determine the choices of page sizes (ie., the range of addresses controlled by a singlepage table entry) when they build the MMU. Usually, they choose to make the page size apower of 2 bytes (e.g., 512, 1024 or 4196 bytes). A page table entry must specify the addressof the frame in which the system has chosen to store the page. Since frames have to start at anaddress which is a multiple of the page size, only the multiple, called the frame nwnber, has tobe stored in the page table entry.The page table entry must also specify what access is allowed for each page. The accesscontrol mechanism allows the operating system to specify, for example, that a given page isread-only, or that it can only be accessed by the operating system.Figure 1 shows one of the page table entry fonnats for the Motorola MC68851 PMMU,used with the popular Motorola MC68020 processor. The page size for the MC68851 issoftware selectable, and must be a power of 2 from 256 bytes to 32K bytes. Hence, the framenumber never occupies the low-order 8 bits of !.he page table entry, and might occupy less than24 bits.3. Implementation of Shared MemoryIt might seem that a paging system makes memory sharing trivial. To share an object, lhesystem arranges to have entries in two different tasks' page tables point to the same frame inreal memory. However, if the shared memory contains structured objects like linked lists, theshared memory must appear at the same address in the address space of all tasks that share it.To achieve this goal, our sharing scheme requires the operating system to reserve a fixed seg·ment of the address space of each task. We call the reserved space the shared segment. Tasks- 4 -~Q ~~ 45~ ~41~D38~J6~J4nn~r-------L-im-i-'-F~-'-N-=-",-,--I RAL I WAL31------------------- 8 7 0Supervisor (S):Modified (M):Used (U):Write Protect (WP):Frame Number:Access restricted to supervisor mode if S=1.Page has been written to if M=l.Page has been referenced if U=1.No writing to this page allowed in any mode if WP=1.Most significant 24 bits of page frame address.Figure 1. Motorola MC68851 page table entry fannat. Each entry tellshow to map one page into a frame in physical memory.get access to the shared segment by asking the operating system to allocate pieces of it, calledshared objects. The operating system keeps a table that describes the state of the shared seg-ment and modifies the tasks' page tables to give them access to shared objects. Tasks acquireaccess rights to shared objects by allocating them, receiving them from another task, or inherit-ing them from their creating task. Tasks invoke system calls to allocate and delete sharedobjects, give access rights to other tasks, and alter the length and access rights of a sharedobject.In order to prevent shared objects wilh different access rights from overlapping the samepage, the operating system allocates memory for all shared objects in multiples of the page sizeused by the hardware. The operating system keeps all the information about the shared segmentin the shared segmeru frame table, which has one entry for each page frame in the shared seg-ment. An entry in the table contains a frame number, a length, and a reference count. Forframes that are part of a shared object, the frame number points to the first frame of the object,the length is the total length of the object, and the reference count of the first frame in theshared object is equal to the number of tasks lhat have access to the object. There is also a freelist of blocks of frames; the first entry in each block contains the frame number of the next freeblock, the length of the block, and a reference count of zero. Figure 2 gives an example of aframe table with a shared object allocated that takes up three frames. Notice that the referencecount field of the first entry for the shared object contains the actual reference count, and thoscfor its other frames are only non-zero to indicate that that frame is part of an allocated object, sothat the reference count can be updated efficiently. Also, lengths are in bytes, so that a sharedobject might not be as large as the space allocated for it, which is rounded up to the nearestpage size.A shared object can be marked wifu the access rights read or write. The access righLseach task has to an object are kept in its page table, and the operating system only implemcntsthe access rights supported by the hardware. Hence, access rights are checked directly by thehardware with every access, and the operating system only has to intervene when an access vio-lation occurs.A task allocates a shared object by executing the systcm call:- 5 -I free -'/frame number length , ref count 0frame number length -r ref count - 0rr. frame number length ref count - 2~frame number length ref count - 1frame number length ref count I... frame number length ref count - 0frame number I length ~ I ref count - 0Figure 2. Shared segment frame table.obj =shalloc(size. access),where size is the requested size of the shared object in bytes and access specifies the requestedaccess rights. Tasks communicate by passing access rights to shared objects. Access to ashared object can be given to another lask with the system callshsend(pid. obj, access).This gives task pid access rights to the shared object obi speci fled by the intersection of accessand lhe access rights of the calling task. When a task is ready to receive a shared object. it exe-cutesobj =shreceive(pid),which blocks until access to a shared object is sent to the calling task by task pid, or by anytask if pid is the constant ANYPROCESS I and then returns the address of the shared object. Anerror value is returned if task pid does not exist or dies before it sends a shared object.When a task is finished using a shared object, it callsshdeleLe(obj),which makes the object inaccessible to the calling task, decrements its reference count, and addsit to the free list if the count goes to zero. An additional system call,obj2 = shexchange(pid, objl, access),is equivalent toshsend(pid, objl, access);shdeleLe(objl);obj2 = shreceive(pid),and is added for convenience. There are also functions that set and retrieve the length andaccess mode of a shared object:- 6 -shsellen(obj, len);len = shgetlen(obj);shsetaccess(obj, access);access = shgetaccess(obj).A task can shorten a shared object. but CaIUlot lengthen it, and it can restrict its own accessrights, but cannot extend them.4. More Efficient CommunicationWe now show some examples of communication using the the shared memory primilives,beginning with a task sending data to a print spooler. The client makes itself known to theserver by sending its task id in a shared object. The server just waits for clients to send itshared objects and then spools data to print by exchanging shared objects with the client andwriting the data in them to the spool area. Here is what the server and client might look like:-7-printspoolerQ(char "'bur;int clienlpid;while (TRUE) (buf = shreceive(ANYPROCESS);clientpid = *(int *) buf;while «but = shexchange(clientpid, buf, RW» != NULL&& buf != ERROR)spool(buf, shgetlen(buf));client(fd, serverpid)int Cd, serverpid;(char *buf;j* file to print and server pid */buf = shalloc(BUFLEN, RW);O(int O)buf = getpidO;shsend(seIVerpid, buf. RW);huf = shalloc(BUFLEN, RW);while ( (len=read(fd, huf, BUFLEN)) = BUFLEN)buf = shexchange(serverpid, buf, RW);if (len> 0) (shsetlen(buf, len);buC = shexchange(servcrpid, buf, RW);}shdelele(buf);buf = shexchange(serverpid, NULL, 0);shdelete(buf);Very little copying is done in this example. The server passes to its print spooling roUline thevery same buffer that the client used to read the data. Hence, the copies from the client addressspace into a buffer in kernel space, and from kernel space into the server address space are elim-inated.The shared segment can also make writing data to a disk file faster. One might considerthis to be a slow operation anyway, because it involves disk I/O, but the write function does notactually wait for the data to be transferred to lhe disk. Instead, the operating system places thedata to be written in a queue of disk I/O requeslS and returns control to the task, Irnowing thatthe disk driver will eventually write the data to the disk. Hence, copying the data from thetask's address space to kernel space adds significantly to the cost of the operation. To make thewrite faster, the operating system can include a new system call,shwrite(fd, buf,len),-8-which adds the shared buffer, bul. to kernel space and removes it from the task's address space,eliminating the copy to kernel space.S. ConclusionOne of the hidden costs in an operating system is the time spent copying data. Communi-cation in a multitasking, virnlal memory system, whether between tasks or between a task andthe operating system, usually requires copying to get data from one address space [0 another.One way to eliminate the need for copying between address spaces is to provide a mechanismby which sections of memory can be shared among tasks and the operating system. Using con-ventional memory management hardware, we have proposed simple system calls that make itpossible to share memory among different address spaces while still providing protection fromnon-cooperating tasks. Examples have shown the elegance and efficiency of the shared segmentprimitives.Some preliminary work has been done to implement a prototype system that uses theshared segment concept. At the Department of Computer Sciences at Purdue University, the4.2BSD UNIX kernel and linker have been successfully modified to insert an inaccessibleshared segment at the beginning of each task's address space, shifting the program text, dam,and heap upward. A team of graduate students has written a version of the XINU operating sys-tem which uses the hardware memory management on a Digital Equipment Corporation Micro-VAX to implement virtual memory and a shared segment through which tasks can sharememory. The experiments show that lhe proposed mechanism is easy to implement and that itworks well in practice.6. REFERENCES[Bas??J F. Baskett, J. H. Howard, and 1. T. Montague, "Task Communication in DEMOS",In Proceedings of the 6th Symposium on Operating Systems Principles, (PurdueUniversity, Nov. 16-18), ACM, New York, 1977, pp. 23-31.[Bob72] D. G. Bobrow, J. D. Burchliel, D. L. Murphy, and R. S. Tomlinson, "TENEX, aPaged Time-Sharing System for the PDP-IO", CACM 15, 3 (March 1972), pp.135-143.[Che?9] D. R. Cheriton, M. A. Malcolm, L. S. Melen and G. R. Sager, "Thoth, a PortableReal-Time Operating System", CACM 22, 2 (Feh" 1979), 105-115.[Che83J D. R. Cheriton and W. Zwaenepoel, "The Distributed V Kernel and its Perfor-mance for Diskless Workstations", In Proceedings of the 9th Symposium onOperating Systems Principles, (Bretton Woods, NH, Oct. 11-13) ACM, New York,1983.[Com84] D. Comer, Operating System Design, the Xinu Approach, Prentice-Hall, EnglewoodCliffs, New Jersey, 1984.[Fit86J R. Fitzgerald and R. F. Rashid, "The Integration of Virtual Memory Managementand Inlerprocess Communication in Accent", ACM Transactions on Computer Sys-tems 4, 2 (May 1986), pp. 147-177.

Efficient Interprocess Communication Using Shared Memory

https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1639&context=cstech

Efficient Interprocess Communication Using Shared Memory

Abstract

Similar works

Full text

Available Versions

Purdue E-Pubs