Makowski, D

Napieralski, A

Swiercz, B

CERN Document Server

EU contract number RII3-CT-2003-506395 CARE-Conf-06-070-SRF  SRF       A NOVEL APPROACH FOR OPERATING SYSTEMS PROTECTION AGAINST SINGLE EVENT UPSET  B. Świercz, D. Makowski, A. Napieralski  Technical University of Łódź, POLAND.     Abstract  Modern high-energy physics experiments require sophisticated and complex control systems. The control systems should be able to tolerate radiation generated by high-energy accelerators and thus reliability is important feature. Reliability of control systems depends on hardware and software quality. Hardware solutions are the most effective techniques to protect system against radiation influence. Commercial of the shelf (COTS) elements are used often and protection mechanisms are moved from hardware to software layer, due to costeffective design. This paper highlights the new approach to protect systems on software level. The protection against soft errors is assured by operating system that is transparent to other applications.     Contribution to the MIXDES 2006, Gdynia (Poland)     Work supported by the European Community-Research Infrastructure Activity under the FP6 “Structuring the European Research Area” programme (CARE, contract number RII3-CT-2003-506395) A NOVEL APPROACH FOR OPERATING SYSTEMS PRO-TECTION AGAINST SINGLE EVENT UPSETB. Świercz, D. Makowski, A. NapieralskiTechnical University of Łódź, POLANDKEYWORDS: Single Event Upset, Radiation Environment,sCore, Fault Tolerant System, Virtual Memory,IA-32Abstract: Modern high-energy physics experiments require sophisticated and complex control systems. Thecontrol systems should be able to tolerate radiation generated by high-energy accelerators and thus reliability isimportant feature. Reliability of control systems depends on hardware and software quality. Hardware solutionsare the most eﬀective techniques to protect system against radiation inﬂuence. Commercial of the shelf (COTS)elements are used often and protection mechanism are moved from hardware to software layer, due to cost-eﬀective design. This paper highlights the new approach to protect systems on software level. The protectionagainst soft errors is assured by operating system that is transparent to other applications.INTRODUCTIONX-Ray Free Electron Laser (X-FEL)[1, 2] is cur-rently designed at DESY Research Center in Ham-burg. X-FEL is a 4th generation of light source[3, 4]. Total length of the X-FEL will be 3.4 km,whereas the length of the main linear accelerator isequal to 2.1 km. Due to the complexity, distributedcontrol system is necessary to control all subsystemsof X-FEL’s modules. In the past accelerating ma-chines where consisted of two tunnels, one for themain accelerator and the other one for control elec-tronics. The accelerator was shielding using concreteand lead, therefore the electronics were not subjectedto radiation. The X-FEL consists only of one tunnelshared between accelerator and control electronics.Some part of electronic equipment (DSP and FPGAboards, embedded computers and microprocessorssystems) designed for control and data acquisitionwill be placed inside accelerator tunnel in the nearestproximity of cavities (the main part of accelerator).Parasitic radiation is produced during normal oper-ation of accelerator. Neutrons (especially thermalneutrons) can aﬀect memory and registers in digi-tal circuits and cause on illegal operation and thusloss of the system functionality [5, 6]. Single memorymalfunction is known as a Single Event Upset (SEU)[7, 8, 9]. Microprocessor systems placed inside the ac-celerator tunnel are subjected to neutron and gammaradiation. To ensure a reliable operation it is neces-sary to protect digital circuits against radiation [10].The more eﬀective way to protect control systemsagainst SEU is to use hardend computer systems de-signed for space and military application. Becauseof the accelerator complexity, number of control sys-tems, sensors and computers, electronics devices de-sign for radiation harden will be too expensive forX-FEL project. Other solution, much cheaper butmore complex, is to use commercial of the shelf com-ponents (COTS) connected with system redundancyand safety algorithms. Protective algorithms can byused on hardware layer (for example embedded in-side FPGA chips) or on software one. The authorsdecided to implement SEU-tolerant algorithms in-side the sCore operating system kernel. The sCoreis designed to work with standard computer archi-tecture such as PC computers or embedded versionsused by industry. There is known many frameworksand libraries supporting fault tolerant environmentsbut all of them requires good knowledge about SEUsnature from application programmers. Moving faulttolerance from an application level to kernel level isbetter way to protect the application against bit-ﬂiperrors. Presented approach is to use a dedicated ker-nel with embedded protection algorithms which al-low to develop and run applications that are not ableto tolerate SEUs. Fault tolerant kernels are morecomfortable for application programmers than ap-plication level libraries.THE SCORE KERNELThe kernel sCore was design during master thesis asa multitasking preemptive kernel based on the mi-crokernel architecture [11]. Operating system andkernel architecture study is the goal of sCore devel-oping. The sCore is design to work on IA-32 archi-tecture. It can be run on standard PC platform andembedded industrial computers. The sCore is writ-ten in C++ and thus portability is assured thanksto C++ abstraction layer. The sCore provides con-stant time of task switching independently of numberof tasks. Predictable time behaviour is important forreal-time application. The sCore has more proper-ties design for real-time systems. Other feature isscheduler with Round Robin scheduling policy and256 priority levels queue. The most of internal struc-tures are static and initialised during system boot-strap to ensure stability and predictable time depen-dences. The sCore kernel is divided into two version.First version uses ﬂat linear memory model and sec-ond version of sCore deliver more advance memorypolicy management based on virtual memory model.THE FIRST APPROACH TO PRO-TECT KERNEL AGAINST SEUThe ﬁrst version of the sCore kernel [13] does not re-quire Memory Management Unit (MMU) dedicatedto advanced memory protection and address trans-lation. Main memory is organized as a ﬂat memoryby sCore and all segments registers are conﬁgured toprovide access to the all physical memory (see ﬁg. 1).Access LimitBase addressAccess LimitBase addressSegment registersCSESSSDSFSGS Segment descriptorsOffsetSegment selectorLogical address4 GB0 BPhysical memoryPhysical addressStack andread−write dataCode andread−only dataFig. 1. The scheme of memory addressing in the firstversion of the sCore kernelThe ﬂat memory model allows to access the linearmemory region (on IA-32 from 0B to 4GB) by ap-plications. The ﬁrst version of the sCore uses onlytwo segments descriptors and only one privilege level.Due to segmentation policy, all linear memory spaceis divided and shared between all tasks. Describedbehaviour of sCore kernel is dangerous for systemstability and rare in modern kernels but it is nec-essary to run sCore on simpler architecture withoutMMU unit (e.g. some ARM processors).AmBeModeratorsCoresCoreconverterelectron−positronRestricted areaFig. 2. Experiment with 241AmBe and inside theLiniac2 tunnelThe MMU-less version of sCore kernel has a sim-ple protection mechanism against SEU called EDACTask [13]. EDAC Task is run periodically to scan andcorrect memory. The sCore system with EDAC Taskwas well examined during simulation with IARad-Sim [12] and experiments with 241AmBe neutronsource and inside accelerator tunnel (ﬁg. 2). Theresults of experiments showed EDAC Task was ableto correct bit-ﬂip error in memory only when seldomSEU was observed. Therefore, more sophisticatedprotection algorithm are required.MEMORY PAGING AND PROTEC-TIONThe next version of sCore kernel employs MMU unitand provided by MMU paging technique to protectmemory against SEU. Paging is a mechanism to di-vide computer memory into small parts, usually withconstant size, and allocate memory using the page asa quantum part of memory [14]. Every page can bemapped into physical memory or mass storage de-vices. Single page is used by sCore to make copies ofinformation and later to vote the correct one. It isnecessary to know virtual memory mechanism pro-vided by IA-32 technology to understand how algo-rithm works.Virtual memory overviewThe second version of the sCore kernel was based onvirtual memory abstraction. Virtual memory mech-anism is a method to provide individual virtual ad-dress space for every task. Memory ManagementUnit is required by the second version of sCore andthus sCore is not able tu run on simple MMU-lessprocessors. The scheme of address translation on theIA-32 architecture is shown in ﬁg. 3. The main mem-ory is divided equally to small parts called pages.The 4 KB page size are used by sCore on the IA-32platform. Every page is addressed by a page direc-tory and a page table entries. The main advantageof a virtual memory is the independent address spacefor every task. The same linear address can be trans-lated into a diﬀerent physical address of every task.When the access to memory address is requested,MMU unit checks if exist the entry of an address in-side a directory and a page table for a given linearaddress. If connection between a directory table ora page table and a linear address does not exist in-terrupt number 14 is generated. Interrupt number14 is called by Intel page fault (PF). The page faultinterrupt is handled by an interrupt service routine(ISR) and the connection between page directory andrequested linear address is associated.Avil GPage base addressPATD APCDPWTU/SR/WP31 0Fig. 4. A page table entryApart from a page address, an additional attributesare stored in the page table entry (ﬁg. 4). From SEUsprotection point of view, the most interesting ﬂagsare:• Accessed (A) — Indicates whether a page hasbeen accessed.Access LimitBase addressAccess LimitBase addressEntry EntryLinear addressDir Table OffsetPage directory Page Tableread−write dataStack andCode andread−only dataPagePhysical memoryPhysical address4 GB0 BSegment registersCSESSSDSFSGS 0 B 4 GBspaceLinear addressLinear addressSegment descriptorsOffsetSegment selectorLogical addressFig. 3. The scheme of memory addressing in the second version of the sCore kernel• Cache disable (PCD) — When the PCD ﬂag isset caching of page is not allowed. The issue ofcache will be discuss later.• Read-write (R/W) — Indicates whether a pagecan be written or page can be only read.• Present (P) — Most important ﬂag for dis-cussed SEUs tolerant algorithm. When the ﬂagis set the page is in the main memory (in phys-ical sense) and task is allowed to use it. Whenthe ﬂag is clear and some task tries to use it apage-fault exception (PF) is generated by pro-cessor. This ﬂag is mostly used to realize swaptechnology by modern operating systems butsCore used it for SEUs protection. The presentﬂag is always set to zero by sCore and interruptis generated whenever page is accessed.The address translation manner is the main diﬀer-ence between ﬂat memory model and virtual memorymodel. The physical address is reﬂected by a logi-cal address (one to one) in the ﬁrst version of sCore.The second version of kernel allows to translate anylogical address to a physical address. This approachprovide a mechanism to realize a complete micro-kernel architecture by separating a address space forevery modules and tasks.The fault tolerant algorithmThe SEUs tolerant algorithm introduced by the sec-ond version of sCore is based on the memory pagingmechanism. When the system is initialized (and alsowhen new task is run or in general when a new blockof memory is granted) all used pages are copied totwo independent memory regions. All pages havecleared a present bit (ﬁg. 4) in every page directoryand table entries. When any task tries to use mem-ory (code or data), page fault exception is generatedby processor. The interrupt procedure (ISR) com-pares requested page and its copies using a triple vot-ing technique. If contents of page is diﬀerent fromcontents of its copies the correct value is restored.When the task is preemptive by system scheduler allpages with set RW bit, used during running time,are copied to copies memory region. The describedmechanism guarantee that task has always valid con-tents of page (SEUs area corrected by ISR procedure)and the page is synchronized with its copies. Thealgorithm of protection against bit-ﬂip errors insidememory are shown on ﬁg. 5 This approach is more ef-ﬁcient and reliable, but also slow, compare to EDACTask. In simple words, EDAC Task in second versionof sCore is moved from system task to ISR procedure.ISR procedureTriple votingPage presentin memory?Memory requestSet pageCompare page Equal?Copy the correct pageYesNoInterrupt (Page Fault)NoYeswith its copy.Fig. 5. A radiation protection algorithmThe inﬂuence of cache memoryThe cache memory is an inseparable part of modernprocessors. Eﬃciency speed up is the goal of cachememory. A large size of cache is strongly requiredto decrease the number of missing hits in a standardapplication but not in the sCore kernel. Cache mem-ory causes an incoherence between main memory anddata used by processor when SEU is observed. Be-cause of described cache behaviour it is not recom-mended to use cache memory with radiation tolerantsystem, like sCore. The second version of sCore dis-able cache memory when ever is possible.THE FUTURE WORKThe future work should will concentrate on experi-ments with the second version of sCore kernel. Theeﬃcacy of new algorithm is expected to show by ex-periments compare to EDAC Task. It is very im-portant to characterize the policy of cache memorymanagement. Cache memory behaviour can be or-ganized in many ways due the advantages of IA-32architecture. The good cache policy should providenecessary a memory coherency and a good eﬃciency.The future work will be concentrated on the multi-processor architecture, standard Symmetric Multi-processing (SMP) as well as custom architecture de-sign from COTS elements.CONCLUSIONThe second version of sCore provide many advan-tages in compare to the ﬁrst version based on EDACTask. EDAC Task can be used solely to protect read-only data. The sCore run EDAC Task in parallel toother tasks. Moreover, there is no time relation be-tween memory checked by EDAC Task and memoryused by other tasks. Very often EDAC Task comparediﬀerent memory region than the one used by usertask. The second version of sCore guarantee thatmemory region used by user task is always checkedby ISR procedure, due to a small memory granularitycalled paging. The virtual memory technique allowsto protect sCore also read-write data. Software fault-tolerant techniques will be more important in thefuture because of microelectronics progress. Tran-sistors are smaller and therefore digital circuits aremore susceptible to SEU and other errors, like elec-tromagnetic interferences. Described SEU-toleranttechnique can be used not only for physics experi-ment but also for medical devices and avionics.ACKNOWLEDGEMENTSWe acknowledge the support of the EuropeanCommunity-Research Infrastructure Activity underthe FP6 "Structuring the European Research Area"program (CARE, contract number RII3-CT-2003-506395), and Polish National Science Council Grant"138/E-370/SPB/6.PR UE-DIE 354/2004-2007".THE AUTHORSBartłomiej Świercz, Dariusz Makowski and AndrzejNapieralski are aﬃliated to the Department of Mi-croelectronics and Computer Science, Technical Uni-versity of Łódź , Poland.e-mail: swierczu@dmcs.plREFERENCES[1] A. Schwarz, "The European X-Ray free electronlaser project at DESY", 26th International Free-Electron Laser Conference, pp. 85-89, Agust2004.[2] W. Shi, "SASE X-Ray Free Electron Laser InDESY", Journal of the Society of Chinese Ph-ysists, vol. 6, pp. 5-16, December 2000.[3] R. Brinkmann, K. Flottmann, J. Rosbach, P.Schmuser, N. Walker, H. Weise, "TESLA Tech-nical Design Report - The Accelerator, part II",DESY, 2001.[4] G. Materlik, T. Tschentscher, "TESLA Tech-nical Design Report. The X-Ray Free ElectronLaser, PART V", DESY, 2001.[5] D.Makowski, M. Grecki, B. Mukherjee, B.Świercz, S. Simrock, "Radiation Tolerant Sys-tem for Neutrons Measurement", 12th MixedDesign of Integrated Circuits and Systems,MIXDES, July 2005.[6] G. Messenger, M. Ash, "The Eﬀects of Radia-tion on Electronic Systems", ISBN 0-442-25417-2. Van Nostrand Reinhold Company Inc., 1986.[7] R. Peterson, "Radiation-induced errors in mem-ory chips", Brazilian Journal of Physics, vol. 33,nr 2, pp. 246-249, June 2003.[8] F. Giustino, "Radiation Eﬀects on Semiconduc-tor Devices", PhD thesis, Politecnico di Torino,March 2001.[9] D.M. Fleetwood, H. A. Eisen, "Total-dose ra-diation hardness assurance", Nuclear Science,IEEE Transactions, vol. 50, pp. 552-564, June2003.[10] D. Makowski, B. Świercz, M. Grecki, A. Napier-alski, "Projektowanie systemów niewrażliwychna wpływ promieniowania na potrzeby akceler-atora X-FEL" (in Polish), Elektronika - Kon-strukcje, Technologie, Zastosowania, nr 7/2005.[11] B. Świercz, D. Makowski, A. Napieralski, "ThesCore - Operating System for Research of Fault-Tolerant Computing", 12th Mixed Design of In-tegrated Circuits and Systems, MIXDES, July2005.[12] B. Świercz, D. Makowski, A. Napieralski,"IAradSim - IA32 architecture under high radi-ation environment simulator", 2005 NSTI Nan-otechnology Conference and Trade Show, Nan-otech 2005, Smart Sensors and Systems[13] B. Świercz, D. Makowski, A. Napieralski,"Research of Fault-Tolerant Computing UsingCOTS Elements", 2006 NSTI NanotechnologyConference and Trade Show, Nanotech 2006,Smart Sensors and Systems[14] Intel Corporation, "IA-32 Intel ArchitectureSoftware Developer’s Manual", Volume 3: Sys-tem Programming Guide, 2004

A Novel Approach for Operating Systems Protection Against Single Event Upset

Abstract

Similar works

Full text

Available Versions

CERN Document Server