KFA started in May 95 to use the advanced tape MSP under DMF 2.2.6. Approximately 200,000 files with about 500 GB of data were moved from the old MSP to the new one. Practical experiences gained within the process are given and critical steps are described in this paper

Naun, Birgit

Romberg, Mathilde

Juelich Shared Electronic Resources

FORSCHUNGSZENTRUM J ¨ULICH GmbHZentralinstitut fu¨r Angewandte MathematikD-52425 Ju¨lich, Tel. (02461) 61-6402Interner BerichtExperiences with theDMF Advanced Tape MSPBirgit Hellwich, Mathilde RombergKFA-ZAM-IB-9532Oktober 1995(Stand 01.10.95)In: Proceedings of the CRAY User Group Meeting (Fall), 1995, pp. 141-144Experiences with the DMF Advanced Tape MSPBirgit Hellwich, Mathilde RombergZentralinstitut fuer Angewandte Mathematik (ZAM)Forschungszentrum Juelich GmbH (KFA)Juelich, GermanyABSTRACT: KFA started in May 95 to use the advanced tape MSP under DMF 2.2.6. Approximately200,000 files with about 500 GB of data were moved from the old MSP to the new one. Practicalexperiences gained within the process are given and critical steps are described in this paper.IntroductionKFA is one of 16 German national research laboratories. Itsmain fields are structure of matter and materials research, in-formation technology, life science, environmental precautionresearch, and energy technology. Among other tasks, the Cen-tral Institute for Applied Mathematics (ZAM) manages KFA’scentral computer systems and communication networks. Thefacilities comprise an IBM ES/9000, an Intel Paragon XP/S10, an IBM SP2, and two Cray Y-MPs as well as FDDI,HiPPI, ATM, and other networks. The Cray systems are aY-MP8/864 and a Y-MP M94/4512 each dedicated to a dif-ferent group of users.KFA is using the data migration facility since 1989 to prima-rily enlarge the limited disk space. On each of the two Craysystems DMF is used on one file system, /arch, where usersshould store their large datasets. Because of limited tape ca-pacity available for the Crays at that time (part of a STK silowith 200MB 3480 cartridges) we chose to have only one copyof a dataset residing in DMF. This was at first implementedthrough a local modification, later - after its introduction - theconfiguration parameter DEFCOP was used. In addition diskquotas were established on the file systems counting onlineand offline data. This is still implemented through a local mo-dification. The data migration facility is a reliable product.During all those years we only lost data from two tapes dueto media problems.KFA is a relatively small DMF site with about 500 GByte ofdata altogether on both machines which reside on 960 3490Etapes (800MByte capacity) managed by dmtpmsp (status as ofMay 1st). The available disk space for the /arch file system is10 GByte per System. On average about 3 Gbytes of data arewritten to tape and 150 datasets are recalled a day. The oldestdatasets are 1290 days old, the average age is 517 days.Currently KFA is running UNICOS 8.0.4 with the PRIV_SUversion of Multilevel Security and DMF 2.2.6. One STKACS 4400 tape silo is dedicated to the Cray systems. It isconnected via a SUN Workstation running ACLS V3.0.Characteristics of dmatmspThe advanced tape MSP introduced with DMF 2.2 is veryvaluable for us as it removes some shortcomings of dmtpmsp.Its major advantages are the capability of absolute blockpositioning and end-of-tape processing, so that it can exploitdata compression, and that it has better recovery features incase of media problems. This leads to a more efficient usageof the tape media, especially when data compression is usedand when there is not enough data to fill up a whole tape atwrite time.The improved way of processing allows dmatmsp to be moreflexible. The data is grouped in a hierarchy of blocks, chunks,and zones. A file fitting on one tape consists of one chunk,if it is spanned across tape boundaries it has multiple chunks.A zone is a logical group of chunks. It contains as manychunks as are written to one tape in one operation. The targetzone size specified in the volume database is the amount ofdata which triggers dmatmsp to write data to tape. The zonesize is the tape size at maximum. Each block, the logical I/Ounit, contains in its header the zone and the chunk numberit belongs to, as well as its position within the chunk. Thetape catalogue database knows about the location of each filechunk by recording volume, zone and chunk number.When writing to tape the medium is always positioned to thenext available zone using absolute block positioning. A tapeis always filled up with data until EOT. Thereby the mediaare fully utilized. This is in contrast to dmtpmsp which onlyputs the predefined tape capacity of data at maximum to tapein one operation leaving the remainder of the tape empty.A disadvantage of the advanced MSP might be that it needsmore tape mounts than the old MSP. This is caused by thesmaller target zone size (50 MByte for non-ER90 tapes) incomparison to a whole tape. Therefore dmatmsp reduces thenumber of mounts by leaving tapes mounted on the drive foranother minute before dismounting them. This allows for apossible next operation to start without remounting.When a file is recalled the tape is positioned to the zonecontaining the file (the tapes are zone addressable). From thebeginning of the zone it moves forward to the first chunk ofthe file. This feature allows fast recalls of datasets even fromhigh capacity tapes. It also enables DMF to recall all dataof a damaged tape except the chunk, in worst case the zone,1containing bad blocks. Thereby less data will be lost in caseof media problem than with tapes managed by dmtpmsp .With dmatmsp a set of new commands is introduced. Theyare mainly to support the advanced MSP and add integritychecking as well as backup tools:dmvoladm Administrative services for the tape volume da-tabase – relates to dmvdbgen and dmtpmerge ofthe old MSPdmcatadm Administrative services for the tape catalog data-base – relates to dmdbvaldmatread Copies files from MSP tape to disk, can be usedfor both tape MSPsdmcopy Copies a byte-range of a file from MSP tape todisk without recalling the entire filedmatsnf Verifies the integrity of (old and new) MSP vo-lumesdmatvfy Tool to correlate the contents of the dmatmspdatabase with the DMF databasedmsnap Takes snapshots of all data migration databasesfor backup purposes without interrupting DMFoperationsdmastat Reports DMF activity statistics; it needs theLOG_ACTIVITY parameter to be set in the datamigration configuration filedmatcvt Utility to convert the databases of the old MSPto the ones of the advanced MSPMigrating to the Advanced Tape MSPIn May ’95 we evaluated the advanced tape MSP to exploitthe data compression capability of the 3490E tape controller.Because of SFN #1155 from April 10, 95, which says thatthere is a risk of loosing data of multireel files when conver-ting the MSP databases, it was decided to build the advancedMSP from scratch. This decision implied that the data had tobe moved from one MSP to the other by hand.We wanted to test dmatmsp with only a few selected useridsfirst and then enlarge the group of users stepwise.I. First Trial to ConfigureFirst of all the DMF configuration needs to be extended bythe new MSP. In addition the archmed value in the UDB,which defines the MSP to be used, has to be specified for allusers who should use the new MSP. The archmed value andthe DEFCOP value (giving the number of copies of a datasetresiding in DMF) are used to determine the index of the MSPin the dmf_config file which manages the data for the particularuser. The default value for archmed is 0, which selects thefirst MSP (cart1 in our case). For archmed set to 1 anotherMSP has to be selected via its index in the DMF configurationfile. The DMF Administrator’s Guide (SG-2135) gives thedefault algorithm for the index calculation for the MSPs to beused in form of the following table:archmedDEFCOP 0 1 2 n1 1,0 3,0 5,0 2n+1,02 1,2 3,4 5,6 2n+1,2n+2This algorithm is not suitable for a site using DEFCOP=1 andwanting to use two MSPs configured byMSP_NAMES cart1 cart2.Obviously, the algorithm assumes that there is a MSP definedwith index 3 in case a user has the archmed value 1 (seethe marked field in the figure above). We verified during ourtests that the second MSP could never be referenced . Thealgorithm is designed only for sites using DEFCOP=2. Tobe able to address cart2 one solution is to add an additional(dummy) MSP to the DMF configuration:MSP_NAMES cart1 dummy cart2.This definition allows to use both MSPs, cart1 and cart2,but results in error messages at startup saying ’VOL database/usr/dm/msp/dummy/tpvol does not exist’ and, after creatingone and adding a tape to it ’Only 1 free migration tapes remainin VOL database ...’.II. Second TrialAn alternative solution is documented in the DMF Admini-strator’s Guide: Modify the function dmmfunc.c to implementa site specific index algorithm. The dmmfunc source is avai-lable to all DMF sites. It is used to implement the calculationof the primary MSP index. Unfortunately, the manual doesnot document the dependencies of the routine, i.e. which partsof DMF are using it. So we modified it by simply setting theprimary index to archmed+1 in case of DEFCOP=1 leavingthe rest unchanged. This worked for all automatic operationsbut failed in case a user explicitly requests two copies of afile with a dmput command not knowing just one is allowed.In this case the user gets the prompt without any error or war-ning message but his/her request is not performed, there is noteven one copy of the file in DMF. Therefore we extended themodification by specifying archmed+1 for the primary indexin both cases, 1 and 2 copies. From our point of view it isnot a good idea that one has to modify a routine being leftblind about the consequences, detailed documentation and auser exit would be far better.The following shows a part of the DMF configuration file weare running with:DEFCOP=1MSP_NAMES cart1 cart2cart1 MSP_TYPE tape COMMAND dmtpmspcart2 MSP_TYPE tape COMMAND dmatmspNTAPE DEVICE_TYPE 3490e \MOUNT_OPTIONS -l\ sl\ -i\ on2III. Privilege SetupThe configuration above still requires additional prerequisitesas we experienced when first wanting to access tapes in thenew MSP. With the mount options also used for tapes ofcart1 we are not able to access the standard label tapes incart2: The tape message says’Flaw Tape Msg: reason=1, msg=process_mount_tape: mo-unt tape error:vsn=...’ andMSP Tape Info: vsn=(NULL), bs=65536, type=NTAPE, la-bel=blp, ...For some unknown reason dmatmsp always wants to mountits tapes with labelbypass label. So, root needs to have thelabelbypass permbit to be able to access tapes in cart2. Inaddition root also needs the tape-manage permbit defined inthe UDB, otherwise dmatmsp cannot position to an absoluteblock address on a tape:’Flaw Tape Msg: reason=6, msg=position_tape_to_zone:Could not position to absolute address’.After providing all these prerequisites, root privileges andindex calculation algorithm, we started to use dmatmsp for afew selected users. All other users can access their migrateddata and new files of the selected users are migrated bythe advanced MSP while they still can access their old datamanaged by dmtpmsp.IV. Moving the DataWith the intention to replace dmtpmsp by dmatmsp step bystep we set archmed to 1 for all users in the UDB. The newMSP then manages all the new data on the /arch file systemwhich needs to be migrated. Access to old datasets is stillpossible via dmtpmsp. This is transparent to the users.dmselect offers the functionality to select datasets from givenMSPs according to a minimal or maximal size, age and/orownership. The chosen datasets can be passed on to thedmmove command which then internally recalls the data,migrates it to the specified MSP, changes the DMF database,and removes (hard-deletes) the entry in the “old” MSP tapedatabase. For recalling the data dmmove uses space on the filesystem specified by the MOVE_FS configuration parameter.The working space on the file system can be limited byspecifying a maximum of Mbytes with the –s parameter.For KFA the appropriate selection criterion was the userid.We started with the top users, occupying tenth of GBytes andended with the numerous having just a few Megabytes. Forthe top users the dataset age was added as selection criterion,which helped to restrict the amount of data to be moved inone step.dmmove recalled all the selected datasets in a very time con-suming process: The data of a single user was spread overmany different tapes in the old MSP which all must be mo-unted and read. This was really an endurance test for thetape equipment. Moving all the data from cart1 to cart2 tookus about three month during working hours having four tapedrives reserved on each machine.In our opinion it would be good to have an option to movethe data tapewise, this would shorten the move process con-siderably.During the process we had the problem that the MOVE_FSfilesystem was flooded with recalled datasets. The recall ofdatasets was aborted when the file system ran out of space.We didn’t loose any data because dmmove proceeded writingto tape until the working space was cleared. To prevent thissituation we used the –s option to restrict the working spacein the following time.As the data was moved on a per user basis we now have the“old” data of a user grouped all together on the same tapes.It is advantageous if a user recalls multiple datasets with asingle dmget command. Otherwise you see a mounted tapebeing winded forward and backward over a long period. Withthe old MSP this burden was normally spread across severaltapes.Having a dmselect option to move data on a per tape basiswould have shortened the move process considerably. Relatedto SFN #1155 the database convert is working for all singletape files so that only multi-reel files must be moved usingdmmove. This would be the best thing to do for most sites.Early ExperienceBeside all the small problems we ran into we are quite satisfiedwith the new MSP. There are now about 700 GByte managedin dmatmsp together on both systems (status as of September10, 95). These data occupy about 750 tapes. Most of thetapes are filled with about 1 GByte. The compression factoris not as high as expected because the majority of files containdense binary data.Like in DMF 2.1 statistic data is gathered in dmdact.*and mspact.* day files in the DMF spool directory, if theLOG_ACTIVITY configuration parameter is set. Now thedmastat command is offered to format the collected data. Thisgives a good overview of the MSP activity as shown in thetable below. It is an extract of the data gathered on one sy-stem over a period of 20 days. The average time to positionin case of recalling data varies a lot from just a few secondsup to several minutes, on average it is about a minute.In the recall statistic given the column containing the ’Aver-age Data Xfer Rate’ is omitted due to lack of space. It variesbetween 0.1 and 3 MB/sec.Until now we don’t experience serious problems with dm-atmsp. The only problem occurring several times is an unre-covered tape error saying ’Backward at BOT’. This is alreadydocumented in SPR 93378 and fixed in DMF 2.3.3Migration Statistics_____________________MSP cart2___________Number DataOf Xfer RateVolume Requests Amount Migr. (MB/sec) Mounts______ ________ ____________ _________ ______118885 41 1915.805 MB 2.122 2118886 112 1667.807 MB 2.121 5118887 258 1671.187 MB 2.006 3118888 296 1231.000 MB 1.813 2118889 23 1266.255 MB 2.425 3118890 4 998.051 MB 1.574 2118891 3 997.342 MB 2.528 1..________ ____________Total 5817 39494.015 MBRecall Statistics__________________MSP cart2___________AverageNum. Time to MountOf Total Position TimeVolume Req. Amount Rec. (Secs) (Mins) Mnts______ ____ ___________ ...________ ______ ____118834 2 131.132 MB 29.820 0.543 2118878 1 147.386 MB 37.717 0.542 1118849 11 513.159 MB 68.222 0.633 9118705 169 0.828 MB 114.832 0.955 3118715 5 5.697 MB 19.259 0.784 2118714 11 21.476 MB 80.155 0.768 4118778 9 67.169 MB 102.742 0.631 6118804 5 291.081 MB 52.843 0.680 5..____ ____________Total 1248 17794.740 MBConclusiondmatmsp fulfils our expectations: It has a far better tape usagethan the old tape MSP and supports backup and verificationtools. It ran reliably up to now. The statistics formatted withdmastat give a good overview of the MSP activity.Looking at the configuration there are some disadvantages.Better documentation of the prerequisites for using the MSPis needed: The labelbypass and tape-manage permbits forthe userid root are required. An explanation for what reasonthese privileges are needed should be given. Furthermore thedefault index calculation algorithm is only suitable for sitesusing two dataset copies residing in DMF. Other sites have tomodify the routine dmmfunc.c and they are left blind about theconsequences. A far better approach would be to implementa well documented user exit.Offering the convert utility dmatcvt is very useful as theprocess of starting with dmatmsp from scratch and movingall the data “by hand” is very time consuming. It is onlyacceptable for small DMF sites.4

Experiences with the DMF Advanced Tape MSP

http://juser.fz-juelich.de/record/188253/files/ib-9532.pdf

Experiences with the DMF Advanced Tape MSP

Abstract

Similar works

Full text

Available Versions

Juelich Shared Electronic Resources