The drastic increase in the data requirements of scientific applications and collaborative research has resulted of transferring a large amount of data among participating sites

Haron, Fazilah

Sodhy, Gian Chand

Sumari, Putra

Yong, Chan Huah

English

Repository@USM

{AbstractThe drastic increase in the data requirements ofscientific applications and collaborative research hasresulted of transferring a large amount of data amongparticipating sites. The heterogeneous nature of massstorage systems employed by the dffirent sites in Gridenyironment has made the data transfer among them adfficult problem. The general tendencies of datatransfer are lacking ofcommon interface betyveen nodes,using a traditional simple scripts, dumping data to tapesand even using postal service. ll/e introduce aframework of data management in transferring the databetween heterogeneous mass storage in Gridenvironments. The system is capable of finding thesuitable routes for data transfer between remote nodeswithout much user intervention.1. IntroductionIn recent years the amount of data forcollaborative research such as in scientific applicationshas increased dramatically. They grow in petabyte andterabyte capacity. The amount of data that has to betransferred between remote participating sites is alsoincreased. The success of transferring data at the righttime and place is indeed important for the successful ofcomputational job. As Grid evolves and data are storedat different sites, a smooth transferring mechanism isnecessary. Mobile protocols such as GridFTP[l], Disk-router and even FTP have been developed to support thetransfer of data between sites. However in many cases,due to the lack of a common interface and the know-howto perform high performance bulk data transfers,researchers have resorted ofsending data either by usingpostal service in tapes form or simple script and manual.Mass-storage system protocols on the other hand aredesigned for local-area access and may not work well inthe wide-area coverage. The resource storage manager(SRM), which acts as a middleware component that caninteract with operating system and mass storage systemto peform file archiving, file staging and filetransferring are limited to a small coverage of network.The SRM has advantages in coordinating local storageresources such as local policy administration, shared0-7803-8482-2t 04 t520. 00 02004 | E EE.,r'/:./ /t,1' +^irrIti;:ttfresources monitoring and client request managementshould be applied to Grid-based mass storageenvironment. SRM normally is designed to handle localstorage management and limited work has been done ininteracting and exchanging information among remotesSRM. Indeed SRM plays an important role in Gridenvironment. There is no documented work ondesigning a schematic data transfer management for aGrid-based mass storage environment. In this work, wepresent a framework of data transfer between remotenodes in Grid environment. We introduce a SRM systemthat is embedded in Grid-based system which is capableto communicate with other remote nodes to find thesuitable route for data transfer between them. Oursystem communicates with other nodes and able tomake decision in finding the suitable cache nodes forrouting data from source to destination. The systemminimizes the problem of common interfacecompatibility amongst nodes during data transfer.Furthermore the transfer is carried out without muchadministrator intervention.Section 2 outlines related studies. Section 3presents our proposed data transfer management system.Section 4 discusses on experimental settings andexpected results, and finally section 5 is the conclusion.2. Related StudyIn [6], tapes were used in sending visualizationdata via Federal Express from Los Alamos NationalLaboratory GANL) to Sandia National Laboratory(SNL). It was faster than electronically transmittingthem via TCP over the 155 Mbps(OC-3) WANbackbone. In [4], an ad hoc data pipeline was builtbetween California and Illinois. The work was focusedon handling error during transmission. While ^tLawrence Berkeley National Laboratory, themanagement of tertiary storage for transferringmechanism of high energy physic analysis data withinlocal network was established [5].GridFTP introduced by Allcock[l] is capable oftransferring data efficiently and secure in highbandwidth wide area network. Later, a Reliable FileTransfer Service(MT) [11] was introduced on top ofGridFTP which allows byte streams to be transferred ini t (.i'L /'ili€7'r.f 11 t:r_{brtq'". , Data Transfer Management in Grid-based Mass Storage Environment;l | , 1..'Putra Sumari,Fazilah Haron, Gian Chand Sodhy, Chan Huah yongSchool of Computer Sciences,Univ er siti S ain s M al ay s ia,I1800 Penang, Malaysia{putr as, fazil ah, s o dhy, hy ch an@c s. u s m. my})a reliable manner. RFT can handle a wide variety ofproblems such as dropped connections, machine reboots,and temporary network outages automatically viaretrying. GridFTP and RFT are example tools that can beused to move data between systems which support theirinterface, but they can not be used to move data betweenheterogeneous storage systems with no (or minimal)common interface.Koranda introduced Lightweight Data Replicator(LDR) [9] that can replicate data sets to the member sitesof a DataGrid. LDR was combined with Globus [8] andlater used for replicating LIGO [0] data. Its goal is touse the minimum collection of components necessary forfast and secure replication of data. Both, RFT and LDRused GridFTP to transfer data. These systems werecarried out on the systems, which do not support acommon data transport protocol. Our work focuses onsystem that lack of common interface between nodes. In[13] used Storage Resource Managers (SRMs) on top oftheir storage system to provide unified interfaceenvironment. Currently, a few data storage systemsworks have been established -- HPSS [12], Jasmin [3]and Enstore [7] - all support SRMs. The SDSC StorageResource Broker (SPA) 12] aims to provide a uniforminterface for connecting to heterogeneous data resourcesand accessing replicated data sets. SRB uses a MetadataCatalog (MCAT) to provide a way to access data setsand resources based on their attributes rather than theirnames or physical locations.3. Data Transfer Management SystemOur data transfer management system consists offour components: a Request Interpreter, a TransferPlanner, a Site Profiler and a Storage Resource Manager(SRM). The architecture of the system is shown infisure 1.most impodant component in the system. It determinesthe route taken to deliver data from source todestination. The transfer planner makes decision on whatroutes should be taken with the help of site profiler. Siteprofiler is the component that will communicate withremote sites to obtain information such as the sitelocation, disk space and transfer protocols. SRM isresponsible to fetch data from storage area and transferto the network according to the plan produced bytransfer planner. A simple scenario would be a clientwants to transfer file from site A to site C (see figure l).Request interpreter accepts query and interprets to obtainfile size, file location and destination address. Thetransfer planner then found that site A and C do not navea common interface. The transfer planner then scans ofall neighboring nodes of site C looking for suitablecache nodes candidate. Transfer planner picks site B as adisk cache. The SRM then fetches the data from localdisk and sends to B and then from site B to site C. SRMwill communicate to all remote nodes in coordinating thetask. The following sections describe each componentin detail.3.1 Request interpreterThe request interpreter provides a graphical userinterface with a query language format to clients forinserting the requests. It then interprets the query toobtain file name, file size, file location (disk, optical,tape), and destination address. These data are laterpassed to transfer planner.3.2 Site ProfilerSite profiler communicates to the destination site.It sends message to destination site to obtain thefollowing information (we referred as basicinformation): the common interface, transfer protocol,size bandwidth between the source site and thedestination site, space available at destination site. Apartfrom these, the site profiler will also scan all neighboringnodes to obtain basic information. This information isstored in profiler database.3.3 Transfer plannerTransfer planner is the most important componentin our data transfer management system. It determinesthe route that should be taken to deliver the data fromsource to destination. At present our system providesthree routes of data pipeline between sites. The nextsection elaborates in detail how transfer planner makesdecision.Request interpreter accepts requests and interpret them 3.3.1 Route Ito obtain query requirement such as file location, filesize, destination address and etc. Transfer planner is theSite AFigure l: Data transfer management system architecture0-7803-8482-2t 04 t$20. 00 @2004 I EEE.Let us assume that site A wants to send data to siteC. The transfer planner communicates with site profilerto check the common interface compatibility, datatransfer protocol and space availability at destinationsite. If there exist a direct interface between the two andthere is available space at site C, the task is then passesto SRM. SRM will communicate with site C for deliveryarrangement and use the underlying protocol of bothsites to send data to site C. This is shown in fisure 2.Mass storage site A Mass storage site CRffiBB BBQffi\--4E-J E +[,4 \_llEJFigure 2: Direct route between two mass stomge sites3.3.2 Route 2The second scenario would be site A wanting tosend data to site C. Transfer planner communicates withsite profiler to check the common interfacecompatibility, data transfer protocol and spaceavailability at both sites. Assuming that there is no directinterface between them, the transfer planner will searchin the profiler database for available neighboring nodesfor a disk cache. In this case site B is the only candidate.Site B is chosen simply due to the common interfacecompatibility between both sites, A and C. Furthermorethere is available space at site B which can act as acache. Then SRM is instructed to handle the transferfrom site A to site B and finally from site B to C. Thetransferring from site A to site B uses the underlyingprotocol of site A. The disk cache from site B to site Cuses the underlying protocol of site C. This is shown infrgure 3. If site B has limited storage space as a cachemedium we have to remove the files to create space forthe next transfer. The staging of transferring is builtusing DAG process as shown by figure 4.The disk cache can be selected either a neighbor ofsite A or site C. This is shown in dash arrows as infigure 3. The figure shows the selected disk cache whichis nearest to site A. The transfer mechanism is similaras outline for route that site B is selected as disk cache.Fig 4: DAG process for staging file among site B,site B and site C.3.3.3 Route 3The third scenario is site A wish to send data to siteD. The transfer planner found that there is no commoninterface between them. It then chose site B and site C asdisk caches nodes. Since site B (disk cache) is nearest tosite A and site C (disk cache) is nearest to site D, SRMtransfers the data first from site A to site B usingunderlying protocol of site A, and then from site B tosite C using the underlying protocol ofsite C and finallysite C to site D using the underlying protocol ofsite D asshown in figure 5. Due to the limited space at site B andC, the SRM of both sites have to make sure to removethe file whenever done to create space for the next file.Figure 6 shows the DAG process of staging file amongsites, site A, B, C and D.Mass storage site A Mass storage site DaffiBB BBaffi\JLJ Fl r*'-l \_lrE-JrrauflFt | | tllll^t-"-,l-{_lj_B#Site CSite BFig 5 : Dala trmsfer routes using cache node B and CMass stomge site A Mass storage site CeffiBB r\__4EJ F*--l Fl-\'-j \BBAffit'-,1 \-iE JfT- =,---i--,,,''I DRM i Site sFig. 3: Data trmsfer routest-t'"*"'"--lI rrome,ton Iusing cache nodes Bf .*.'"'-";lt"lyjTransfer xi+lFrom B to CRemove xi+lFromB0-7803-8482-2t O4t$20.00 @2004 | E EE.Fig 6: DAG process ofstaging file among foursites: site A, site B, site C and site D3.4 SRMStorage resource manager interacts with operatingsystem and mass storage system, to perform file transfer,file archive and invoke middleware components (such asGridFTP) to perform file transfer operations. SRMconsist of disk storage manager (DRM), tape resourcemanager (TRM), and hierarchical storage manager(HRM). The SRM also interacts to other sites tocoordinate the data delivery. We applied the existingmethod in the literature available to perform the task.4. Experimental settings and expected resultsIn our experiment, we will test our system betweentwo remote sites: USMnetl mass storage server inPenang and USMnet2 mass storage server in KualaLumpur. Both sites are 200 Km in distance. We alsohave set up several caches nearest to both sites. The totalof data to be transferred from USMnetl to USMnet2 isaround 3 TB (3000 files of 1.0 GB each). There is nodirect interface between USMnetl and USMnet2. Spaceavailable at caches nodes during experiment is set to l0GB. All servers have 100 Mb/s fast Ethernet cardinstalled.4.1 Performance ofdifferent routesFor route 2 delivery: data transfer used theunderlying protocol of USMnetl to cache node, thenfrom cache node to USMnet2 using the underlyingprotocol of USMnet2. For route 3 delivery: data transferused the underlying protocol of USMnetl to cache nodeof USMnetl, then from cache node of USMnetl tocache node of USMnet2 using underlying protocol ofcache node of USMnetl, and finally from cache node ofUSMnet2 to USMnet2 using the underlying protocol ofUSMnet2. For both routes we are having cache nodes atboth sides and that therefore we had control over thecache nodes in the case of we had no control of bothUSMnetl and USMnet2. It was easy to deal withinterface compatibility by having flexibility on cachenodes. For all routes average of l0 image filestransmission at one time using GridFTP with DAGprocess applied at neighbor servers. The reason is simplydue to the limited space at cache node which is only 10MB, the end to end transfer is around 46 Mb/s. When weused disk router instead of GridFTP of route 3 and wefound that we have smooth data transfer with disk routercompare to using GridFTP. This is because the autotuning of mechanism of disk router has automaticallyoptimized the workload. This has shown that addingcache nodes in between do not affect the overall sysrem.0-7803-8482-2t 04 t$20.00 @2004 | EEE.The most significant advantage ofhaving cache nodesat both sides was that we had control over those cachenodes, whereas we had no control over the USMnetland the USMnet2 mass-storage servers. To transfer datatolfrom the mass-storage servers, we need to usewhatever protocol they provide. These protocols maynot work well under certain conditions or may havecertain limitations and may not allow us to tune theirperformance. This is a drawback of for wide-areatransfers as they benefit considerably from tuning. Witha source and a destination cache node, we made themass-storage system work in the environment of local_area (where they are known to work well) and we haveability to choose an appropriate protocol for the wide-area transfer and got the ability to perform the necessarytune-up.5. ConclusionIn this paper, we have shown a mechanism of datatransfer management in mass storage system within widearea network. Our method allows a smooth data transferbetween heterogeneous systems without a commoninterface. The system automatically finds suitable cachenodes to route data delivery. We present our method asan alternative of using tapes by postal service andwriting scripts for handling data transfer and failures.We will perform an experiment to show that by addingadditional nodes the end-to-end performance does notnecessarily decrease but may in fact improveperformance if done properly. We plan to look intohandling failures in automated manner and automatictuning of our system during the data transfer as well asincorporating searching of optimal routes.6. References[1] B. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I.Foster, C. Kesselman, S. Meder, V. Nefedova, D.Quesnel, and S. Tuecke. Secure, efficient data transportand replica management for high-performance data-intensive computing. ln IEEE Mass Storage Conference,San Diego, California, April 2001.[2] C. Baru, R.Moore, A. Rafasekar, and M. Wan. TheSDSC Storage Resource Broker. In Proceeding ofCASCON T oronto, Canada, I 998[3] I. Bird, B. Hess, and A. Kowalski. Building the massstorage system at Jefferson Lab. In Proceeding of I8'hIEEE Symposium on Mass Storage Systems, San Diego,California, April 200 1.t4l T. Kosar, G. Kola, M. Livny. Building DataPipe-line for High Performance Bulk Data Transfer in aHeterogeneous Grid Environment. Technical reportUniversity of Wisconsin, Computer SciencesDepartment, 2003.4t51 L. M. Bemardo, A. Shoshani, A. Sim. H.Nordberg, Access Coordination of Tertiary Storage forHigh Energy Physics Applications. Techn'ical ..p-ort ofLawrence Berkeley National Laboratory Berkelea200O.[6] D. Koester. Demonstrating the TeraGrid _ ADistributed Supercomputer Machine Room. The Edge,The MITRE Advanced Technologt Newsletter, 6e),2002.t7l FNAL. Enstore mass storage systems.lr ttp ://www. fnal. gov/docslproductslenstore/[8] L Foster and C. Kesselmann. Globus: A Toolkit-Based Grid Architecture. ln The Grid: Blueprints for aNew Computing Infrastructure, pages 259-2l.9, MorganKaufmann, 1999.l9l S. Koranda and B. Moe. Lightrreight DataReplicator. http://www.lsc-group.phys.uwm.edu/lscdatagrid/LDR"/overview.html, 2003.tlOl LIGO. Laser Interferometer Gravitational WaveObservation. http:/lwww.ligo.caltech.edrl, 2003I l] S. Son. Network Bandwidth Regulation In Cedar,2003.[2] SDSC. High Performance Storage System(HpSS).httn ://wwtv. sd sc. edu/hpss/[3] A. Shishani, A.Sim, and J. Gu, Storage ResourceManagers: Middleware Components for Grid Storage.In nineteenth IEEE Symposium on Mass Storage System,2002[4] D. Thain, and M. Livny. The Ethernet approach togrid computing. In Proceeding of the twetfth IEEESymposium on High Performance DistributedComputing, Seattle, Washington, June 2003[15] D. Thain, J. Basney, S. Son, and M. Livny. Thekangaroo approach to data movement on the grid. InProceedings of the Tenth IEEE Symposium on HighPedormance Distributed Computing, San Francisco,California, August 2001.0-7803-8482-2t O4t$20.00 02004 I EEE.

Building the mass storage system at Jefferson Lab. In Proceeding of I8'h IEEE Symposium on Mass Storage Systems,

Demonstrating the TeraGrid _ A Distributed Supercomputer Machine Room. The Edge,

Globus: A ToolkitBased Grid Architecture. ln The Grid: Blueprints for a New Computing Infrastructure,

Performance Storage System(HpSS). httn ://wwtv. sd sc. edu/hpss/

Secure, efficient data transport and replica management for high-performance dataintensive computing.

Storage Resource Managers: Middleware Components for Grid Storage.

The Ethernet approach to grid computing.

The kangaroo approach to data movement on the grid.

The SDSC Storage Resource Broker.

Data Transfer Management In Grid-Based Mass Storage Environment.

http://eprints.usm.my/6862/1/Data_transfer_management_in_grid-based_mass_storage_environment.pdf

Data Transfer Management In Grid-Based Mass Storage Environment.

Abstract

Similar works

Full text

Available Versions

Repository@USM