D'Hollander, Erik

WANG, Q

YU, YJ

English

Ghent University Academic Bibliography

Visualizing the Iteration Space in PEFPTQi Wang Yu Yijun and Erik DHollanderUniversity of GhentDept of Electrical EngineeringStPietersnieuwstraat B GhentwangelisrugacbeTel 	Fax 	Abstract Sucient and precise semantic information is essential to interactive parallel programming In this paper we present a feasible implementation of the iteration space dependence graph and discuss relevanttechnical problems Moreover we give a further prospect of interactiveloop optimization guided by the graph IntroductionIn the past decade high performance computing has become a critical techniquefor scientists and engineers Parallel processing is considered as the essential wayto speedup massive computational application and largescale parallel architectures composed of microprocessors become a fashion To exploit parallelism andmemory hierarchy eectively for these machines the compiler must be able toget data dependence precisely and distribute the computations among the processors accordingly Unfortunately the results of recent research have been bothencouraging and disappointing	 One of principal drawback is the inaccuracy of current dependence analysis techniques symbolic expression procedurecalls induction and reduction variables and complex control ow all of them introduce conservative assumption so as to invalidate the feasibility of parallelismA tradeo is to provide an interactive programming or optimizing environment sharing part of the responsibility with the user under the guidance of thesystem which the user can aord to This compromise is accepted widely by theuser and the researcher There have been much signicant experiments in thisarea such as Parafrase	 and Parascope	 MotivationWe devote ourselves to design a new programming environment  PEFPT	the Parallel Programming Environment for FPT the Fortran Parallel Transformer based on the research of FPT	 proposing to develop and integrateThis work was supported in part by European Community under grant ITDCand by the Ministry of Education project CHINA joint project between the universities of GhentB and FudanPRCseries of meaningful tools and methods and testing them through practiceIt is well known that most of the parallelism in a program dwells in loopsand current parallel architectures also pay great emphasis on developing coarsegrained parallelismTherefore it is vital to exhibit loopcarried dependence information Parascope provides a table to show the dependencies between statements in dierentloops This allows the user to inspect and potentially remove false dependenciesHowever a table is not very elegant to analyze the parallelism on iteration levelIn PEFPT we employ an iteration space dependence graph as a supplementillustration for the user Data dependenceThe analysis of precedence constraints on the execution of the statements is afundamental step in parallelizing the programThere are four types of data dependence 	 between two statements Sand STrue ow dependence occurs when Swrites a memory location that Slaterreads and there is no Swrite this location between Sand SAnti dependence occurs when Sreads a memory location that Slater writesand there is no Swrite this location between Sand SOutput dependence occurs when Swrites a memory location that Slaterwrites again and there is no Swrite this location between Sand SInput dependence occurs when Sreads a memory location that Slater readsand there is no Swrite this location between Sand S Iteration space and Iteration space dependence graphSuppose I  Rn i     inare the iteration indices L U     Ln Un arethe respective loop bounds	 and both Lland Ulare linear functions of iteration indices i     il so that the iteration space isI  fi     injL i U     Lni     in  in Uni     ingMoreover the iteration space dependence graph of the loops is an iteration space picture with dependence arrows which draw from the point corresponding to iteration I  i     in to the point corresponding to iterationJ  j     jn whenever there exist statements SI and SJ in the loopbody such that Sdependent on S where I  J  Analysis of the iteration space dependence graphA straight solution to get the dependence relationship between iterations is torecord all variable reference information occurred during the execution of theloops then to analyze them using the denition of dependence For each readoperation the iteration space is searched from this point until a write operation to the same memory location is found If read and write are not in thesame iteration a dependence results It is a ow dependence if the write comeslexicographically before the read otherwise it is an antidependenceOutput dependence occurs when a write operation is followed lexicographically by another write operation Static analysis of the iteration spaceAt rst we implement such an idea in PEFPT using a strategy of static simulated execution on the syntax tree The user selects particular loops and givesnecessary data especially the values of symbolic variables Throught calculating all loop indices and array subscripts for each iteration on the lexicographicorder we get a real iteration space with all reference information of variablesThen the technique described above is used to get all dependence arrows betweeniterations In order to handle complex control ow especially GOTO statementsa powerful algorithm 	 exists within FPT which converts GOTO statementsinto structured codeSometimes nested loops have large iteration distances and a loop body maybe very complex In that case the symbolic execution is time consuming andbecomes impossible when there are call statements in the loop body Runtime simulationIn order to solve these limits and deal with more general cases we propose to insert a marker for each variable reference in the source program This marker willstore the readwrite behavior during the execution After running the programthe required data are availableThe question is how to implement the marker to process the raw data quicklyat runtime and to minimize the information records needed by the postprocessorFor each analysis variable two integer shadow variables are declared with thesame dimension as the original RS read shadow and WS write shadow eachelement of them records the iteration ID which occurs last readwrite operationon this element See the tracer algorithm in gure  Generation of the test programIn order to generate the dependence test program automatically we need domore work and more information We divide all process into ve stepsInitialization For each variable A declare two shadow variables RsA and WsA andset RsAi     imWsAi     im  Input For each reference site in the loop give an instance of array As subscriptsS     Sm enclosed in nnested loops with indices I     InIF it is a read operation doSET This iteration  IS EncodeI     InIF WsAS     Sm   THENno dependenceELSEIF WsAS     Sm  This iteration THENThere exists a ow dependence arrow from iterationIS DecodeWsAS     Sm to I     InENDIFENDIFIF RsAS     Sm  THENno dependenceELSEThere exists an input dependence arrow from iterationIS DecodeRsAS     Sm to I     InENDIFSET RsAS     Sm  This iterationENDIFIF it is a write operation doSET This iteration  IS EncodeI     InIF RsAS     Sm   THENno dependenceELSEThere exists an anti dependence arrow from iterationIS DecodeRsAS     Sm to I     InENDIFIF WsAS     Sm  THENno dependenceELSEThere exists an output dependence arrow from iterationIS DecodeRsAS     Sm to I     InENDIFSET RsAS     Sm  SET WsAS     Sm  This iterationENDIF IS Encode and IS Decode are functions that convert NmN real iterationaddressan unique integer iteration ID Just see scalar variable as one element arrayFig  Iteration dependence tracer algorithm Prepare required syntax informationRecord all variable nodes of syntax tree which appear in the selected loopsand belong to both read reference set and write reference set Generate runtime test program which is appended necessary function callsGenerate a call statement with necessary information for each variable nodeon dataow order If there exists call statement in the loop body we have toduplicate this subroutine treat it as a new one then pass actual argumentswith their shadow variables together In subroutine body we do the samething and insert marker functions for all variable reference sites Of coursethere are also need a few initialization and cleanup functions Compiler and run the programUse normal Fortran compiler and predesigned library we get an executableprogram Run it and record intermediate data in a temporary le the datalike iteration iteration dependence kind memory location Post process the rsthand dataBy now we get accurate ow and output dependencies For anti dependenceit is not as precise as the former The cure algorithm see Figure  Visualize iteration space dependence graphDue to the complexity both on iteration space dependence graph itself andpresentation we only show it in two dimensions within a predened regionNow we illustrate it using a samplesee g For all records doIf it is an input dependence from iteration Ito I just insert it into a listIf it is a ow dependence or output dependence delete all elements in thelist which relate to this memory locationIf it is an anti dependence from iteration Ito I then retrieve all recordsin the list which relate to the same memory location from Jto J andconvert it to an anti dependence arrow from Ito JFig  Post process of intermediate data ExperimentsIn a sense as compared with dependence graph the iteration space dependencegraph see g   is more comprehensible to the user which summarizes thedataow restricted relationship between iterations and gives a vivid picture ofthe loopcarried data dependencies of a given nested loops It should be moreeasily accepted by the userPROGRAM gauss REAL a(100,100)         INTEGER UPBOUND(10),LOWBOUND(10),UP,INNERCOMMON  /REGION/UPBOUND,LOWBOUND,UP,INNERINTEGER r_f,w_f         INTEGER r_a(100,100),w_a(100,100)         n=10      ...         call ISINIT(1,10,1,10)         DO i = 1,n              DO j = 1,n                   call RECISG(2,i,j)      IF (j.ne.i)  THEN         f=a(j,i)/a(i,i)              call TESTFRE(1,i,j,r_a(j,i),w_a(j,i),1000,0,j+(i-1)*100)         call TESTFRE(1,i,j,r_a(i,i),w_a(i,i),1000,0,i+(i-1)*100)         call TESTFWR(1,i,j,r_f,w_f,1000,1,1)         DO k = i+1,n+1            a(j,k)=a(j,k)-f*a(i,k)            call TESTFRE(1,i,j,r_a(j,k),w_a(j,k),1000,0,j+(k-1)*100)            call TESTFRE(1,i,j,r_f,w_f,1000,1,1)            call TESTFRE(1,i,j,r_a(i,k),w_a(i,k),1000,0,i+(k-1)*100)            call TESTFWR(1,i,j,r_a(j,k),w_a(j,k),1000,0,j+(k-1)*100)         ENDDO      ENDIF            ENDDO    ENDDOcall ISCLEAR()         ENDDeclare environment variables and shadow variables.Initialization . The parameters mean the display region of iteration space.Record iteration instancesTESTFRE / TESFWR = test for read/write----------------------------------------------------6th parameter  used for IS_Decode  and IS_Encode.7th parameter  represents the variable address of the id_table.8th parpameter is the subscript expressionCleanup functionFig  The source code of Gauss test programFig  The iteration space dependence graph of Gauss on variable f consider loopI and J with bounds    In order to help the user to understand the semantics and guide the user toparallelize the loops we not only distinguish between dierent dependencies byrespective colors but also design a dependence lter to get a partial picture ofvarious types of dependencies on specic variablesGenerally the user cannot violate and eliminate existent ow dependenciesunless he adopts a new algorithm to rewrite the loops However the other dependencies can be eliminated using appropriate techniques such as scalar expansionand variable privatization In a way it will simplify the graph and give more possibilities to parallelize the loopsFig  The iteration space dependence graph of Gauss on variable a considerloop I and J with    Although there have already deposited many known techniques on loop transformations for decades which can result in better speedup through rearrangingor partitioning iteration space most of them are only suitable for certain loopmodels Unfortunately the compiler does not realize it easily by itselfMoreover in order to verify the correctness for certain case most of themneed strict dependence information For example the premise of wellknownunimodular transformation is to get constant dependence distance vectors It isanother barrier that there is no known optimal order in which these transformations should be applied The iteration space dependence graph gives the usermore opportunities to play a part in optimization Through the graph the userwill easily nd out some solutionFor example see g the arrows represent the overwrites of scalar variablef it exposes the lexicography of the loops The g tells the user that eachiteration on vertical direction does not exist restraint all arrows cross left lineto right line Therefore the inner loop can be parallelizedFor more complicated cases the user needs more sophisticated interactivemethod to describe and perform desired rearrangement and partition Unimodular transformations are a potential area of interest here Eg the user composessuitable unimodular matrix under the guidance of the graph then the systemautomatically calculates new loop indices and bounds ConclusionIteration space dependence graph is an attractive compiler information It promisesthe user new opportunities to exploit more parallelismwhich are normally abandoned by the compiler for inaccuracy and complexity of analysis In a way themethod we implemented is eective and ecient especially using runtime simulation which eectively avoids the drawback of dependence analysis we mentionbeforeReferences Michael Wolfe Optimizing Supercompilers for Supercomputers PhD thesisUniversity of Illinois  Utpal Banerjee Dependence Analysis for Supercomputing Kluwer AcademicPublishers  KMcKinliey Evaluation Automatic Parallelization for Ecient Execution onShared Memory Multiprocessors ICS pp  WBlume and REigenmann Performance Analysis of Parallelizing Compilerson the Perfect Benchmarks Programs IEEE Transaction on Parallel DistributedSystems  pp  Nov  K Cooper et al The ParaScope Parallel Programming Environment Proceedings of the IEEE  Feb  CDPolychronopoulos et al Parafrase An Environment for Parallelizing Partitioning Synchronizing and Scheduling Programs on Multiprocessors Inter Conference on Parallel processing pp IIII  FBZhangThe FPT Parallel Programming Environment PhD thesis University of Gent  QWang YJYu and EHDHollander Interactive Programming using PEFPTSyllabus of the Parallel Computing Seminar TUDelft pp  This article was processed using the LTEX macro package with LLNCS style

Visualizing the iteration space in PEFPT.

https://biblio.ugent.be/publication/178292/file/3239878

Visualizing the iteration space in PEFPT.

Abstract

Similar works

Full text

Available Versions

Ghent University Academic Bibliography