The paper investigates the interaction between software pipelining and different software prefetching techniques for VLIW machines. It is shown that processor stalls due to memory dependencies have a great impact into execution time. A novel heuristic is proposed and it is show to outperform previous proposals.Peer ReviewedPostprint (published version

González Colás, Antonio María

Sánchez, Jesús

English

UPCommons. Portal del coneixement obert de la UPC

Software Prefetching for Software Pipelined LoopsF. Jesús Sánchez and Antonio GonzálezDepartment of Computer ArchitectureUniversitat Politècnica de CatalunyaCampus Nord - c./ Jordi Girona, 1-3 - Mòdul D608034 - Barcelona (SPAIN)E-mail: {fran,antonio}@ac.upc.esAbstractThis paper investigates the interaction between softwarepipelining and different software prefetching techniquesfor VLIW machines. It is shown that processor stalls due tomemory dependences have a great impact into executiontime. A novel heuristic is proposed and it is show to outper-form previous proposals.1. IntroductionSoftware pipelining represents a family of loop sched-uling techniques that tries to exploit ILP by executing inparallel consecutive iterations of a loop. The most popularscheme is called modulo scheduling, and it consists of find-ing a fixed pattern of operations (of length II or initiationinterval) from distinct iterations([3]).Several schemes have been proposed in the literaturewith the goal of minimize the II and/or register pressure,but none of them has evaluated the effect of memory. Whensoftware pipelining is applied in VLIW architectures,where instruction latencies and scheduling are fixed atcompile-time, execution time can be highly degraded dueto the stall time provoked by dependences with memoryinstructions. Even if a nonblocking cache is used, truedependences with previous memory operations at a neardistance1 can make the processor to stall afterwards. Thechoice of scheduling all loads using the cache-miss latencyrequires considerable ILP and increases register pres-sure([1]).Different techniques to improve memory behaviorexist and are well-known, and software prefetching is oneof them. The main idea of this method is to bring to cachethe data that will be used in a near future([2]).In this paper we investigate the interactions betweensoftware pipelining and software prefetching in a VLIWarchitecture. Some alternatives to perform softwareprefetching are described, and a novel heuristic is pre-sented. An evaluation in execution time terms is reported aswell as some conclusions.1.Almost all modulo scheduling schemes use a fixed cache-hitlatency for all memory operations1060-3425/98 $10.02. Software prefetching schemesSoftware prefetching is an effective technique to toler-ate memory latency. When it is used with a nonblockingcache, this technique allows the processor to hide part or allthe memory latency by overlapping the fetch of data andthe computation.Software prefetching can be performed through twoalternative schemes: binding and nonbinding prefetching.The first alternative, also known as early scheduling ofmemory operations, moves memory instructions awayfrom those instructions that depend on them. The secondalternative introduces in the code special instructions,which are called prefetch instructions. These are nonfault-ing instructions that perform a cache lookup but do notmodify any register.In the study presented in this paper we have evaluatedtwo techniques of binding prefetching:• Early scheduling always (ESA): all memory opera-tions of the loop are scheduled using cache-misslatency.• Early scheduling according to locality (ESL): sched-ule instructions that have some type of locality usingthe cache-hit latency and schedule the remaining onesusing the cache-miss latency.We have also evaluated three distinct schemes forinserting prefetch instructions (nonbinding prefetch):• Insert prefetch always (IPA): insert a prefetch instruc-tion for every memory operation.• Insert prefetch according to temporal locality (IPT):insert prefetch for those references without temporallocality even if they exhibit spatial locality.• Insert prefetch according to locality (IPL): insertprefetch for those instructions without any type oflocality.3. A novel software prefetching techniqueThe proposed software prefetching scheme is calledcache sensitive modulo scheduling (CSMS), and it tries tominimize both the compute time and the stall time. Theseterms are not independent and reducing one of them may0 (c) 1998 IEEECHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBND0.00.20.40.60.81.0Normalized Loop Execution Timetomcatv swim su2cor hydro2d mgrid turb3dSimple architecture1.5931.371CHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBND0.00.20.40.60.81.0Normalized Loop Execution Timetomcatv swim su2cor hydro2d mgrid turb3dAggressive architecture1.2573.6183.034Figure 1. Software prefetching schemes performanceresult in an increase in the other. The proposed algorithmtries to find the best trade-off between the two terms.The CSMS algorithm is based on early scheduling ofsome selectively chosen memory operations. Scheduling amemory operation using the cache-miss latency can hidealmost all memory latency without increasing much thenumber of instructions (as opposed to the use of prefetchinstructions). However, it can increase the execution timein three ways:• It may increase the register pressure, and therefore, itmay increase the II due to spill code.• It may increase IIrec because the latency of memoryoperations is augmented.• It may increase the SC (stage counter) because thelength of individual loop iterations may be increased.Two of the main issues of the CSMS algorithm is thereduction of the impact of recurrences on the II and theminimization of the stall time. The problem of the cost ofthe prolog and epilog is handled by computing two alterna-tive schedules. Both focus on minimizing the stall time andthe II. However, one of them reduces the impact of the pro-log and the epilog at the expense of an increase in the stalltime whereas the other does not care about the prolog andepilog cost. Then, depending on the number of iterations ofthe loop, the most effective one is chosen.The algorithm consists of creating two dependencegraphs, one using the cache-miss latency for schedulingeach memory operation, and another one using cache-missor hit latency according to a static locality analysis. Theeffect of recurrences that limit de initiation interval isreduced by changing, from cache-miss to cache-hit, thelatency of some memory operations (following a localityorder) until this recurrence minimizes the II. An upperbound in the number of iterations of the loop help us tochoose between the scheduling of both graphs.More details about the CSMS algorithm are reportedin [4].1060-3425/98 $10.4. Some performance resultsThe performance of the software prefetching schemeshas been studied for some SPECfp95 benhmarks, and fortwo VLIW architectures: simple (4-issue and the cache-miss latency is 10 cycles) and aggressive (8-issue and thecache-miss latency is 20 cycles).In addition to the above-mentioned schemes,we havemeasured the scheduling using cache-hit latency always(CHL) and a lower bound of the execution time (LBND).In Figure 1 results are presented. Black bars representsstall time, and grey bar represents compute time, all ofthem normalized to CHL. It is show that the CSMS schemeachieves the best trade-off between stall and compute time,and its performance is close to the lower bound.5. ConclusionsIn this paper we have compared the effect that somesoftware prefetching techniques have in software pipelinedloops for VLIW architectures. We have seen that the pro-posed CSMS scheme significally outperforms previous pro-posals.References[1] S.G. Abraham, R.A. Sugumar, B.R. Rau and R. Gupta, “Pre-dictability of Load/Store Instruction Latencies”, in Procs. of26th Int. Symp. on Microarchitecture, pp. 129-152, Dec. 1993[2] D. Callahan, K. Kennedy and A. Porterfield, “SoftwarePrefetching”, in Procs of the IV Symp. on Arch. Support forProg. Lang. and Oper. Syst. (ASPLOS), pp. 40-52, April 1991[3] M.S. Lam, “Software Pipelining: an Effective SchedulingTechnique for VLIW Machines”, in Procs. of Conf. on Prog.Lang. Desing and Impl. (PLDI), pp. 43-53, May 1991[4] F.J. Sánchez and A. González, “Cache Sensitive ModuloScheduling”, in Procs. of 30th Int. Symp. on Microarchitec-ture, Dec. 199700 (c) 1998 IEEE

Software prefetching for software pipelined loops

The paper investigates the interaction between software pipelining and different software prefetching techniques for VLIW machines. It is shown that processor stalls due to memory dependencies have a great impact into execution time. A novel heuristic is proposed and it is show to outperform previous proposals.Peer Reviewe

UPCommons

UPCommons (Universitat Politècnica de Catalunya)

Software Prefetching for Software Pipelined LoopsF. Jesús Sánchez and Antonio GonzálezDepartment of Computer ArchitectureUniversitat Politècnica de CatalunyaCampus Nord - c./ Jordi Girona, 1-3 - Mòdul D608034 - Barcelona (SPAIN)E-mail: {fran,antonio}@ac.upc.esres totioer-d-inlarndtur,e, aueryueeahencesrneheenrere allyAbstractThis paper investigates the interaction between softwapipelining and different software prefetching techniquefor VLIW machines. It is shown that processor stalls duememory dependences have a great impact into executime. A novel heuristic is proposed and it is show to outpform previous proposals.1. IntroductionSoftware pipelining represents a family of loop scheuling techniques that tries to exploit ILP by executing parallel consecutive iterations of a loop. The most popuscheme is called modulo scheduling, and it consists of fiing a fixed pattern of operations (of lengthII  or initiationinterval) from distinct iterations([3]).Several schemes have been proposed in the literawith the goal of minimize theII  and/or register pressurebut none of them has evaluated the effect of memory. Whsoftware pipelining is applied in VLIW architectureswhere instruction latencies and scheduling are fixedcompile-time, execution time can be highly degraded dto the stall time provoked by dependences with memoinstructions. Even if a nonblocking cache is used, trdependences with previous memory operations at a ndistance1 can make the processor to stall afterwards. Tchoice of scheduling all loads using the cache-miss laterequires considerable ILP and increases register prsure([1]).Different techniques to improve memory behavioexist and are well-known, and software prefetching is oof them. The main idea of this method is to bring to cacthe data that will be used in a near future([2]).In this paper we investigate the interactions betwesoftware pipelining and software prefetching in a VLIWarchitecture. Some alternatives to perform softwaprefetching are described, and a novel heuristic is psented. An evaluation in execution time terms is reportedwell as some conclusions.1.Almost all modulo scheduling schemes use a fixed cache-hitlatency for all memory operations1060-3425/98 $10.0n-entry--s2. Software prefetching schemesSoftware prefetching is an effective technique to toler-ate memory latency. When it is used with a nonblockingcache, this technique allows the processor to hide part or athe memory latency by overlapping the fetch of data andthe computation.Software prefetching can be performed through twoalternative schemes: binding and nonbinding prefetching.The first alternative, also known as early scheduling ofmemory operations, moves memory instructions awayfrom those instructions that depend on them. The secondalternative introduces in the code special instructions,which are called prefetch instructions. These are nonfault-ing instructions that perform a cache lookup but do notmodify any register.In the study presented in this paper we have evaluatedtwo techniques of binding prefetching:• Early scheduling always (ESA): all memory opera-tions of the loop are scheduled using cache-misslatency.• Early scheduling according to locality (ESL): sched-ule instructions that have some type of locality usingthe cache-hit latency and schedule the remaining onesusing the cache-miss latency.We have also evaluated three distinct schemes forinserting prefetch instructions (nonbinding prefetch):• Insert prefetch always (IPA): insert a prefetch instruc-tion for every memory operation.• Insert prefetch according to temporal locality (IPT):insert prefetch for those references without temporallocality even if they exhibit spatial locality.• Insert prefetch according to locality (IPL): insertprefetch for those instructions without any type oflocality.3. A novel software prefetching techniqueThe proposed software prefetching scheme is calledcache sensitive modulo scheduling (CSMS), and it tries tominimize both the compute time and the stall time. Theseterms are not independent and reducing one of them ma0 (c) 1998 IEEECHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBND0.00.20.40.60.81.0Normalized Loop Execution Timetomcatv swim su2cor hydro2d mgrid turb3dSimple architecture1.5931.371CHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBNDCHLESAESLIPAIPTIPLCSMSLBND0.00.20.40.60.81.0Normalized Loop Execution Timetomcatv swim su2cor hydro2d mgrid turb3dAggressive architecture1.2573.6183.034Figure 1. Software prefetching schemes performancehmfgidetc,yed.fandrosta ocelinesthtytoedr,result in an increase in the other. The proposed algorittries to find the best trade-off between the two terms.The CSMS algorithm is based on early scheduling osome selectively chosen memory operations. Schedulinmemory operation using the cache-miss latency can halmost all memory latency without increasing much thnumber of instructions (as opposed to the use of prefeinstructions). However, it can increase the execution timin three ways:• It may increase the register pressure, and thereforemay increase theII  due to spill code.• It may increaseII rec because the latency of memoroperations is augmented.• It may increase theSC (stage counter) because thlength of individual loop iterations may be increaseTwo of the main issues of theCSMS algorithm is thereduction of the impact of recurrences on theII  and theminimization of the stall time. The problem of the cost othe prolog and epilog is handled by computing two alterntive schedules. Both focus on minimizing the stall time atheII . However, one of them reduces the impact of the plog and the epilog at the expense of an increase in the time whereas the other does not care about the prolog epilog cost. Then, depending on the number of iterationsthe loop, the most effective one is chosen.The algorithm consists of creating two dependengraphs, one using the cache-miss latency for schedueach memory operation, and another one using cache-mor hit latency according to a static locality analysis. Theffect of recurrences that limit de initiation interval ireduced by changing, from cache-miss to cache-hit, latency of some memory operations (following a localiorder) until this recurrence minimizes theII . An upperbound in the number of iterations of the loop help us choose between the scheduling of both graphs.More details about the CSMS algorithm are reportin [4].1060-3425/98 $10. aehe it--allndfgisse4. Some performance resultsThe performance of the software prefetching schemeshas been studied for some SPECfp95 benhmarks, and fotwo VLIW architectures: simple (4-issue and the cache-miss latency is 10 cycles) and aggressive (8-issue and thecache-miss latency is 20 cycles).In addition to the above-mentioned schemes,we havemeasured the scheduling using cache-hit latency always(CHL) and a lower bound of the execution time (LBND).In Figure 1 results are presented. Black bars representsstall time, and grey bar represents compute time, all ofthem normalized toCHL. It is show that theCSMS schemeachieves the best trade-off between stall and compute timeand its performance is close to the lower bound.5. ConclusionsIn this paper we have compared the effect that somesoftware prefetching techniques have in software pipelinedloops for VLIW architectures. We have seen that the pro-posedCSMS scheme significally outperforms previous pro-posals.References[1] S.G. Abraham, R.A. Sugumar, B.R. Rau and R. Gupta, “Pre-dictability of Load/Store Instruction Latencies”, inProcs. of26th Int. Symp. on Microarchitecture, pp. 129-152, Dec. 1993[2] D. Callahan, K. Kennedy and A. Porterfield, “SoftwarePrefetching”, inProcs of the IV Symp. on Arch. Support forProg. Lang. and Oper. Syst. (ASPLOS), pp. 40-52, April 1991[3] M.S. Lam, “Software Pipelining: an Effective SchedulingTechnique for VLIW Machines”, inProcs. of Conf. on Prog.Lang. Desing and Impl. (PLDI), pp. 43-53, May 1991[4] F.J. Sánchez and A. González, “Cache Sensitive ModuloScheduling”, inProcs. of 30th Int. Symp. on Microarchitec-ture, Dec. 199700 (c) 1998 IEEE

https://upcommons.upc.edu/bitstream/2117/105151/1/00649285.pdf

Software prefetching for software pipelined loops

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC

UPCommons

UPCommons (Universitat Politècnica de Catalunya)