The behavior of many algorithms is heavily determined by the input data. Furthermore, this often means that multiple and completely different execution paths can be followed, also internal data usage and handling is frequently quite different. Therefore, static compile time memory allocation is not efficient, especially on embedded systems where memory is a scarce resource, and dynamic memory management is the only feasible alternative. Including applications with dynamic memory in embedded systems introduces new challenges as compared to traditional signal processing applications. In this session, an automated framework is presented to optimize embedded applications with extensive use of dynamic memory management. The proposed methodology automates the exploration and identification of optimal data type implementations based on power estimates, memory accesses and normalized memory usage

Atienza, David

De Florio, Vincenzo

Deconinck, Geert

Leeman, Marc

Ykman, Chantal

Infoscience - École polytechnique fédérale de Lausanne

Automated Dynamic Memory Data Type ImplementationExploration and OptimizationMarc Leeman∗, Chantal Ykman‡, David Atienza‡†, Vincenzo De Florio Geert DeconinckESAT K.U.LeuvenKasteelpark Arenberg 10B-3001, Leuven (Heverlee), Belgium firstname.lastname@esat.kuleuven.ac.be. ‡ lastname@imec.be.AbstractThe behavior of many algorithms is heavily determinedby the input data. Furthermore, this often means that mul-tiple and completely different execution paths can be fol-lowed, also internal data usage and handling is frequentlyquite different. Therefore, static compile time memory al-location is not efficient, especially on embedded systemswhere memory is a scarce resource, and dynamic mem-ory management is the only feasible alternative. Includingapplications with dynamic memory in embedded systemsintroduces new challenges as compared to traditional sig-nal processing applications. In this session, an automatedframework is presented to optimize embedded applicationswith extensive use of dynamic memory management. Theproposed methodology automates the exploration and iden-tification of optimal data type implementations based onpower estimates, memory accesses and normalized memoryusage.1 IntroductionIn order to optimize algorithms, the embedded devel-oper must be aware of a number of constraints: these sys-tems have typically a more constrained memory hierarchythan General Purpose Processors (GPP) where algorithmstypically are developed upon. Furthermore, the optimiza-tion objectives are not only dominated by performance,but memory accesses and power consumption are (at least)equally important.∗Partially supported by the Fund for Scientific Research - Flanders(Belgium, F.W.O.) through project G.0036.99 and a Postdoctoral Fellow-ship for Geert Deconinck.†Also at DACYA/UCM. Madrid, Spain. Partially supported by theSpanish Government Research Grant TIC2002/0750 and E.C. Marie CurieFellowship contract HPMT-CT-2000-00031.When memory usage is deterministic, all memory usageand access patterns can be determined at compile time. Therequired memory is reserved in one single allocation andfreed when no longer needed. Moreover, since this allo-cated memory is contiguous, the addresses of the elementsare known and accessible in a minimum of time. However,this is not longer certain when Dynamic Memory Manage-ment (DMM) is required.MATISSE [4, 3, 1, 2] is a methodology for system-leveldynamic memory management. It is intended for applica-tions operating on large and irregular data structures, dy-namically allocated and stored in sets, called dynamic datasets. It uses a fast, stepwise and cost-driven exploration andspecification refinement at the system level of the neededdynamic data sets. As a result, optimized distributed mem-ory architectures can be achieved in embedded systems.In the remainder, automation and modification of the firstsub-step of the MATISSE methodology1 is discussed in or-der to handle applications with multiple dynamic data setswhich change their behavior during the application: Ab-stract Data Type refinement (ADT). In addition, a compari-son of dynamic memory usage between the original versionand the optimized one of the main dynamic data type in theapplication is shown in Figure 2 to illustrate the final resultsaccomplished.2 Method and ResultsIn an ADT context, data dependencies can be dividedinto two distinct types. First, design or algorithmic depen-dencies dependencies built in the program as a result of thecontrol flow of the algorithm, e.g. ordered or unordered1The first sub-step generates for each dynamic data set an optimizedimplementation. This is called Abstract Data Type (ADT) Transforma-tion and Refinement. The second step organizes the dynamic memoryspace and selects custom memory managers to handle dynamic memory(de)allocation of the needed data structures. This sub-step is call DynamicMemory Management Refinement.222Proceedings of the IEEE Computer Society Annual Symposium on VLSI (ISVLSI’03) 0-7695-1904-0/03 $17.00 © 2003 IEEE Figure 1. Automatic Profile Framework withapplication objects reporting their behaviorto profile objects.sets. This kind of dependencies cannot be removed and/orchanged without modifying the algorithm itself. Second,data dependencies that are implementational data depen-dencies. These are introduced while moving from speci-fication to implementation due to rigid (immutable) choicesthat concern e.g. the data type implementation. Once thesechoices are made, a set of data dependencies will affect thecost of the final implementation, e.g. cache conscious struc-ture reorganizations. These data type induced dependenciesare avoidable as long as the final application is not deployedon the target system.MATISSE employs three basic data types, namely lists,arrays and trees, and organizes them in multi-layered struc-tures. All these structures share a number of basic opera-tions like add, insert, delete, etc. In addition, theycan have multiple implementations, reflecting different fea-tures of the pursued final system, e.g. a different garbagecollection system. As a result, this rises exponentially thesearch space as more alternatives are considered.This MATISSE methodology provides as one of its keyconcepts a representative estimate on algorithm and codechanges early in the development flow for system develop-ers. Thus, it reduces the number of ulterior costly and timeconsuming reconfigurations during system design. In orderto have these early estimates on the implementation and onimportant objectives (e.g. power, memory usage and ac-cess), MATISSE automatically adds timing information atthe source code level.The driver application that is presented is part of a larger3D image processing application and has several propertiesthat challenge dynamic memory optimization:• The application has 4 dynamic data sets with severaldistinct access patterns.• The behavior of some data types changes during theapplication run.Figure 2. Result of the profiling: the memoryusage of dynamic data types is plotted. In thiscase, the data types of Candidates and StaticCopy are optimized. The DDTR optimized plotshows the result after removal of a crucialapplication dependency.The presence of multiple data sets required a new profil-ing and instrumentation framework (see Figure 1) both easyto use in an automated code instrumentation tool and flexi-ble enough to handle all kinds of combinations and loose re-lationships between the data sets. Since MATISSE uses C++as an abstract data type description language, the profilinginformation cannot be collected at the object level. In fact,several objects, possibly from different classes can definea multi-layered dynamic data type description. Therefore,the profiling framework decouples this information fromthe data type class hierarchy. Furthermore, these objectsare able to report their memory and timing behavior to ded-icated profiling objects without the intervention from thedeveloper, enhancing their versatility. Hence, the numberof these objects depends on the amount of data type occur-rences visible from the application (e.g. if an applicationhas 2 variables to be optimized, 2 profiling objects are cre-ated during the code transformation). . The code trans-formation accomplishes several tasks, next to basic codetransformations, a reconstruction of the class hierarchy, aninstrumentation of the dynamic data type definition codeand the creation and and inclusion of the MATISSE specificcode. This instrumented source code is used to composerun time information on memory accesses and usage, tim-ing information and function calls. Because the data setschange their behavior at run time, normalized memory us-age is used to compute power estimates.As a matter of fact, in the presence of multi-objectiveoptimizations (optimizing for memory accesses, usage andpower consumption), a single optimal point is unlikely. In-stead, Pareto optima are constructed during exploration of223Proceedings of the IEEE Computer Society Annual Symposium on VLSI (ISVLSI’03) 0-7695-1904-0/03 $17.00 © 2003 IEEE Figure 3. Effect of applying Matisse on the driver application in a logarithmical scale.the search space2.The output of the simulation gives feedback to the de-veloper about possible algorithmic bottlenecks, while theimplementational ones are optimized automatically:• Profiling information can indicate clear algorithmicproblems that were hidden in the code. In one exam-ple, a very high access count in one part of the programwas identified while scanning a large data structure.After a closer code inspection, the algorithm assumedthat the data was created without order and had to ac-cess all the elements in order to construct exhaustivesubsets. In fact, the data did contain order and the sub-sets were located one after another.• In a different part of the execution of the applica-tion, data in one large dynamic data structure was de-stroyed before the following dynamic data set was con-structed. This showed a hidden matrix where the datawas copied for faster access. As a consequence of this,the original implementation doubled the memory bot-tleneck. With the combined profiling information, thiswas removed safely, without compromising the per-formance.Like many multimedia applications, the driver applica-tion has multiple dynamic data types. In figure 2, the re-sult of the Dynamic Data Type Refinement (DDTR) stepfor one of them is illustrated. As the figure shows, the largeCandidates list is copied in a dynamically allocated ar-ray. From that point onwards, this array is used for fastaccess to the data that is processed. Even though this obvi-ously speeds up the application to a great extent, it comesat a high price since there is a duplication of the memorybottleneck when the data is copied. In addition, Figure 2shows the dynamic memory usage for the DDTR optimizedversion during the application execution. Evidently, an ef-ficient selection of the dynamic data type results in a muchlower memory usage and a faster execution (note that whilethe data is being copied to the fast array, the algorithm can-not make progress).2A point is Pareto Optimal when it is no longer possible to improve oneof the objectives without worsening another.As a summary, the first step is to change algorithmic de-pendencies before examining algorithmic ones. Then, oncethe optimal hierarchical implementation is selected for eachdata set under investigation, the code is instantiated (obvi-ously without the profiling information of the framework).In fact, for a number of implementations, C counterparts ofthe C++ code are available, but their instantiation is cur-rently not included in the tools. These C definitions havethe advantage that they eliminate much of the C++ codesize overhead.For the driver application where we have applied ourmethodology, it allows to improve memory footprint, mem-ory accesses and power by 98.5 %, 51.7 % and 99.9 % re-spectively, and is illustrated in Figure 3. In the bar charts,the results for memory accesses, memory usage and powerare plotted for the original implementation, a DDTR opti-mized version where the copy to the array (see Figure 2) iskept and the case where both DDTR and the removal of thealgorithmic dependency are applied . Similar behavior hasbeen identified in several multimedia applications and algo-rithms, thus, without any doubt, they can be optimized withthis methodology as well.References[1] D. Verkest, J. da Silva, C. Ykman, K. Croes, M. Miranda,S. Wuytack, G. de Jong, F. Catthoor, and H. De Man. Matisse:A system-on-chip design methodology emphasizing dynamicmemory management. Journal of VLSI Signal Processing,21(3):277–291, July 1999.[2] S. Wuytack, J. da Silva, F. Catthoor, G. de Jong, and C. Yk-man. Memory management for embedded network appli-cations. IEEE Transactions on Computer-Aided Design,18(5):533–544, May 1999.[3] C. Ykman, J. Lambrecht, A. Van der Togt, and F. Catthoor.Multi-objective abstract data type refinement for mapping ta-bles in telecom network applications. In ACM SIGPLANWorkshop on Memory System Performance, Berlin, Germany,June 2002.[4] C. Ykman, J. Lambrecht, D. Verkest, F. Catthoor, B. Svantes-son, A. Hemani, and F. Wolf. Dynamic memory managementmethodology applied to embedded telecom network systems.Accepted for publication in IEEE Transactions on VLSI Sys-tems, 2002.224Proceedings of the IEEE Computer Society Annual Symposium on VLSI (ISVLSI’03) 0-7695-1904-0/03 $17.00 © 2003 IEEE 

Automated dynamic memory data type implementation exploration and optimization

http://infoscience.epfl.ch/record/131012/files/ISVLSI03-01183476.pdf

Automated dynamic memory data type implementation exploration and optimization

Abstract

Similar works

Full text

Available Versions

Infoscience - École polytechnique fédérale de Lausanne