Dynamic code mapping for limited local memory systems

Abstract—This paper presents heuristics for dynamic man-agement of application code on limited local memories present in high-performance multi-core processors. Previous techniques formulate the problem using call graphs, which do not capture the temporal ordering of functions. In addition, they only use a conservative estimate of the interference cost between functions to obtain a mapping. As a result previous techniques are unable to achieve efficient code mapping. Techniques proposed in this paper overcome both these limitations and achieve superior code mapping. Experimental results from executing benchmarks from MiBench onto the Cell processor in the Sony Playstation 3 demonstrate upto 29 % and average 12 % performance improve-ment, at tolerable compile-time overhead. I

Reducing memory space consumption through dataflow analysis

Memory is a key parameter in embedded systems since both code complexity of embedded applications and amount of data they process are increasing. While it is true that the memory capacity of embedded systems is continuously increasing, the increases in the application complexity and dataset sizes are far greater. As a consequence, the memory space demand of code and data should be kept minimum. To reduce the memory space consumption of embedded systems, this paper proposes a control flow graph (CFG) based technique. Specifically, it tracks the lifetime of instructions at the basic block level. Based on the CFG analysis, if a basic block is known to be not accessible in the rest of the program execution, the instruction memory space allocated to this basic block is reclaimed. On the other hand, if the memory allocated to this basic block cannot be reclaimed, we try to compress this basic block. This way, it is possible to effectively use the available on-chip memory, thereby satisfying most of instruction/data requests from the on-chip memory. Our experiments with this framework show that it outperforms the previously proposed CFG-based memory reduction approaches. © 2011 Elsevier Ltd. All rights reserved

Scratchpad memory management in a multitasking environment

This paper presents a dynamic scratchpad memory (SPM) code allocation technique for embedded systems running an operating system with preemptive multitasking. Existing SPM allocation schemes do not support multiple tasks or only a fixed number of processes that are known at compile time. These schemes rely on algorithms that select code depending on the size of the SPM. In contemporary portable devices, however, processes are created and terminated on demand and the SPM is shared among them. We introduce a dynamic scratchpad memory code alloca-tion technique for code that supports dynamically created processes. At runtime, an SPM manager (SPMM) loads code pages of the running applications into the SPM on de-mand. It supports different sharing strategies that deter-mine how the SPM is distributed among the running pro-cesses. We analyze several sharing strategies with regard to several preferable properties of multiprocess SPM allocation schemes. We evaluate the proposed multiprocess SPM allocation techniques and compare them to a fully-cached reference system by running several multiprocess benchmarks. The benchmarks comprise of multiple embedded applications such as H.264, MP3, MPEG-4, and PGP. On average, we achieve a 47 % improvement in throughput and a 32 % re-duction in energy consumption. A comparison with the un-achievable lower bound shows that the best SPM sharing strategy exploits 87 % of the runtime improvements and 89% of the energy savings possible

Memory Allocation for Embedded Systems with a Compile-Time-Unknown Scratch-Pad Size

This paper presents the first memory allocation scheme for embedded systems having a scratch-pad memory(SPM) whose size is unknown at compile-time. All existing memory allocation schemes for SPM require the SPM size to be known at compile-time; therefore tie the resulting executable to that size of SPM and not portable to other platforms having different SPM sizes. As size-portable code is valuable in systems supporting downloaded codes, our work presents a compiler method whose esulting executable is portable across SPMs of any size. Our technique is to employ a customized installer software, which decides the SPM allocation just before the program's first run, then modifies the program executable accordingly to implement the decided SPM allocation. Results show that our benchmarks average a 41% speedup versus an all-DRAM allocation, with overheads of 1.5% in code-size, 2% in run-time, and 3% in compile-time for our benchmarks. Meanwhile, an unrealistic upper-bound is approximated only slightly faster at 45% better than all-DRAM

Scratchpad Management in Software Managed Manycore Architectures

abstract: Caches have long been used to reduce memory access latency. However, the increased complexity of cache coherence brings significant challenges in processor design as the number of cores increases. While making caches scalable is still an important research problem, some researchers are exploring the possibility of a more power-efficient SRAM called scratchpad memories or SPMs. SPMs consume significantly less area, and are more energy-efficient per access than caches, and therefore make the design of on-chip memories much simpler. Unlike caches, which fetch data from memories automatically, an SPM requires explicit instructions for data transfers. SPM-only architectures are thus named as software managed manycore (SMM), since the data movements of such architectures rely on software. SMM processors have been widely used in different areas, such as embedded computing, network processing, or even high performance computing. While SMM processors provide a low-power platform, the hardware alone does not guarantee power efficiency, if applications on such processors deliver low performance. Efficient software techniques are therefore required. A big body of management techniques for SMM architectures are compiler-directed, as inserting data movement operations by hand forces programmers to trace flow of data, which can be error-prone and sometimes difficult if not impossible. This thesis develops compiler-directed techniques to manage data transfers for embedded applications on SMMs efficiently. The techniques analyze and find out the proper program points and insert data movement instructions accordingly. The techniques manage code, stack and heap data of applications, and reduce execution time by 14%, 52% and 80% respectively compared to their predecessors on typical embedded applications. On top of managing local data, a technique is also developed for shared data in SMM architectures. Experimental results show it achieves more than 2X speedup than the previous technique on average.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Application-aware Performance Optimization for Software Managed Manycore Architectures

abstract: One of the main goals of computer architecture design is to improve performance without much increase in the power consumption. It cannot be achieved by adding increasingly complex intelligent schemes in the hardware, since they will become increasingly less power-efficient. Therefore, parallelism comes up as the solution. In fact, the irrevocable trend of computer design in near future is still to keep increasing the number of cores while reducing the operating frequency. However, it is not easy to scale number of cores. One important challenge is that existing cores consume too much power. Another challenge is that cache-based memory hierarchy poses a serious limitation due to the rapidly increasing demand of area and power for coherence maintenance. In this dissertation, opportunities to resolve the aforementioned issues were explored in two aspects. Firstly, the possibility of removing hardware cache altogether, and replacing it with scratchpad memory with software management was explored. Scratchpad memory consumes much less power than caches. However, as data management logic is completely shifted to Software, how to reduce software overhead is challenging. This thesis presents techniques to manage scratchpad memory judiciously by exploiting application semantics and knowledge of data access patterns, thereby enabling optimization of data movement across the memory hierarchy. Experimental results show that the optimization was able to reduce stack data management overhead by 13X, produce better code mapping in more than 80% of the case, and improve performance by 83% in heap management. Secondly, the possibility of using software branch hinting to replace hardware branch prediction to completely eliminate power consumption on corresponding hardware components was explored. As branch predictor is removed from hardware, software logic is responsible for reducing branch penalty. Techniques to minimize the branch penalty by optimizing branch hint placement were proposed, which can reduce branch penalty by 35.4% over the state-of-the-art.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Compiler and Runtime for Memory Management on Software Managed Manycore Processors

abstract: We are expecting hundreds of cores per chip in the near future. However, scaling the memory architecture in manycore architectures becomes a major challenge. Cache coherence provides a single image of memory at any time in execution to all the cores, yet coherent cache architectures are believed will not scale to hundreds and thousands of cores. In addition, caches and coherence logic already take 20-50% of the total power consumption of the processor and 30-60% of die area. Therefore, a more scalable architecture is needed for manycore architectures. Software Managed Manycore (SMM) architectures emerge as a solution. They have scalable memory design in which each core has direct access to only its local scratchpad memory, and any data transfers to/from other memories must be done explicitly in the application using Direct Memory Access (DMA) commands. Lack of automatic memory management in the hardware makes such architectures extremely power-efficient, but they also become difficult to program. If the code/data of the task mapped onto a core cannot fit in the local scratchpad memory, then DMA calls must be added to bring in the code/data before it is required, and it may need to be evicted after its use. However, doing this adds a lot of complexity to the programmer's job. Now programmers must worry about data management, on top of worrying about the functional correctness of the program - which is already quite complex. This dissertation presents a comprehensive compiler and runtime integration to automatically manage the code and data of each task in the limited local memory of the core. We firstly developed a Complete Circular Stack Management. It manages stack frames between the local memory and the main memory, and addresses the stack pointer problem as well. Though it works, we found we could further optimize the management for most cases. Thus a Smart Stack Data Management (SSDM) is provided. In this work, we formulate the stack data management problem and propose a greedy algorithm for the same. Later on, we propose a general cost estimation algorithm, based on which CMSM heuristic for code mapping problem is developed. Finally, heap data is dynamic in nature and therefore it is hard to manage it. We provide two schemes to manage unlimited amount of heap data in constant sized region in the local memory. In addition to those separate schemes for different kinds of data, we also provide a memory partition methodology.Dissertation/ThesisPh.D. Computer Science 201

Schedulability-driven scratchpad memory swapping for resource-constrained real-time embedded systems

In resource-constrained real-time embedded systems, scratchpad memory (SPM) is utilized in place of cache to increase performance and enforce consistent behavior of both hard and soft real-time tasks via software-controlled SPM management techniques (SPMMTs). Real-time systems depend on time critical (hard) tasks to complete execution before their deadline times. Many real-time systems also depend on the execution of soft tasks that do not have to complete by hard deadlines. This thesis evaluates a new SPMMT that increases both worst-case task slack time (TST) and soft task processing capabilities, by combining two existing SPMMTs. The schedulability-driven ACETRB / WCETRB swapping (SDAWS) SPMMT of this thesis uses task schedulability characteristics to control the selection of either the average-case execution time reduction based (ACETRB) SPMMT or the worst-case execution time reduction based (WCETRB) SPMMT. While the literature contains examples of combined management techniques, until now there have been none that combine both WCETRB and ACETRB SPMMTs. The advantage of combining them is to achieve WCET reduction comparable to what can be achieved with the WCETRB SPMMT, while achieving significantly reduced ACET relative to the WCETRB SPMMT. Using a stripped-down RTOS and an SPMMT simulator implemented for this work, evaluated resource-constrained scenarios show a reduction in task slack time from the SDAWS SPMMT relative to the WCETRB SPMMT between 20% and 45%. However, the evaluated scenarios also conservatively show that SDAWS can reduce ACET relative to the WCETRB SPMMT by up to 60%

Reducing Waste in Memory Hierarchies

Memory hierarchies play an important role in microarchitectural design to bridge the performance gap between modern microprocessors and main memory. However, memory hierarchies are inefficient due to storing waste. This dissertation quantifies two types of waste, dead blocks and data redundancy. This dissertation studies waste in diverse memory hierarchies and proposes techniques to reduce waste to improve performance with limited overhead. This dissertation observes that waste of dead blocks in an inclusive last level cache consists of two kinds of blocks: blocks that are highly accessed in core caches and blocks that have low temporal locality in both core caches and the last-level cache. Blindly replacing all dead blocks in an inclusive last level cache may degrade performance. This dissertation proposes temporal-based multilevel correlating cache replacement to improve performance of inclusive cache hierarchies. This dissertation observes that waste exists in private caches of graphics processing units (GPUs) as zero-reuse blocks. This dissertation defines zero-reuse blocks as blocks that are dead after being inserted into caches. This dissertation proposes adaptive GPU cache bypassing technique to improve performance as well as reducing power consumption by dynamically bypassing zero-reuse blocks. This dissertation exploits waste of data redundancy at the block-level granularity and finds that conventional cache design wastes capacity because it stores duplicate data. This dissertation quantifies the percentage of data duplication and analyze causes. This dissertation proposes a practical cache deduplication technique to increase the effectiveness of the cache with limited area and power consumption

Alocação de dados e de código em memórias embarcadas: uma aborgagem pós-compilação

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Ciência da Computação, Florianópolis, 2010Memórias do tipo scratchpad (SPMs) são alternativas promissoras para sistemas embarcados energeticamente eficientes. Muitas das técnicas de otimização para o mapeamento de dados e código para SPMs assumem a disponibilidade do código-fonte da aplicação. Porém, o desenvolvimento de software embarcado deve lidar com código legado, bibliotecas de ter- ceiros e blocos de propriedade intelectual (IPs) para os quais podem estar disponíveis somente os arquivos-binários. As poucas técnicas que reali- zam otimizações diretamente em arquivos binários operam em arquivos executáveis e limitam-se a tratar somente código ou somente dados. Este trabalho propõe uma nova técnica que pode alocar tanto código quanto dados para a SPM. Operando diretamente em binários, a técnica permite que elementos encapsulados em bibliotecas façam parte do ma- peamento para SPMs. A técnica consiste de três principais mecanismos: o profiler, o mapeador e o patcher. O patcher foi projetado para operar utilizando arquivos-objeto relocáveis, contornando assim a limitação de se gerenciar relocações para SPM em arquivos-objeto executáveis. A maior eficiência ao se tratar arquivos binários relocáveis resultou em tempos de execução inferiores em pelo menos uma ordem de magnitude se compara- dos às técnicas relacionadas. O tempo médio de patching foi de 0,7s em uma estação de trabalho com quatro núcleos de processamento. Comparando-se com o mapeamento de somente código, a proposta de também alocar dados em SPM resultou em economia extra de energia de 21%, em média, para um variado conjunto de programas do benchmark MiBench e diversas configurações de memória. Apesar de uma média re- lativamente baixa, instâncias de caso de uso reais com maior conteúdo de dados estáticos puderam ser encontradas, para as quais economias extra de energia de 67% e 91% foram observadas, quando dados também são passíveis de serem mapeados para SPM. Foi também observado que, para todos os casos de uso reais usados nos experimentos, os parâmetros de caracterização para mapeamento em SPM são bastante descorrelatados, o que significa - à luz de trabalhos anteriores - que o mapeamento exato obtido pelo assim-chamado algoritmo MINKNAP provavelmente manterá sua alta eficiência, mesmo diante do crescimento do número de elementos de programas resultante da crescente complexidade do software embarcado. Ao se combinar a inclusão de dados e bibliotecas na alocação em SPM com a eficiência do mapeamento e a eficiência do patching, a técnica proposta torna-se um enfoque promissor para a otimização energeticamente consciente de código embarcado pré-compilado