Adaptive runtime-assisted block prefetching on chip-multiprocessors

Memory stalls are a significant source of performance degradation in modern processors. Data prefetching is a widely adopted and well studied technique used to alleviate this problem. Prefetching can be performed by the hardware, or be initiated and controlled by software. Among software controlled prefetching we find a wide variety of schemes, including runtime-directed prefetching and more specifically runtime-directed block prefetching. This paper proposes a hybrid prefetching mechanism that integrates a software driven block prefetcher with existing hardware prefetching techniques. Our runtime-assisted software prefetcher brings large blocks of data on-chip with the support of a low cost hardware engine, and synergizes with existing hardware prefetchers that manage locality at a finer granularity. The runtime system that drives the prefetch engine dynamically selects which cache to prefetch to. Our evaluation on a set of scientific benchmarks obtains a maximum speed up of 32 and 10 % on average compared to a baseline with hardware prefetching only. As a result, we also achieve a reduction of up to 18 and 3 % on average in energy-to-solution.Peer ReviewedPostprint (author's final draft

Using Intelligent Prefetching to Reduce the Energy Consumption of a Large-scale Storage System

Many high performance large-scale storage systems will experience significant workload increases as their user base and content availability grow over time. The U.S. Geological Survey (USGS) Earth Resources Observation and Science (EROS) center hosts one such system that has recently undergone a period of rapid growth as its user population grew nearly 400% in just about three years. When administrators of these massive storage systems face the challenge of meeting the demands of an ever increasing number of requests, the easiest solution is to integrate more advanced hardware to existing systems. However, additional investment in hardware may significantly increase the system cost as well as daily power consumption. In this paper, we present evidence that well-selected software level optimization is capable of achieving comparable levels of performance without the cost and power consumption overhead caused by physically expanding the system. Specifically, we develop intelligent prefetching algorithms that are suitable for the unique workloads and user behaviors of the world\u27s largest satellite images distribution system managed by USGS EROS. Our experimental results, derived from real-world traces with over five million requests sent by users around the globe, show that the EROS hybrid storage system could maintain the same performance with over 30% of energy savings by utilizing our proposed prefetching algorithms, compared to the alternative solution of doubling the size of the current FTP server farm

Live Prefetching for Mobile Computation Offloading

The conventional designs of mobile computation offloading fetch user-specific data to the cloud prior to computing, called offline prefetching. However, this approach can potentially result in excessive fetching of large volumes of data and cause heavy loads on radio-access networks. To solve this problem, the novel technique of live prefetching is proposed in this paper that seamlessly integrates the task-level computation prediction and prefetching within the cloud-computing process of a large program with numerous tasks. The technique avoids excessive fetching but retains the feature of leveraging prediction to reduce the program runtime and mobile transmission energy. By modeling the tasks in an offloaded program as a stochastic sequence, stochastic optimization is applied to design fetching policies to minimize mobile energy consumption under a deadline constraint. The policies enable real-time control of the prefetched-data sizes of candidates for future tasks. For slow fading, the optimal policy is derived and shown to have a threshold-based structure, selecting candidate tasks for prefetching and controlling their prefetched data based on their likelihoods. The result is extended to design close-to-optimal prefetching policies to fast fading channels. Compared with fetching without prediction, live prefetching is shown theoretically to always achieve reduction on mobile energy consumption.Comment: To appear in IEEE Trans. on Wireless Communicatio