Improving capacity-performance tradeoffs in the storage tier

Data-set sizes are growing. New techniques are emerging to organize and analyze these data-sets. There is a key access pattern emerging with these new techniques, large sequential file accesses. The trend toward bigger files exists to help amortize the cost of data accesses from the storage layer, as many workloads are recognized to be I/O bound. The storage layer is widely recognized as the slowest layer in the system. This work focuses on the tradeoff one can make with that storage capacity to improve system performance. ^ Capacity can be leveraged for improved availability or improved performance. This tradeoff is key in the storage layer, as this allows for data loss prevention and bandwidth aggregation. Typically these tradeoffs do not allow much choice with regard to capacity use. This work will leverage replication as the enabling mechanism to improve the capacity-performance tradeoff in the storage tier, while still providing for availability. ^ This capacity-performance tradeoff can be made at both the local and distributed file system level. I propose two techniques that allow for an improved tradeoff of capacity. The local file system can be employed on scale-out or scale-up infrastructures to improve performance. The distributed file system is targeted at distributed frameworks, such as MapReduce, to improve the cluster performance. The local file system design is MorphStore, and the distributed file system is BoostDFS. ^ MorphStore is a file system that significantly improves performance when accessing large files by using two innovations. MorphStore combines (a) load-adaptive I/O access scheduling to dynamically optimize throughput (aggregation), and (b) utility-xiii driven replication to best use capacity for performance. Additionally, adaptive-access scheduling can be utilized to optimize scheduling of requests (for throughput) on systems with a large number of storage devices. Replication is utilized to make available high utility files and then optimize throughput of these high utility files based on system load. ^ BoostDFS is a distributed file system that allows a better capacity-performance tradeoff via inter-node file replication. BoostDFS is built on the observation that distributed file systems currently inter-node replication for availability, but provide no mechanism to further improve performance. Replication for availability provides diminishing returns on performance, this is due to saturation of locality. BoostDFS exploits the common by improving I/O performance of these local tasks. This is done via intra-node replication by leveraging MorphStore as the local file system. This technique allows for capacity to be traded for availability as well as performance, with a small capacity overhead under constant availability. ^ Both MorphStore and BoostDFS utilize replication. Replication allows for both bandwidth aggregation and availability, This work primarily focuses on the performance utility of replication, but does not sacrifice availability in the process. These techniques provide an improved capacity-performance tradeoff while allowing the desired level of availability

NOMA Assisted Wireless Caching: Strategies and Performance Analysis

Conventional wireless caching assumes that content can be pushed to local caching infrastructure during off-peak hours in an error-free manner; however, this assumption is not applicable if local caches need to be frequently updated via wireless transmission. This paper investigates a new approach to wireless caching for the case when cache content has to be updated during on-peak hours. Two non-orthogonal multiple access (NOMA) assisted caching strategies are developed, namely the push-then-deliver strategy and the push-and-deliver strategy. In the push-then-deliver strategy, the NOMA principle is applied to push more content files to the content servers during a short time interval reserved for content pushing in on-peak hours and to provide more connectivity for content delivery, compared to the conventional orthogonal multiple access (OMA) strategy. The push-and-deliver strategy is motivated by the fact that some users' requests cannot be accommodated locally and the base station has to serve them directly. These events during the content delivery phase are exploited as opportunities for content pushing, which further facilitates the frequent update of the files cached at the content servers. It is also shown that this strategy can be straightforwardly extended to device-to-device caching, and various analytical results are developed to illustrate the superiority of the proposed caching strategies compared to OMA based schemes

Fleets: Scalable Services in a Factored Operating System

Current monolithic operating systems are designed for uniprocessor systems, and their architecture reflects this. The rise of multicore and cloud computing is drastically changing the tradeoffs in operating system design. The culture of scarce computational resources is being replaced with one of abundant cores, where spatial layout of processes supplants time multiplexing as the primary scheduling concern. Efforts to parallelize monolithic kernels have been difficult and only marginally successful, and new approaches are needed. This paper presents fleets, a novel way of constructing scalable OS services. With fleets, traditional OS services are factored out of the kernel and moved into user space, where they are further parallelized into a distributed set of concurrent, message-passing servers. We evaluate fleets within fos, a new factored operating system designed from the ground up with scalability as the first-order design constraint. This paper details the main design principles of fleets, and how the system architecture of fos enables their construction. We describe the design and implementation of three critical fleets (network stack, page allocation, and file system) and compare with Linux. These comparisons show that fos achieves superior performance and has better scalability than Linux for large multicores; at 32 cores, fos's page allocator performs 4.5 times better than Linux, and fos's network stack performs 2.5 times better. Additionally, we demonstrate how fleets can adapt to changing resource demand, and the importance of spatial scheduling for good performance in multicores

Discovering Network Neighborhoods Using Peer-to-Peer Lookups

In many distributed applications, end hosts need to know the network locations of other nearby participating hosts in order to enhance overall performance. Potential applications that can benefit from the location information include automatic selection of nearby Web servers, proximity routing in a peer-to-peer system, and loss recovery in reliable multicasting. We focus in this paper on the network neighborhood discovery problem in large-scale distributed systems. In these systems, the number of participating nodes can be very large, and the membership can dynamically change. Our goal is for each node to discover other "nearby" participating nodes in a completely decentralized manner, where each node probes only a small subset of other nodes in the system. This approach will lead to improved overall performance by matching client requests for services with participants in the peer-to-peer service system that are, on average, nearby in the network sense. Recent works in distributed peer-to-peer systems, such as Chord, CAN, Tapestry and Pastry, provide efficient distributed lookup structures. In this paper, we investigate a rendezvous-based scheme for a node to discover other nearby participating nodes using a peer-to-peer lookup system such as Chord. Given a key, the Chord protocol maps the key onto a node. Our idea for network neighborhood discovery is for each host to compute a key that characterizes its network location on the Internet. We call such a key the location key, and the nodes that these location keys are mapped to the Rendezvous Points. To lookup other nearby participating nodes, a node seeking some service queries its corresponding rendezvous point using its location key. We focus on the issue of how to generate the location key in a distributed fashion such that nodes that are close to each other in the actual network will have similar location key values, and therefore be mapped to nearby locations on the Chord ring. In this paper, we examine the performance tradeoffs of such a rendezvous scheme using the Global Network Positioning (GNP) approach to generate the location keys. In GNP, each node measures its network distances to a few landmark nodes to derive its coordinates in a D-dimensional geometric space. We generate a host's Chord location key from its 1-dimensional GNP coordinate, and use coordinates from a higher dimensional space to refine the searching process for the closest node. We evaluate our scheme in the context of the nearest neighbor discovery problem. Using data from the Active Measurement Project of the National Laboratory for Applied Network Research (NLANR), we compare its performance with a random mapping scheme, where location keys are randomly generated. Using our coordinate-based rendezvous scheme, 66% of the nodes found their actual closest network neighbor by pinging only a small number of nodes.Singapore-MIT Alliance (SMA