A storage architecture for data-intensive computing

The assimilation of computing into our daily lives is enabling the generation of data at unprecedented rates. In 2008, IDC estimated that the "digital universe" contained 486 exabytes of data [9]. The computing industry is being challenged to develop methods for the cost-effective processing of data at these large scales. The MapReduce programming model has emerged as a scalable way to perform data-intensive computations on commodity cluster computers. Hadoop is a popular open-source implementation of MapReduce. To manage storage resources across the cluster, Hadoop uses a distributed user-level filesystem. This filesystem --- HDFS --- is written in Java and designed for portability across heterogeneous hardware and software platforms. The efficiency of a Hadoop cluster depends heavily on the performance of this underlying storage system. This thesis is the first to analyze the interactions between Hadoop and storage. It describes how the user-level Hadoop filesystem, instead of efficiently capturing the full performance potential of the underlying cluster hardware, actually degrades application performance significantly. Architectural bottlenecks in the Hadoop implementation result in inefficient HDFS usage due to delays in scheduling new MapReduce tasks. Further, HDFS implicitly makes assumptions about how the underlying native platform manages storage resources, even though native filesystems and I/O schedulers vary widely in design and behavior. Methods to eliminate these bottlenecks in HDFS are proposed and evaluated both in terms of their application performance improvement and impact on the portability of the Hadoop framework. In addition to improving the performance and efficiency of the Hadoop storage system, this thesis also focuses on improving its flexibility. The goal is to allow Hadoop to coexist in cluster computers shared with a variety of other applications through the use of virtualization technology. The introduction of virtualization breaks the traditional Hadoop storage architecture, where persistent HDFS data is stored on local disks installed directly in the computation nodes. To overcome this challenge, a new flexible network-based storage architecture is proposed, along with changes to the HDFS framework. Network-based storage enables Hadoop to operate efficiently in a dynamic virtualized environment and furthers the spread of the MapReduce parallel programming model to new applications

Imagining Consumers

Winner of the Hagley Prize in Business History from The Hagley Museum and Library and the Business History ConferenceSelected by Choice Magazine as an Outstanding Academic TitleOriginally published in 1999. Imagining Consumers tells for the first time the story of American consumer society from the perspective of mass-market manufacturers and retailers. It relates the trials and tribulations of china and glassware producers in their contest for the hearts of the working- and middle-class women who made up more than eighty percent of those buying mass-manufactured goods by the 1920s. Based on extensive research in untapped corporate archives, Imagining Consumers supplies a fresh appraisal of the history of American business, culture, and consumerism. Case studies illuminate decision making in key firms—including the Homer Laughlin China Company, the Kohler Company, and Corning Glass Works—and consider the design and development of ubiquitous lines such as Fiesta tableware and Pyrex Ovenware