Green Master based on MapReduce Cluster

[[abstract]]MapReduce is a kind of distributed computing system, and also many people use it nowadays. In this paper, the Green Master based on MapReduce is proposed to solve the problem between load balance and power saving. There are three mechanism proposed by this paper to improve the MapReduce system efficiency. First, a brand new architecture called Green Master is designed in the system. Second, Benchmark Score is added to each services in the cluster. In the last, an algorithm about how to distinguish the high score service and the low score service, and how to use them effectively.[[conferencetype]]國際[[conferencedate]]20130823~20130825[[iscallforpapers]]Y[[conferencelocation]]Taipei, Taiwa

The Family of MapReduce and Large Scale Data Processing Systems

In the last two decades, the continuous increase of computational power has produced an overwhelming flow of data which has called for a paradigm shift in the computing architecture and large scale data processing mechanisms. MapReduce is a simple and powerful programming model that enables easy development of scalable parallel applications to process vast amounts of data on large clusters of commodity machines. It isolates the application from the details of running a distributed program such as issues on data distribution, scheduling and fault tolerance. However, the original implementation of the MapReduce framework had some limitations that have been tackled by many research efforts in several followup works after its introduction. This article provides a comprehensive survey for a family of approaches and mechanisms of large scale data processing mechanisms that have been implemented based on the original idea of the MapReduce framework and are currently gaining a lot of momentum in both research and industrial communities. We also cover a set of introduced systems that have been implemented to provide declarative programming interfaces on top of the MapReduce framework. In addition, we review several large scale data processing systems that resemble some of the ideas of the MapReduce framework for different purposes and application scenarios. Finally, we discuss some of the future research directions for implementing the next generation of MapReduce-like solutions.Comment: arXiv admin note: text overlap with arXiv:1105.4252 by other author