MC64-Cluster: Many-Core CPU Cluster Architecture and Performance Analysis in B-Tree Searches

The MC64-Cluster computer platform was designed, based on many-core CPU microprocessors: Tile64. MC64-Cluster architecture was outlined in terms of both hardware and software, including commands available to manage jobs and provided application programming interfaces to communicate and synchronize tiles, making this system easy to use. Massively, concurrent-searches of keys in B-trees, which are used in many applications, including bioinformatics, were used. Remarkable performance improvements were obtained when the cluster resources were combined with those available in host machine (hybrid or heterogeneous environments). These results were even more outstanding when analyzed in terms of performance-per-watt, highlighting their green-computing advantages. Together with the cluster architecture, they represent the main contributions of this work. To our knowledge, this is the first cluster implementation of this kind being developed

Elastic provisioning of virtual Hadoop clusters in OpenStack-based Clouds

MapReduce programming model and its implementations, such as the widely diffused Apache Hadoop framework, are spreading more and more due to their inherent capacity of enabling scalable processing of large-scale datasets. The advent of Cloud has further boosted this trend through the provisioning of virtual Hadoop clusters, easily configurable and accessible according to the Platform as a Service (PaaS) model, deployed over existing Cloud Infrastructure as a Service (IaaS) platforms. However, the coexistence of multiple virtual Hadoop clusters competing for the same shared physical resources requires new management solutions able to dynamically reconfigure and rebalance the placement of Hadoop service components over the virtualized IaaS platform. This paper proposes ESAMAR (Elastic Sahara MApReduce), a novel support based on a cross-layer PaaS-IaaS management approach to transparently grant elasticity and efficiency at the Hadoop PaaS level. ESAMAR monitors the performance of Hadoop clusters at both IaaS and physical layers and exploits load balancing techniques, with full awareness of virtual Hadoop clusters and resources at PaaS/IaaS levels. We deeply assessed our framework in a realistic scenario based on the open source OpenStack; collected results demonstrate the effectiveness and the suitability of our management techniques that contribute to reduce Hadoop job completion time, even under challenging heavy-loaded Cloud system conditions