Sorting Technique- An Improvement towards HDD Utilization in HADOOP

A data is tremendously increasing everyday due to updates that are updated after a particular time interval. Therefore it is very hard to handle such a large data that is updated on common basis. Ample number of techniques was proposed in order to enhance the performance and efficiency. In this paper, a technique called as “sorting technique” is implemented so as to increase the mean value. Load of hadoop cluster will decrease in steeply manner when this technique is implemented. The results clearly defines the efficiency of the technique which is used and hence it also proves to be efficient than the traditional techniques

Evolutionary Neural Network Based Energy Consumption Forecast for Cloud Computing

The success of Hadoop, an open-source framework for massively parallel and distributed computing, is expected to drive energy consumption of cloud data centers to new highs as service providers continue to add new infrastructure, services and capabilities to meet the market demands. While current research on data center airflow management, HVAC (Heating, Ventilation and Air Conditioning) system design, workload distribution and optimization, and energy efficient computing hardware and software are all contributing to improved energy efficiency, energy forecast in cloud computing remains a challenge. This paper reports an evolutionary computation based modeling and forecasting approach to this problem. In particular, an evolutionary neural network is developed and structurally optimized to forecast the energy load of a cloud data center. The results, both in terms of forecasting speed and accuracy, suggest that the evolutionary neural network approach to energy consumption forecasting for cloud computing is highly promising

Efficient Mapping of Large-scale Data under Heterogeneous Big Data Computing Systems

Hadoop biological systems become progressively significant for professionals of huge scale information examination, they likewise acquire huge energy cost. This pattern is dynamic up the requirement for planning energy-effective Hadoop clusters so as to lessen the operational costs and the carbon emanation related with its energy utilization. Be that as it may, in spite of broad investigations of the issue, existing methodologies for energy proficiency have not completely measured the heterogeneity of both workloads. So that here enhancing the model by find that heterogeneity-unaware task task methodologies are hindering to both execution and energy effectiveness of Hadoop clusters. Our perception demonstrates that even heterogeneity-mindful methods that intend to decrease the job fulfillment time don't ensure a decrease in energy utilization of heterogeneous machines. We propose E-Ant which plans to get better the general energy utilization in a heterogeneous Hadoop group without giving up job execution. It adaptively plans heterogeneous workloads on energy-effective machines. E-Ant utilizes a subterranean insect state improvement approach that creates task assignment arrangements dependent on the input of each jobs energy utilization by Tasktrackers and also we incorporate DVFS method with E-Ant to further improve the energy proficiency

Hadoop Performance Analysis on Raspberry Pi for DNA Sequence Alignment

The rapid development of electronic data has brought two major challenges, namely, how to store big data and how to process it. Two main problems in processing big data are the high cost and the computational power. Hadoop, one of the open source frameworks for processing big data, uses distributed computational model designed to be able to run on commodity hardware. The aim of this research is to analyze Hadoop cluster on Raspberry Pi as a commodity hardware for DNA sequence alignment. Six B Model Raspberry Pi and a Biodoop library were used in this research for DNA sequence alignment. The length of the DNA used in this research is between 5,639 bp and 13,271 bp. The results showed that the Hadoop cluster was running on the Raspberry Pi with average usage of processor 73.08%, 334.69 MB of memory and 19.89 minutes of job time completion. The distribution of Hadoop data file blocks was found to reduce processor usage as much as 24.14% and memory usage as much as 8.49%. However this increased job processing time as much as 31.53%

On the Energy Efficiency of MapReduce Shuffling Operations in Data Centers

This paper aims to quantitatively measure the impact of different data centers networking topologies on the performance and energy efficiency of shuffling operations in MapReduce. Mixed Integer Linear Programming (MILP) models are utilized to optimize the shuffling in several data center topologies with electronic, hybrid, and all-optical switching while maximizing the throughput and reducing the power consumption. The results indicate that the networking topology has a significant impact on the performance of MapReduce. They also indicate that with comparable performance, optical-based data centers can achieve an average of 54% reduction in the energy consumption when compared to electronic switching data centers

Energy-aware cluster reconfiguration algorithm for the big data analytics platform spark

The development of Cloud computing and data analytics technologies has made it possible to process big data faster. Distributed computing schemes, for instance, can help to reduce the time required for data analysis and thus enhance its efficiency. However, fewer researchers have paid attention to the problem of the high-energy consumption of the cluster, placing a heavy burden on the environment, especially when the number of nodes is extremely large. As a consequence, the principle of sustainable development is violated. Considering this problem, this paper proposes an approach that can be applied to remove less-efficient nodes or to migrate over-utilized nodes of the cluster so as to adjust the load of the cluster properly and thereby achieve the goal of energy conservation. Furthermore, in order to testify the performance of the proposed methodology, we present the simulation results implemented by using CloudSim