Design Architecture-Based on Web Server and Application Cluster in Cloud Environment

Cloud has been a computational and storage solution for many data centric organizations. The problem today those organizations are facing from the cloud is in data searching in an efficient manner. A framework is required to distribute the work of searching and fetching from thousands of computers. The data in HDFS is scattered and needs lots of time to retrieve. The major idea is to design a web server in the map phase using the jetty web server which will give a fast and efficient way of searching data in MapReduce paradigm. For real time processing on Hadoop, a searchable mechanism is implemented in HDFS by creating a multilevel index in web server with multi-level index keys. The web server uses to handle traffic throughput. By web clustering technology we can improve the application performance. To keep the work down, the load balancer should automatically be able to distribute load to the newly added nodes in the server

Dynamic Physiological Partitioning on a Shared-nothing Database Cluster

Traditional DBMS servers are usually over-provisioned for most of their daily workloads and, because they do not show good-enough energy proportionality, waste a lot of energy while underutilized. A cluster of small (wimpy) servers, where its size can be dynamically adjusted to the current workload, offers better energy characteristics for these workloads. Yet, data migration, necessary to balance utilization among the nodes, is a non-trivial and time-consuming task that may consume the energy saved. For this reason, a sophisticated and easy to adjust partitioning scheme fostering dynamic reorganization is needed. In this paper, we adapt a technique originally created for SMP systems, called physiological partitioning, to distribute data among nodes, that allows to easily repartition data without interrupting transactions. We dynamically partition DB tables based on the nodes' utilization and given energy constraints and compare our approach with physical partitioning and logical partitioning methods. To quantify possible energy saving and its conceivable drawback on query runtimes, we evaluate our implementation on an experimental cluster and compare the results w.r.t. performance and energy consumption. Depending on the workload, we can substantially save energy without sacrificing too much performance

Improving Energy Efficiency of MapReduce Framework using Dynamic Scheduling of Work

Most common huge volume data processing programs do counting, sorting, merging etc. Such programs require to perform first a computation on each record that is it requires to map an operation to each record. Then combine the output of these operations in appropriate way to get the answer that is apply a reduce operation to groups of records. MapReduce runtime environment takes care of parallelizing their execution and coordinating their inputs/outputs. Here we are concern about energy efficiency in MapReduce framework so we are proposing dynamic scheduling of workload which offers dynamic load balancing method. Load balancing is the methodology of distributing the load among different node of a distributed framework to enhance both resource usage and reaction time while likewise keeping away from a circumstance where a percentage of the node are intensely stacked while different node are sit out of gear or doing next to no work. An answer for unbalance circumstance is to utilize parallelization approaches yet at the same time node will stay overwhelming. In this paper, we propose an integrated. We are proposing a methodology where the MapReduce concept introduced into the MongoDB with NoSQL as a back end to implement the MapReduce

Clustered Node Based Load Balancing In Distributed Environment

Cloud computing having tremendous growth on recent years but it is not segregation on shared clouds.  Distributed file systems are key building blocks for cloud computing applications based on the Map Reduce programming paradigm. In such file systems, nodes simultaneously serve computing and storage functions; a file is partitioned into a number of chunks allocated in distinct nodes so that Map Reduce tasks can be performed in parallel over the nodes. Data storage and communication which are to be done in huge amount, in such cases clouds are most provably used. "The cloud", also focuses on increasing the effectiveness of the public resources. Cloud resources are usually not only shared by multiple users but are also vigorously reallocated per demand. This can work for apportioning resources to users .But In the time of apportionment  these are indeed .So In this paper we are introducing novel mechanism. We investigate to implement security provided for cloud computing and Evaluate the Quality of Service-QOS (Ex. Response Time) of whole system. In cloud computing one server controls number of sub servers, files, it can add, delete, and append dynamically  Freight stabilization in the cloud computing surroundings has an imperative impact on the performance. Excellent freight stabilizing makes cloud computing more efficient and improves user satisfaction. In this paper we are presenting freight stabilizing techniques for cloud segregating

Load Balancing Model in Cloud Computing Environment

As organizations need to focus on maintaining their datacenter in order to store huge amount of data of their clients. So cloud computing is one of the greatest platform which provides storage of data in very lower cost to organizations and available for all time over the internet. But it has some critical issues like load management. Load Balancing approach is based on Cloud partitioning concept. Load balancing is the process of distributing load over the different nodes which provides good resource utilization when nodes are overloaded with job. In this approach, we are using model in which memory size of every partition will be checked linearly and for efficient retrieval of user’s file, we use Bloom filter algorithm. DOI: 10.17762/ijritcc2321-8169.15036

Self-management for large-scale distributed systems

Autonomic computing aims at making computing systems self-managing by using autonomic managers in order to reduce obstacles caused by management complexity. This thesis presents results of research on self-management for large-scale distributed systems. This research was motivated by the increasing complexity of computing systems and their management. In the first part, we present our platform, called Niche, for programming self-managing component-based distributed applications. In our work on Niche, we have faced and addressed the following four challenges in achieving self-management in a dynamic environment characterized by volatile resources and high churn: resource discovery, robust and efficient sensing and actuation, management bottleneck, and scale. We present results of our research on addressing the above challenges. Niche implements the autonomic computing architecture, proposed by IBM, in a fully decentralized way. Niche supports a network-transparent view of the system architecture simplifying the design of distributed self-management. Niche provides a concise and expressive API for self-management. The implementation of the platform relies on the scalability and robustness of structured overlay networks. We proceed by presenting a methodology for designing the management part of a distributed self-managing application. We define design steps that include partitioning of management functions and orchestration of multiple autonomic managers. In the second part, we discuss robustness of management and data consistency, which are necessary in a distributed system. Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of Robust Management Elements, which are able to heal themselves under continuous churn. Our approach is based on replicating a management element using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. For data consistency, we propose a majority-based distributed key-value store supporting multiple consistency levels that is based on a peer-to-peer network. The store enables the tradeoff between high availability and data consistency. Using majority allows avoiding potential drawbacks of a master-based consistency control, namely, a single-point of failure and a potential performance bottleneck. In the third part, we investigate self-management for Cloud-based storage systems with the focus on elasticity control using elements of control theory and machine learning. We have conducted research on a number of different designs of an elasticity controller, including a State-Space feedback controller and a controller that combines feedback and feedforward control. We describe our experience in designing an elasticity controller for a Cloud-based key-value store using state-space model that enables to trade-off performance for cost. We describe the steps in designing an elasticity controller. We continue by presenting the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores that combines feedforward and feedback control