11,596 research outputs found
Power Management Techniques for Data Centers: A Survey
With growing use of internet and exponential growth in amount of data to be
stored and processed (known as 'big data'), the size of data centers has
greatly increased. This, however, has resulted in significant increase in the
power consumption of the data centers. For this reason, managing power
consumption of data centers has become essential. In this paper, we highlight
the need of achieving energy efficiency in data centers and survey several
recent architectural techniques designed for power management of data centers.
We also present a classification of these techniques based on their
characteristics. This paper aims to provide insights into the techniques for
improving energy efficiency of data centers and encourage the designers to
invent novel solutions for managing the large power dissipation of data
centers.Comment: Keywords: Data Centers, Power Management, Low-power Design, Energy
Efficiency, Green Computing, DVFS, Server Consolidatio
A Low Cost Two-Tier Architecture Model For High Availability Clusters Application Load Balancing
This article proposes a design and implementation of a low cost two-tier
architecture model for high availability cluster combined with load-balancing
and shared storage technology to achieve desired scale of three-tier
architecture for application load balancing e.g. web servers. The research work
proposes a design that physically omits Network File System (NFS) server nodes
and implements NFS server functionalities within the cluster nodes, through Red
Hat Cluster Suite (RHCS) with High Availability (HA) proxy load balancing
technologies. In order to achieve a low-cost implementation in terms of
investment in hardware and computing solutions, the proposed architecture will
be beneficial. This system intends to provide steady service despite any system
components fails due to uncertainly such as network system, storage and
applications.Comment: Load balancing, high availability cluster, web server cluster
Computing Web-scale Topic Models using an Asynchronous Parameter Server
Topic models such as Latent Dirichlet Allocation (LDA) have been widely used
in information retrieval for tasks ranging from smoothing and feedback methods
to tools for exploratory search and discovery. However, classical methods for
inferring topic models do not scale up to the massive size of today's publicly
available Web-scale data sets. The state-of-the-art approaches rely on custom
strategies, implementations and hardware to facilitate their asynchronous,
communication-intensive workloads.
We present APS-LDA, which integrates state-of-the-art topic modeling with
cluster computing frameworks such as Spark using a novel asynchronous parameter
server. Advantages of this integration include convenient usage of existing
data processing pipelines and eliminating the need for disk writes as data can
be kept in memory from start to finish. Our goal is not to outperform highly
customized implementations, but to propose a general high-performance topic
modeling framework that can easily be used in today's data processing
pipelines. We compare APS-LDA to the existing Spark LDA implementations and
show that our system can, on a 480-core cluster, process up to 135 times more
data and 10 times more topics without sacrificing model quality.Comment: To appear in SIGIR 201
Towards Autonomic Service Provisioning Systems
This paper discusses our experience in building SPIRE, an autonomic system
for service provision. The architecture consists of a set of hosted Web
Services subject to QoS constraints, and a certain number of servers used to
run session-based traffic. Customers pay for having their jobs run, but require
in turn certain quality guarantees: there are different SLAs specifying charges
for running jobs and penalties for failing to meet promised performance
metrics. The system is driven by an utility function, aiming at optimizing the
average earned revenue per unit time. Demand and performance statistics are
collected, while traffic parameters are estimated in order to make dynamic
decisions concerning server allocation and admission control. Different utility
functions are introduced and a number of experiments aiming at testing their
performance are discussed. Results show that revenues can be dramatically
improved by imposing suitable conditions for accepting incoming traffic; the
proposed system performs well under different traffic settings, and it
successfully adapts to changes in the operating environment.Comment: 11 pages, 9 Figures,
http://www.wipo.int/pctdb/en/wo.jsp?WO=201002636
Scheduling of data-intensive workloads in a brokered virtualized environment
Providing performance predictability guarantees is increasingly important in cloud platforms, especially for data-intensive applications, for which performance depends greatly on the available rates of data transfer between the various computing/storage hosts underlying the virtualized resources assigned to the application. With the increased prevalence of brokerage services in cloud platforms, there is a need for resource management solutions that consider the brokered nature of these workloads, as well as the special demands of their intra-dependent components. In this paper, we present an offline mechanism for scheduling batches of brokered data-intensive workloads, which can be extended to an online setting. The objective of the mechanism is to decide on a packing of the workloads in a batch that minimizes the broker's incurred costs, Moreover, considering the brokered nature of such workloads, we define a payment model that provides incentives to these workloads to be scheduled as part of a batch, which we analyze theoretically. Finally, we evaluate the proposed scheduling algorithm, and exemplify the fairness of the payment model in practical settings via trace-based experiments
Adaptive Dispatching of Tasks in the Cloud
The increasingly wide application of Cloud Computing enables the
consolidation of tens of thousands of applications in shared infrastructures.
Thus, meeting the quality of service requirements of so many diverse
applications in such shared resource environments has become a real challenge,
especially since the characteristics and workload of applications differ widely
and may change over time. This paper presents an experimental system that can
exploit a variety of online quality of service aware adaptive task allocation
schemes, and three such schemes are designed and compared. These are a
measurement driven algorithm that uses reinforcement learning, secondly a
"sensible" allocation algorithm that assigns jobs to sub-systems that are
observed to provide a lower response time, and then an algorithm that splits
the job arrival stream into sub-streams at rates computed from the hosts'
processing capabilities. All of these schemes are compared via measurements
among themselves and with a simple round-robin scheduler, on two experimental
test-beds with homogeneous and heterogeneous hosts having different processing
capacities.Comment: 10 pages, 9 figure
A horizontally-scalable multiprocessing platform based on Node.js
This paper presents a scalable web-based platform called Node Scala which
allows to split and handle requests on a parallel distributed system according
to pre-defined use cases. We applied this platform to a client application that
visualizes climate data stored in a NoSQL database MongoDB. The design of Node
Scala leads to efficient usage of available computing resources in addition to
allowing the system to scale simply by adding new workers. Performance
evaluation of Node Scala demonstrated a gain of up to 74 % compared to the
state-of-the-art techniques.Comment: 8 pages, 7 figures. Accepted for publication as a conference paper
for the 13th IEEE International Symposium on Parallel and Distributed
Processing with Applications (IEEE ISPA-15
- …