450 research outputs found
Observing the clouds : a survey and taxonomy of cloud monitoring
This research was supported by a Royal Society Industry Fellowship and an Amazon Web Services (AWS) grant. Date of Acceptance: 10/12/2014Monitoring is an important aspect of designing and maintaining large-scale systems. Cloud computing presents a unique set of challenges to monitoring including: on-demand infrastructure, unprecedented scalability, rapid elasticity and performance uncertainty. There are a wide range of monitoring tools originating from cluster and high-performance computing, grid computing and enterprise computing, as well as a series of newer bespoke tools, which have been designed exclusively for cloud monitoring. These tools express a number of common elements and designs, which address the demands of cloud monitoring to various degrees. This paper performs an exhaustive survey of contemporary monitoring tools from which we derive a taxonomy, which examines how effectively existing tools and designs meet the challenges of cloud monitoring. We conclude by examining the socio-technical aspects of monitoring, and investigate the engineering challenges and practices behind implementing monitoring strategies for cloud computing.Publisher PDFPeer reviewe
C2MS: Dynamic Monitoring and Management of Cloud Infrastructures
Server clustering is a common design principle employed by many organisations
who require high availability, scalability and easier management of their
infrastructure. Servers are typically clustered according to the service they
provide whether it be the application(s) installed, the role of the server or
server accessibility for example. In order to optimize performance, manage load
and maintain availability, servers may migrate from one cluster group to
another making it difficult for server monitoring tools to continuously monitor
these dynamically changing groups. Server monitoring tools are usually
statically configured and with any change of group membership requires manual
reconfiguration; an unreasonable task to undertake on large-scale cloud
infrastructures.
In this paper we present the Cloudlet Control and Management System (C2MS); a
system for monitoring and controlling dynamic groups of physical or virtual
servers within cloud infrastructures. The C2MS extends Ganglia - an open source
scalable system performance monitoring tool - by allowing system administrators
to define, monitor and modify server groups without the need for server
reconfiguration. In turn administrators can easily monitor group and individual
server metrics on large-scale dynamic cloud infrastructures where roles of
servers may change frequently. Furthermore, we complement group monitoring with
a control element allowing administrator-specified actions to be performed over
servers within service groups as well as introduce further customized
monitoring metrics. This paper outlines the design, implementation and
evaluation of the C2MS.Comment: Proceedings of the The 5th IEEE International Conference on Cloud
Computing Technology and Science (CloudCom 2013), 8 page
Technical support for Life Sciences communities on a production grid infrastructure
Production operation of large distributed computing infrastructures (DCI)
still requires a lot of human intervention to reach acceptable quality of
service. This may be achievable for scientific communities with solid IT
support, but it remains a show-stopper for others. Some application execution
environments are used to hide runtime technical issues from end users. But they
mostly aim at fault-tolerance rather than incident resolution, and their
operation still requires substantial manpower. A longer-term support activity
is thus needed to ensure sustained quality of service for Virtual Organisations
(VO). This paper describes how the biomed VO has addressed this challenge by
setting up a technical support team. Its organisation, tooling, daily tasks,
and procedures are described. Results are shown in terms of resource usage by
end users, amount of reported incidents, and developed software tools. Based on
our experience, we suggest ways to measure the impact of the technical support,
perspectives to decrease its human cost and make it more community-specific.Comment: HealthGrid'12, Amsterdam : Netherlands (2012
Monitoring Cluster on Online Compiler with Ganglia
Ganglia is an open source monitoring system for high performance computing (HPC) that collect both a whole cluster and every nodes status and report to the user. We use Ganglia to monitor our spasi.informatika.lipi.go.id (SPASI), a customized-fedora10-based cluster, for our cluster online compiler, CLAW (cluster access through web). Our experience on using Ganglia shows that Ganglia has a capability to view our cluster status and allow us to track them
Development of Grid e-Infrastructure in South-Eastern Europe
Over the period of 6 years and three phases, the SEE-GRID programme has
established a strong regional human network in the area of distributed
scientific computing and has set up a powerful regional Grid infrastructure. It
attracted a number of user communities and applications from diverse fields
from countries throughout the South-Eastern Europe. From the infrastructure
point view, the first project phase has established a pilot Grid infrastructure
with more than 20 resource centers in 11 countries. During the subsequent two
phases of the project, the infrastructure has grown to currently 55 resource
centers with more than 6600 CPUs and 750 TBs of disk storage, distributed in 16
participating countries. Inclusion of new resource centers to the existing
infrastructure, as well as a support to new user communities, has demanded
setup of regionally distributed core services, development of new monitoring
and operational tools, and close collaboration of all partner institution in
managing such a complex infrastructure. In this paper we give an overview of
the development and current status of SEE-GRID regional infrastructure and
describe its transition to the NGI-based Grid model in EGI, with the strong SEE
regional collaboration.Comment: 22 pages, 12 figures, 4 table
Beyond Nagios - Design of a cloud monitoring system
The paper describes a monitoring system specially designed for cloud infrastructures. The features that are relevant for such distributed application are -) scalability, that allows utilization in systems of thousands of nodes, -) flexibility, to be customized for a large number of applications, -) openness, to allow the coexistence of user and administration monitoring. We take as a starting point the Nagios monitoring system, that has been successfully used for Grid monitoring and is still used for clouds. We analyze its shortcomings when applied to cloud monitoring, and propose a new monitoring system, that we call Rocmon, that sums up Nagios experience with a cloud perspective. Like Nagios, Rocmon is plugin-oriented to be flexible. To be fully inter-operable and long-living, it uses standard tools: the OGF OCCI for the configuration interface, the REST paradigm to take advantage of Web tools, and HTML5 WebSockets for data transfers. The design is checked with an open source Ruby implementation featuring the most relevant aspects
The INFN-grid testbed
The Italian INFN-Grid Project is committed to set-up, run and manage an unprecedented nation-wide Grid infrastructure. The implementation and use of this INFN-Grid Testbed is presented and discussed. Particular care and attention are devoted to those activities, relevant for the management of the Testbed, carried out by the INFN within international Grid Projects
RSV: OSG Fabric Monitoring and Interoperation with WLCG Monitoring Systems
Presented at CHEP 2009 (Computing in High Energy and Nuclear Physics)
- …