Skip to main content
Article thumbnail
Location of Repository

Self-adaptive Grid Resource Monitoring and discovery

By Hélène Niuklan Lim Choi Keung

Abstract

The Grid provides a novel platform where the scientific and engineering communities can share data and computation across multiple administrative domains. There are several key services that must be offered by Grid middleware; one of them being the Grid Information Service( GIS). A GIS is a Grid middleware component which maintains information about hardware, software, services and people participating in a virtual organisation( VO). There is an inherent need in these systems for the delivery of reliable performance. This thesis describes a number of approaches which detail the development and application of a suite of benchmarks for the prediction of the process of resource discovery and monitoring on the Grid. A series of experimental studies of the characterisation of performance using benchmarking, are carried out. Several novel predictive algorithms are presented and evaluated in terms of their predictive error. Furthermore, predictive methods are developed which describe the behaviour of MDS2 for a variable number of user requests. The MDS is also extended to include job information from a local scheduler; this information is queried using requests of greatly varying complexity. The response of the MDS to these queries is then assessed in terms of several performance metrics. \ud The benchmarking of the dynamic nature of information within MDS3 which is based on the Open Grid Services Architecture (OGSA), and also the successor to MDS2, is also carried out. The performance of both the pull and push query mechanisms is analysed. GridAdapt (Self-adaptive Grid Resource Monitoring) is a new system that is proposed, built upon the Globus MDS3 benchmarking. It offers self-adaptation, autonomy and admission control at the Index Service, whilst ensuring that the MIDS is not overloaded and can meet its quality-of-service,f or example,i n terms of its average response time for servicing synchronous queries and the total number of queries returned per unit time

Topics: QA76
OAI identifier: oai:wrap.warwick.ac.uk:1176

Suggested articles

Citations

  1. (2005). 2.2: Creating a Hierarchical GIIS. http: //www. globus. org/toolkit/docs/2-4/mds/ hierarchical-GIIS. pdf, last accessed on
  2. 22,2005. http: //infnforge. cnaf. infn. it /glue inf omode l/uplo ads /Spe c/GLUEInf oModel%- 1 -2-draf t -8. pdf , last accessed on
  3. (2002). A blueprint for introducing disruptive technology into the Internet. doi
  4. A constraint language approach to Grid resource selection.
  5. (1997). A directory service for configuring high-performance distributed computations. doi
  6. (2002). A Grid monitoring architecture. Informational Document, GGF Performance Working Group,
  7. A metacomputing infrastructure toolkit. doi
  8. (2005). A monitoring and management tool for distributed systems.
  9. (2003). A performance oriented Migration framework for the Grid. In doi
  10. (2003). A performance study of monitoring and information services for distributed systems. doi
  11. (2005). Access Protocol (0). http: //www. ietf . org/rf c/rf c225 1. txt, last accessed on September 3,2005. 317 [65] Lightweight Directory Access Protocol (0): Attribute Syntax Definitions. ftp: //ftp-isiedu/in-notes/rf c2252. txt, last accessed on
  12. (2005). Administrator's Guide. http: //www. openldap. org/devel/admin/ guide. html, last accessed on
  13. (2002). Agent-based resource management for Grid computing. doi
  14. (2005). An overview of the Web Services Inspection Language. http: //www. ibm. com/developerworks /webs ervi ce sA ibrary/ws - ws i lover/, last accessed on
  15. (2005). Analysis of the globus toolkit grid information service, version 1.0.
  16. (2002). ARMS: An Agent-based resource management system for Grid computing. Scientific Programming Special Issue on Grid Computing, doi
  17. (2001). Autonomic Computing: IBM's perspective on the state of information technology.
  18. (2005). Configuration Files.
  19. (2001). DAML+OIL Reference Description.
  20. (2002). DataGrid information and monitoring services (WP3) architecture: Design, requirements and evaluation criteria.
  21. (2002). Design and evaluation of a resource selection framework for Grid applications. In doi
  22. DiPerF: An Automated DIstributed PERformance Testing Framework. doi
  23. (2001). Dynamic instrumentation and performance prediction of application execution. doi
  24. (1998). Dynamically forecasting network performance using the Network Weather Service. doi
  25. (2004). editors. The Grid2: Blueprint for a New Computing Infrastructure.
  26. (2005). Enabling Grids for e-Science in Europe.
  27. (2001). Event service specification, Version 1.1,
  28. (2001). Event services in high performance systems. Cluster Computing. doi
  29. (1997). Forecasting network performance to support dynamic scheduling using the Network Weather Service. In doi
  30. (2005). Forum. http: //www. ggf. org, last accessed on November 8,2005. [411 Globus: Extending GRIS Functionality. http: //www-f p. globus. org/mds/ext ending- gr is. html, last accessed on
  31. (2001). G-commerce: Market formulations controlling resource allocation in the computational Grid. doi
  32. (2005). Ganglia monitoring system. http: //ganglia. doi
  33. (2005). Globus Toolkit Information Services: Monitoring and Discovery System (MDS). http: //www. globus. org/toolkit/mds/, last accessed on November 3,2005. [441 GLUE Information Model. http: //inf nf orge. cnaf . inf n. it/glueinf omodel, last accessed on
  34. Globus Toolkit version 4: Software for service-oriented systems. doi
  35. (2001). Grid information services for distributed resource sharing. doi
  36. Grid Information Services using software agents. doi
  37. Grid resource management information services for scientific computing. doi
  38. Grid services for distributed systems integration. doi
  39. (2005). GRIS Specification Document: Creating New Information Providers.
  40. (2005). http: //www. globus. org/t oolkit /mds /glues chemalink. html, last accessed on
  41. (2004). http: //www. w3. org/ TR/ws-arch/, last accessed on November 9,2005. 321 [121] Web services description language (WSDL) 1.1. http: //www. w3. org/TR/wsdl, last accessed on
  42. (1999). Joint research activity 1: Information and monitoring, Enabling Grids for e-Science in Europe. http: //hepunx. rl. ac. uk/egee/jra. 1-uk/, last accessed on November 9,2005.
  43. Localised workload management using performance prediction and QoS contracts.
  44. (1998). Matchmaking: distributed resource management for high throughput computing. In doi
  45. (2003). MonALISA: A distributed monitoring service architecture.
  46. (2005). MonALISA: MONitoring Agents using a Large Integrated Services Architecture.
  47. (2006). Monitoring and Discovery in a Web Services Framework: Functionality and Performance
  48. (2005). OASIS: Advancing e-business standards since 1993. http: //www. oas is- open. org/, last accessed on November 12,2005. [871 OASIS UIDDI: Advancing web services discovery standard, http: //www. uddi. org/, last accessed on
  49. (2005). Openl-DAP: Comm u nity-developed LDAP software.
  50. (2001). Optimisation of application execution on dynamic systems. doi
  51. (2005). org/, last accessed on November 16,2005.
  52. (2003). Perform a nce- based middleware services for Grid Computing computing systems. doi
  53. Performance modelling of a self-adaptive and self-optimising resource monitoring system for dynamic Grid environments. doi
  54. (2003). Performance prediction and its use in parallel and distributed computing systems computing systems. doi
  55. (2006). Performance prediction and its use in parallel and distributed computing systems. Future Generation Computer Systems, Special Issue on System Performance Analysis and Evaluation, doi
  56. (2003). Predicting the performance of Globus monitoring and discovery service (MDS-2) queries. doi
  57. (2005). R-GMA: An Information Integration System for Grid Monitoring. doi
  58. (1994). Realizing the information future: The internet and beyond. doi
  59. (2002). Representing dynamic performance information in Grid environments with the Network Weather Service. doi
  60. (2005). Research Councils UK.
  61. (2005). RRDTool Time-Series Data Archiving System and Graphing.
  62. Self-adaptive and self-optimising resource monitoring for dynamic Grid environments. doi
  63. (2005). Services Inspection Language (WS-Inspection) 1.0. http: //www- 128. ibm. com/developerworks/library/specif ication/ws-wsilspec/, last accessed on
  64. (2000). Simple Object Access Protocol (SOAP). http: //www. w3. org/TR/soap/, last accessed on November 9,2005.
  65. (2002). Specification: Web services security (WS-Security), Version 1.0,
  66. (2005). Task Force. http: //www. dmtf . org/, last accessed on
  67. (2005). Technology. http: //www. sun. com/j ini, last accessed on
  68. (2000). The anatomy of the Grid: Enabling scalable virtual organizations. doi
  69. (2005). The DataGrid project. http: //eu-datagrid.
  70. (2005). The EU DataTAG Project, Research and Technological Development for a Data TransAtlantic Grid.
  71. (2000). The Globus Alliance. http: //www. globus. org, last accessed on September 5,2005. [1101 The LDAP Data Interchange Format (LDIF)
  72. The Globus monitoring and discovery service (MDS-2): a performance analysis. doi
  73. (2001). The GrADS project: Software support for high-level Grid application development.
  74. (1998). The GRID: Blueprint for a New Computing Infrastructure. doi
  75. (1999). The Jini specification.
  76. (2005). The MySQL Database. http: //mysql. com, last accessed on doi
  77. (1998). The NetLogger methodology for high performance distributed systems performance analysis. doi
  78. (2005). The Network Weather Service. doi
  79. The White Rose Grid e-Science Centre.
  80. (2005). The WS-Resource Framework: Version 1.0,03/05/2004.
  81. (1994). Time Series Analysis: Forecasting and Control. Englewood Cliffs,
  82. (1990). Toward a universal random number generator. doi
  83. Trusting agents for Grid Computing. Special Track, held as part of the doi
  84. Usage of LDAP in Globus. http: //www. cs. ucy. ac. cy/ crossgrid/cygriddl/ldap-in-globus. pdf, last accessed on September 5,2005.
  85. (2000). Use of performance technology for the management of distributed systems. doi
  86. Web Ontology Language (OWL) Reference version 1.0.2002.
  87. Wide area cluster monitoring with Ganglia. In doi
  88. (2005). World Wide Web Consortium 14-January-1999. http: //www. w3. org/TR/ 1999/REC- xmi- names - 19990114/#dt - qname, last accessed on
  89. (2001). WX Semantic Web activity. Semantic Web Kick-off Seminar in Finland, part of Semantic Web in Finland effort,
  90. (2005). XDR: External Data Representation Standard. http: //www. faqs. org/rfcs/ rf cl832. html, last accessed on
  91. (2006). XSL Transformations (XSLT) Version 1.0. http: //www. w3. org/TR/xslt, WX Recornmendation, last accessed on

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.