Enabling quantitative data analysis through e-infrastructures

This paper discusses how quantitative data analysis in the social sciences can engage with and exploit an e-Infrastructure. We highlight how a number of activities which are central to quantitative data analysis, referred to as ‘data management’, can benefit from e-infrastructure support. We conclude by discussing how these issues are relevant to the DAMES (Data Management through e-Social Science) research Node, an ongoing project that aims to develop e-Infrastructural resources for quantitative data analysis in the social sciences

An Information system for grid environment

the function of virtualization has ended up crucial in the grid computing community to bridge the distance between computation useful resource configuration and task requirements. The undertaking of virtualization inside the grid computing environment has set every other project to monitor and control the digital machines within the grid. To solve this hassle of tracking the virtual machines we advise a virtual information device on the way to provide the whole records about the digital parameters of the gadget. This information might be higher useful for Grid control device to decorate the management of digital sources.

An ActOn-based Semantic Information Service for EGEE

We describe a semantic information service that aggregates metadata from a large number of information sources of a large-scale Grid infrastructure. It uses an ontology-based information integration architecture (ActOn) suitable for the highly dynamic distributed information sources available in Grid systems, where information changes frequently and where the information of distributed sources has to be aggregated in order to solve complex queries. These two challenges are addressed by a Metadata Cache that works with an update-on-demand policy and by an information source selection module that selects the most suitable source at a given point in time. We have evaluated the quality of this information service, and compared it with other similar services from the EGEE production testbed, with promising results

XtreemOS application execution management: a scalable approach

Designing a job management system for the Grid is a non-trivial task. While a complex middleware can give a lot of features, it often implies sacrificing performance. Such performance loss is especially noticeable for small jobs. A Job Manager’s design also affects the capabilities of the monitoring system. We believe that monitoring a job or asking for a job status should be fast and easy, like doing a simple ’ps’. In this paper, we present the job management of XtreemOS - a Linux-based operating system to support Virtual Organizations for Grid. This management is performed inside the Application Execution Manager (AEM). We evaluate its performance using only one job manager plus the built-in monitoring infrastructure. Furthermore, we present a set of real-world applications using AEM and its features. In XtreemOS we avoid reinventing the wheel and use the Linux paradigm as an abstraction.Peer ReviewedPostprint (published version

Optimization Scheme for Storing and Accessing Huge Number of Small Files on HADOOP Distributed File System

Hadoop is a distributed framework which uses a simple programming model for the processing of huge datasets over the network of computers. Hadoop is used across multiple machines to store very large files, which are normally in the range of gigabytes to terabytes. High throughput access is acquired using HDFS for applications with huge datasets. In Hadoop Distributed File System(HDFS), a small file is the one which is smaller than 64MB which is the default block size of HDFS. Hadoop performance is better with a small number of large files, as opposed to a huge number of small files. Many organizations like financial firms need to handle a large number of small files daily. Low performance and high resource consumption are the bottlenecks of traditional method. To reduce the processing time and memory required to handle a large set of small files, an efficient solution is needed which will make HDFS work better for large data of small files. This solution should combine many small files into a large file and treat these large files as an individual file. It should also be able to store these large files into HDFS and retrieve any small file when needed

Cloud Computing and Grid Computing 360-Degree Compared

Cloud Computing has become another buzzword after Web 2.0. However, there are dozens of different definitions for Cloud Computing and there seems to be no consensus on what a Cloud is. On the other hand, Cloud Computing is not a completely new concept; it has intricate connection to the relatively new but thirteen-year established Grid Computing paradigm, and other relevant technologies such as utility computing, cluster computing, and distributed systems in general. This paper strives to compare and contrast Cloud Computing with Grid Computing from various angles and give insights into the essential characteristics of both.Comment: IEEE Grid Computing Environments (GCE08) 200

Research and development of accounting system in grid environment

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The Grid has been recognised as the next-generation distributed computing paradigm by seamlessly integrating heterogeneous resources across administrative domains as a single virtual system. There are an increasing number of scientific and business projects that employ Grid computing technologies for large-scale resource sharing and collaborations. Early adoptions of Grid computing technologies have custom middleware implemented to bridge gaps between heterogeneous computing backbones. These custom solutions form the basis to the emerging Open Grid Service Architecture (OGSA), which aims at addressing common concerns of Grid systems by defining a set of interoperable and reusable Grid services. One of common concerns as defined in OGSA is the Grid accounting service. The main objective of the Grid accounting service is to ensure resources to be shared within a Grid environment in an accountable manner by metering and logging accurate resource usage information. This thesis discusses the origins and fundamentals of Grid computing and accounting service in the context of OGSA profile. A prototype was developed and evaluated based on OGSA accounting-related standards enabling sharing accounting data in a multi-Grid environment, the World-wide Large Hadron Collider Grid (WLCG). Based on this prototype and lessons learned, a generic middleware solution was also implemented as a toolkit that eases migration of existing accounting system to be standard compatible.Engineering and Physical Sciences Research Council (EPSRC), Stanford Universit

Quality of Experience Framework for Cloud Computing (QoC)

Cloud computing provides platform for pay per use services such as software (e.g., database, data processing, application servers, etc.), hardware (e.g., GPUs, CPUs, storage, etc.) and platforms (e.g., Linux, Microsoft Windows and Apple macOS). Previous cloud frameworks use fix policies that do not have the functionality to upgrade services on demand when the user do not receive services according to Service Level Agreement (SLA). Also, there was a lack of functionality to monitor external network and client device resources. This paper presents Quality of experience framework for Cloud computing (QoC) for monitoring the Quality of Experience (QoE) of the end user using video streaming services in the cloud computing environment. The management platform is used for administration purpose in QoC framework that provides facility to easily manage the cloud environment and provide services according to SLA via runtime policy change. The objective QoE/QoS section will automatically monitor the Quality of Service (QoS) data. It will also compare and analyze the subjective QoE submitted by the users and objective QoS data collected by agent based framework for accurate QoE prediction and proper management. The proposed QoC framework has new features of real time network monitoring, client device monitoring and allows changing policy in runtime environment which to our knowledge is currently not provided by existing frameworks