Q-Strategy: A Bidding Strategy for Market-Based Allocation of Grid Services

The application of autonomous agents by the provisioning and usage of computational services is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic service provisioning and usage of Grid services, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems. The contributions of the paper are threefold. First, we present a bidding agent framework for implementing artificial bidding agents, supporting consumers and providers in technical and economic preference elicitation as well as automated bid generation by the requesting and provisioning of Grid services. Secondly, we introduce a novel consumer-side bidding strategy, which enables a goal-oriented and strategic behavior by the generation and submission of consumer service requests and selection of provider offers. Thirdly, we evaluate and compare the Q-strategy, implemented within the presented framework, against the Truth-Telling bidding strategy in three mechanisms – a centralized CDA, a decentralized on-line machine scheduling and a FIFO-scheduling mechanisms

Automated Bidding in Computing Service Markets. Strategies, Architectures, Protocols

This dissertation contributes to the research on Computational Mechanism Design by providing novel theoretical and software models - a novel bidding strategy called Q-Strategy, which automates bidding processes in imperfect information markets, a software framework for realizing agents and bidding strategies called BidGenerator and a communication protocol called MX/CS, for expressing and exchanging economic and technical information in a market-based scheduling system

Survey on job scheduling mechanisms in grid environment

Grid systems provide geographically distributed resources for both computational intensive and data-intensive applications.These applications generate large data sets.However, the high latency imposed by the underlying technologies; upon which the grid system is built (such as the Internet and WWW), induced impediment in the effective access to such huge and widely distributed data.To minimize this impediment, jobs need to be scheduled across grid environments to achieve efficient data access.Scheduling multiple data requests submitted by grid users onto the grid environment is NP-hard.Thus, there is no best scheduling algorithm that cuts across all grids computing environments.Job scheduling is one of the key research area in grid computing.In the recent past many researchers have proposed different mechanisms to help scheduling of user jobs in grid systems.Some characteristic features of the grid components; such as machines types and nature of jobs at hand means that a choice needs to be made for an appropriate scheduling algorithm to march a given grid environment.The aim of scheduling is to achieve maximum possible system throughput and to match the application needs with the available computing resources.This paper is motivated by the need to explore the various job scheduling techniques alongside their area of implementation.The paper will systematically analyze the strengths and weaknesses of some selected approaches in the area of grid jobs scheduling.This helps researchers better understand the concept of scheduling, and can contribute in developing more efficient and practical scheduling algorithms.This will also benefit interested researchers to carry out further work in this dynamic research area

Decentralised Workload Scheduler for Resource Allocation in Computational Clusters

This paper presents a detailed design of a decentralised agent-based scheduler, which can be used to manage workloads within the computing cells of a Cloud system. Our proposed solution is based on the concept of service allocation negotiation, whereby all system nodes communicate between themselves, and scheduling logic is decentralised. The presented architecture has been implemented, with multiple simulations run using real-world workload traces from the Google Cluster Data project. The results were then compared to the scheduling patterns of Google’s Borg system

An Improved Min-Min Task Scheduling Algorithm for Load Balancing in Cloud Computing

Cloud computing is a model that uses computing resources (CPU, memory, processor etc.) which are delivered as a service over a network internet. It is a new field for large scale distributed computing. In cloud computing, one of the major task is to execute large number of tasks with given available resources to achieve high performance, minimal total time for completion, Minimum response time, effective utilization of resources etc. Keeping all these perspective, we need to design, develop and propose a scheduling algorithm to develop an allocation map to represent the set of tasks on appropriate resources. Though traditional Min-Min algorithm is a simple, yet efficient algorithm that provides a better scheduling to minimize the total time for completion of tasks compared to the other algorithms. However the drawback of this algorithm is the load imbalanced, which is one of the major issue for cloud provider. In this paper, we introduce an improved load balancing algorithm keeping Min-Min algorithm as base in order to reduce the makespan and increase the resource utilization (ILBMM). The proposed method has two different phases, in the first phase traditional min-min algorithm is executed and in second phase it selects the task with maximum completion time and assigns it to appropriate resource to produce better makespan and utilize resource effectivel

Comparative Study of Neural Networks Algorithms for Cloud Computing CPU Scheduling

Cloud Computing is the most powerful computing model of our time. While the major IT providers and consumers are competing to exploit the benefits of this computing model in order to thrive their profits, most of the cloud computing platforms are still built on operating systems that uses basic CPU (Core Processing Unit) scheduling algorithms that lacks the intelligence needed for such innovative computing model. Correspdondingly, this paper presents the benefits of applying Artificial Neural Networks algorithms in regards to enhancing CPU scheduling for Cloud Computing model. Furthermore, a set of characteristics and theoretical metrics are proposed for the sake of comparing the different Artificial Neural Networks algorithms and finding the most accurate algorithm for Cloud Computing CPU Scheduling

Stochastic Greedy-Based Particle Swarm Optimization for Workflow Application in Grid

The workflow application is a common grid application. The objective of a workflow application is to complete all the tasks within the shortest time, i.e., minimal makespan. A job scheduler with a high-efficient scheduling algorithm is required to solve workflow scheduling based on grid information. Scheduling problems are NP-complete problems, which have been well solved by metaheuristic algorithms. To attain effective solutions to workflow application, an algorithm named the stochastic greedy PSO (SGPSO) is proposed to solve workflow scheduling; a new velocity update rule based on stochastic greedy is suggested. Restated, a stochastic greedy-driven search guidance is provided to particles. Meanwhile, a stochastic greedy probability (SGP) parameter is designed to help control whether the search behavior of particles is exploitation or exploration to improve search efficiency. The advantages of the proposed scheme are retaining exploration capability during a search, reducing complexity and computation time, and easy to implement. Retaining exploration capability during a search prevents particles from getting trapped on local optimums. Additionally, the diversity of the proposed SGPSO is verified and analyzed. The experimental results demonstrate that the SGPSO proposed can effectively solve workflow class problems encountered in the grid environment

A hybrid approach for scheduling applications in cloud computing environment

Cloud computing plays an important role in our daily life. It has direct and positive impact on share and update data, knowledge, storage and scientific resources between various regions. Cloud computing performance heavily based on job scheduling algorithms that are utilized for queue waiting in modern scientific applications. The researchers are considered cloud computing a popular platform for new enforcements. These scheduling algorithms help in design efficient queue lists in cloud as well as they play vital role in reducing waiting for processing time in cloud computing. A novel job scheduling is proposed in this paper to enhance performance of cloud computing and reduce delay time in queue waiting for jobs. The proposed algorithm tries to avoid some significant challenges that throttle from developing applications of cloud computing. However, a smart scheduling technique is proposed in our paper to improve performance processing in cloud applications. Our experimental result of the proposed job scheduling algorithm shows that the proposed schemes possess outstanding enhancing rates with a reduction in waiting time for jobs in queue list

Scheduling, Characterization and Prediction of HPC Workloads for Distributed Computing Environments

As High Performance Computing (HPC) has grown considerably and is expected to grow even more, effective resource management for distributed computing sys- tems is motivated more than ever. As the computational workloads grow in quantity, it is becoming more crucial to apply efficient resource management and workload scheduling to use resources efficiently while keeping the computational performance reasonably good. The problem of efficiently scheduling workloads on resources while meeting performance standards is hard. Additionally, non-clairvoyance of job dimen- sions makes resource management even harder in real-world scenarios. Our research methodology investigates the scheduling problem compliant for HPC and researches the challenges for deploying the scheduling in real world-scenarios using state of the art machine learning and data science techniques.To this end, this Ph.D. dissertation makes the following core contributions: a) We perform a theoretical analysis of space-sharing, non-preemptive scheduling: we studied this scheduling problem and proposed scheduling algorithms with polyno- mial computation time. We also proved constant upper-bounds for the performance of these algorithms. b) We studied the sensitivity of scheduling algorithms to the accuracy of runtime and devised a meta-learning approach to estimate prediction accuracy for newly submitted jobs to the HPC system. c) We studied the runtime prediction problem for HPC applications. For this purpose, we studied the distri- bution of available public workloads and proposed two different solutions that can predict multi-modal distributions: switching state-space models and Mixture Density Networks. d) We studied the effectiveness of recent recurrent neural network models for CPU usage trace prediction for individual VM traces as well as aggregate CPU usage traces. In this dissertation, we explore solutions to improve the performance of scheduling workloads on distributed systems.We begin by looking at the problem from the theoretical perspective. Modeling the problem mathematically, we first propose a scheduling algorithm that finds a constant approximation of the optimal solution for the problem in polynomial time. We prove that the performance of the algorithm (average completion time is the constant approximation of the performance of the optimal scheduling. We next look at the problem in real-world scenarios. Considering High-Performance Computing (HPC) workload computing environments as the most similar real-world equivalent of our mathematical model, we explore the problem of predicting application runtime. We propose an algorithm to handle the existing uncertainties in the real world and show-case our algorithm with demonstrative effectiveness in terms of response time and resource utilization. After looking at the uncertainty problem, we focus on trying to improve the accuracy of existing prediction approaches for HPC application runtime. We propose two solutions, one based on Kalman filters and one based on deep density mixture networks. We showcase the effectiveness of our prediction approaches by comparing with previous prediction approaches in terms of prediction accuracy and impact on improving scheduling performance. In the end, we focus on predicting resource usage for individual applications during their execution. We explore the application of recurrent neural networks for predicting resource usage of applications deployed on individual virtual machines. To validate our proposed models and solutions, we performed extensive trace-driven simulation and measured the effectiveness of our approaches

An Energy Aware and Secure MAC Protocol for Tackling Denial of Sleep Attacks in Wireless Sensor Networks

Wireless sensor networks which form part of the core for the Internet of Things consist of resource constrained sensors that are usually powered by batteries. Therefore, careful energy awareness is essential when working with these devices. Indeed,the introduction of security techniques such as authentication and encryption, to ensure confidentiality and integrity of data, can place higher energy load on the sensors. However, the absence of security protection c ould give room for energy drain attacks such as denial of sleep attacks which have a higher negative impact on the life span ( of the sensors than the presence of security features. This thesis, therefore, focuses on tackling denial of sleep attacks from two perspectives A security perspective and an energy efficiency perspective. The security perspective involves evaluating and ranking a number of security based techniques to curbing denial of sleep attacks. The energy efficiency perspective, on the other hand, involves exploring duty cycling and simulating three Media Access Control ( protocols Sensor MAC, Timeout MAC andTunableMAC under different network sizes and measuring different parameters such as the Received Signal Strength RSSI) and Link Quality Indicator ( Transmit power, throughput and energy efficiency Duty cycling happens to be one of the major techniques for conserving energy in wireless sensor networks and this research aims to answer questions with regards to the effect of duty cycles on the energy efficiency as well as the throughput of three duty cycle protocols Sensor MAC ( Timeout MAC ( and TunableMAC in addition to creating a novel MAC protocol that is also more resilient to denial of sleep a ttacks than existing protocols. The main contributions to knowledge from this thesis are the developed framework used for evaluation of existing denial of sleep attack solutions and the algorithms which fuel the other contribution to knowledge a newly developed protocol tested on the Castalia Simulator on the OMNET++ platform. The new protocol has been compared with existing protocols and has been found to have significant improvement in energy efficiency and also better resilience to denial of sleep at tacks Part of this research has been published Two conference publications in IEEE Explore and one workshop paper