Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Modern approaches to modeling user requirements on resource and task allocation in hierarchical computational grids

Peer ReviewedPostprint (published version

Maximising microprocessor reliability through game theory and heuristics

PhD ThesisEmbedded Systems are becoming ever more pervasive in our society, with most routine daily tasks now involving their use in some form and the market predicted to be worth USD 220 billion, a rise of 300%, by 2018. Consumers expect more functionality with each design iteration, but for no detriment in perceived performance. These devices can range from simple low-cost chips to expensive and complex systems and are a major cost driver in the equipment design phase. For more than 35 years, designers have kept pace with Moore's Law, but as device size approaches the atomic limit, layouts are becoming so complicated that current scheduling techniques are also reaching their limit, meaning that more resource must be reserved to manage and deliver reliable operation. With the advent of many-core systems and further sources of unpredictability such as changeable power supplies and energy harvesting, this reservation of capability may become so large that systems will not be operating at their peak efficiency. These complex systems can be controlled through many techniques, with jobs scheduled either online prior to execution beginning or online at each time or event change. Increased processing power and job types means that current online scheduling methods that employ exhaustive search techniques will not be suitable to define schedules for such enigmatic task lists and that new techniques using statistic-based methods must be investigated to preserve Quality of Service. A new paradigm of scheduling through complex heuristics is one way to administer these next levels of processor effectively and allow the use of more simple devices in complex systems; thus reducing unit cost while retaining reliability a key goal identified by the International Technology Roadmap for Semi-conductors for Embedded Systems in Critical Environments. These changes would be beneficial in terms of cost reduction and system exibility within the next generation of device. This thesis investigates the use of heuristics and statistical methods in the operation of real-time systems, with the feasibility of Game Theory and Statistical Process Control for the successful supervision of high-load and critical jobs investigated. Heuristics are identified as an effective method of controlling complex real-time issues, with two-person non-cooperative games delivering Nash-optimal solutions where these exist. The simplified algorithms for creating and solving Game Theory events allow for its use within small embedded RISC devices and an increase in reliability for systems operating at the apex of their limits. Within this Thesis, Heuristic and Game Theoretic algorithms for a variety of real-time scenarios are postulated, investigated, refined and tested against existing schedule types; initially through MATLAB simulation before testing on an ARM Cortex M3 architecture functioning as a simplified automotive Electronic Control Unit.Doctoral Teaching Account from the EPSRC

Review on Radio Resource Allocation Optimization in LTE/LTE-Advanced using Game Theory

Recently, there has been a growing trend toward ap-plying game theory (GT) to various engineering fields in order to solve optimization problems with different competing entities/con-tributors/players. Researches in the fourth generation (4G) wireless network field also exploited this advanced theory to overcome long term evolution (LTE) challenges such as resource allocation, which is one of the most important research topics. In fact, an efficient de-sign of resource allocation schemes is the key to higher performance. However, the standard does not specify the optimization approach to execute the radio resource management and therefore it was left open for studies. This paper presents a survey of the existing game theory based solution for 4G-LTE radio resource allocation problem and its optimization

Game-theoretic, market and meta-heuristics approaches for modelling scheduling and resource allocation in grid systems

Task scheduling and resource allocation are the crucial issues in any large scale distributed system, such as Computational Grids (CGs). However, traditional computational models and resolution methods cannot effectively tackle the complex nature of Grid, where the resources and users belong to many administrative domains with their own access policies, users' privileges, etc. Recently, researchers are investigating the use of game theoretic approaches for modelling task and resource allocation problems in CGs. In this paper, we present a compact survey of the most relevant research proposals in the literature to use game-based models for the resource allocation problems and their resolution using metaheuristic methods. We emphasize the need of the translation of the traditional economical models into the game scenarios and the use of metaheuristic schedulers for solving such games in order to address the new complex scheduling and allocation criterions. We study the case of asymmetric Stackelberg game used for modelling the Grid users' behavior, where the security and reliability criterions are aggregated and defined as the users' costs functions. The obtained results show the efficiency of the hybridization of heuristic-based approaches with game models, which enables to include additional requirements and features into the computational models and tackle more effectively the resolution of the applied schedulers.Peer ReviewedPostprint (published version