Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

A simple hybrid algorithm for improving team sport AI

In the very popular genre of team sports games defeating the opposing AI is the main focus of the gameplay experience. However the overall quality of these games is significantly damaged because, in a lot of cases, the opposition is prone to mistakes or vulnerable to exploitation. This paper introduces an AI system which overcomes this failing through the addition of simple adaptive learning and prediction algorithms to a basic ice hockey defence. The paper shows that improvements can be made to the gameplay experience without overly increasing the implementation complexity of the system or negatively affecting its performance. The created defensive system detects patterns in the offensive tactics used against it and changes elements of its reaction accordingly; effectively adapting to attempted exploitation of repeated tactics. This is achieved using a fuzzy inference system that tracks player movement, which greatly improves variation of defender positioning, alongside an N-gram pattern recognition-based algorithm that predicts the next action of the attacking player. Analysis of implementation complexity and execution overhead shows that these techniques are not prohibitively expensive in either respect, and are therefore appropriate for use in games

Bad Data Injection Attack and Defense in Electricity Market using Game Theory Study

Applications of cyber technologies improve the quality of monitoring and decision making in smart grid. These cyber technologies are vulnerable to malicious attacks, and compromising them can have serious technical and economical problems. This paper specifies the effect of compromising each measurement on the price of electricity, so that the attacker is able to change the prices in the desired direction (increasing or decreasing). Attacking and defending all measurements are impossible for the attacker and defender, respectively. This situation is modeled as a zero sum game between the attacker and defender. The game defines the proportion of times that the attacker and defender like to attack and defend different measurements, respectively. From the simulation results based on the PJM 5 Bus test system, we can show the effectiveness and properties of the studied game.Comment: To appear in IEEE Transactions on Smart Grid, Special Issue on Cyber, Physical, and System Security for Smart Gri