Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

A Personalized Rolling Optimal Charging Schedule for Plug-In Hybrid Electric Vehicle Based on Statistical Energy Demand Analysis and Heuristic Algorithm

To alleviate the emission of greenhouse gas and the dependence on fossil fuel, Plug-in Hybrid Electrical Vehicles (PHEVs) have gained an increasing popularity in current decades. Due to the fluctuating electricity prices in the power market, a charging schedule is very influential to driving cost. Although the next-day electricity prices can be obtained in a day-ahead power market, a driving plan is not easily made in advance. Although PHEV owners can input a next-day plan into a charging system, e.g., aggregators, day-ahead, it is a very trivial task to do everyday. Moreover, the driving plan may not be very accurate. To address this problem, in this paper, we analyze energy demands according to a PHEV owner’s historical driving records and build a personalized statistic driving model. Based on the model and the electricity spot prices, a rolling optimization strategy is proposed to help make a charging decision in the current time slot. On one hand, by employing a heuristic algorithm, the schedule is made according to the situations in the following time slots. On the other hand, however, after the current time slot, the schedule will be remade according to the next tens of time slots. Hence, the schedule is made by a dynamic rolling optimization, but it only decides the charging decision in the current time slot. In this way, the fluctuation of electricity prices and driving routine are both involved in the scheduling. Moreover, it is not necessary for PHEV owners to input a day-ahead driving plan. By the optimization simulation, the results demonstrate that the proposed method is feasible to help owners save charging costs and also meet requirements for driving

Application-specific modified particle swarm optimization for energy resource scheduling considering vehicle-to-grid

This paper presents a modified Particle Swarm Optimization (PSO) methodology to solve the problem of energy resources management with high penetration of distributed generation and Electric Vehicles (EVs) with gridable capability (V2G). The objective of the day-ahead scheduling problem in this work is to minimize operation costs, namely energy costs, regarding he management of these resources in the smart grid context. The modifications applied to the PSO aimed to improve its adequacy to solve the mentioned problem. The proposed Application Specific Modified Particle Swarm Optimization (ASMPSO) includes an intelligent mechanism to adjust velocity limits during the search process, as well as self-parameterization of PSO parameters making it more user-independent. It presents better robustness and convergence characteristics compared with the tested PSO variants as well as better constraint handling. This enables its use for addressing real world large-scale problems in much shorter times than the deterministic methods, providing system operators with adequate decision support and achieving efficient resource scheduling, even when a significant number of alternative scenarios should be considered. The paper includes two realistic case studies with different penetration of gridable vehicles (1000 and 2000). The proposed methodology is about 2600 times faster than Mixed-Integer Non-Linear Programming (MINLP) reference technique, reducing the time required from 25 h to 36 s for the scenario with 2000 vehicles, with about one percent of difference in the objective function cost value

Systematic categorization of optimization strategies for virtual power plants

Due to the rapid growth in power consumption of domestic and industrial appliances, distributed energy generation units face difficulties in supplying power efficiently. The integration of distributed energy resources (DERs) and energy storage systems (ESSs) provides a solution to these problems using appropriate management schemes to achieve optimal operation. Furthermore, to lessen the uncertainties of distributed energy management systems, a decentralized energy management system named virtual power plant (VPP) plays a significant role. This paper presents a comprehensive review of 65 existing different VPP optimization models, techniques, and algorithms based on their system configuration, parameters, and control schemes. Moreover, the paper categorizes the discussed optimization techniques into seven different types, namely conventional technique, offering model, intelligent technique, price-based unit commitment (PBUC) model, optimal bidding, stochastic technique, and linear programming, to underline the commercial and technical efficacy of VPP at day-ahead scheduling at the electricity market. The uncertainties of market prices, load demand, and power distribution in the VPP system are mentioned and analyzed to maximize the system profits with minimum cost. The outcome of the systematic categorization is believed to be a base for future endeavors in the field of VPP development

A Review of Energy Management of Renewable Multisources in Industrial Microgrids

This review aims to consolidate recent advancements in power control within microgrids and multi-microgrids. It specifically focuses on analyzing the comparative benefits of various architectures concerning energy sharing and demand cost management. The paper provides a comprehensive technical analysis of different architectures found in existing literature, which are designed for energy management and demand cost optimization. In summary, this review paper provides a thorough examination of power control in microgrids and multi-microgrids and compares different architectural approaches for energy management and demand cost optimization