Toward scalable stochastic unit commitment. Part 1: load scenario generation

Unit commitment decisions made in the day-ahead market and during subsequent reliability assessments are critically based on forecasts of load. Tra- ditional, deterministic unit commitment is based on point or expectation-based load forecasts. In contrast, stochastic unit commitment relies on multiple load sce- narios, with associated probabilities, that in aggregate capture the range of likely load time-series. The shift from point-based to scenario-based forecasting necessi- tates a shift in forecasting technologies, to provide accurate inputs to stochastic unit commitment. In this paper, we discuss a novel scenario generation method- ology for load forecasting in stochastic unit commitment, with application to real data associated with the Independent System Operator for New England (ISO- NE). The accuracy of the expected scenario generated using our methodology is consistent with that of point forecasting methods. The resulting sets of realistic scenarios serve as input to rigorously test the scalability of stochastic unit com- mitment solvers, as described in the companion paper. The scenarios generated by our method are available as an online supplement to this paper, as part of a novel, publicly available large-scale stochastic unit commitment benchmark

Scenario generation quality assessment for two-stage stochastic programs

In minimization problems with uncertain parameters, cost savings can be achieved by solving stochastic programming (SP) formulations instead of using expected parameter values in a deterministic formulation. To obtain such savings, it is crucial to employ high quality probabilistic scenarios for the uncertain parameters. A convincing way to assess the quality of a scenario generation method is to simulate employing the resulting scenarios when solving the SP problem while measuring the costs incurred when the solution is implemented and observed parameter values occur. Simulation studies to assess the quality in this way are computationally very demanding. This research is aimed at developing faster methods to assess the quality via statistical metrics. Relibility, which is defined as the statistical consistency between scenarios and observation, is a prerequisite for quality. The dissertation is presented in a three-paper format. The stochastic unit commitment problem in electric power system operation is an application of SP that motivated this study. In power systems with high penetration of wind generation, probabilistic scenarios for the available wind energy are generated for use in stochastic formulations of day-ahead thermal unit commitment problems. To minimize the expected cost of dispatching the committed units, the wind energy scenarios should accurately represent the stochastic process for available wind energy. In the first paper, aiming to assess the reliability of probabilistic scenarios for wind energy time series, we employ some existing forecast verification approaches and introduce a mass transportation distance rank histogram to assess the reliability of unequally likely scenarios. In the second paper, we examine the relationship between the statistical reliability assessment metrics and the cost results of solving SUC using the assessed scenario generation method. Based on this relationship, we understand the importance of scenario reliability to ensure scenario quality for the SUC problem. In the third paper, we extend this work to make it more robust and general enough to be applied to any two-stage SP problem that is repeatedly solved and for which observational data exist for some historical period. Focusing on scenario quality, we develop two novel approaches: expected value based and perfect information based scenario generation assessment. With the proposed approaches, we can assess the quality of scenario sets without having to repeatedly solve the related SP problem. Instead of comparing scenarios to observations directly, these approaches take into consideration the impact of each scenario on the solution to the SP problem

Reliability of Wind Power Scenarios and Stochastic Unit Commitment Cost

Probabilistic wind power scenarios constitute a crucial input for stochastic day-ahead unit commitment in power systems with deep penetration of wind generation. To minimize the expected cost, the scenario time series of wind power amounts available should accurately represent the stochastic process for available wind power as it is estimated on the day ahead. The high computational demands of stochastic programming motivate a search for ways to evaluate scenarios without extensively simulating the stochastic unit commitment procedure. Reliability of wind power scenario sets can be assessed by statistical verification approaches. In this study, we examine the relationship between the statistical evaluation metrics and the results of stochastic unit commitment. Lack of uniformity in a mass transportation distance rank histogram can eliminate scenario sets that might lead to either excessive no-load costs of committed units or high penalty costs for violating energy balance. Event-based metrics can help to predict the cost performance of the remaining scenario sets

Statistical reliability of wind power scenarios and stochastic unit commitment cost

Probabilistic wind power scenarios constitute a crucial input for stochastic day-ahead unit commitment in power systems with deep penetration of wind generation. To minimize the cost of implemented solutions, the scenario time series of wind power amounts available should accurately represent the stochastic process for available wind power as it is estimated on the day ahead. The high computational demands of stochastic programming motivate a search for ways to evaluate scenarios without extensively simulating the stochastic unit commitment procedure. The statistical reliability of wind power scenario sets can be assessed by approaches extended from ensemble forecast verification. We examine the relationship between the statistical reliability metrics and the results of stochastic unit commitment when implemented solutions encounter the observed available wind power. Lack of uniformity in a mass transportation distance rank histogram can eliminate scenario sets that might lead to either excessive no-load costs of committed units or high penalty costs for violating energy balance when the committed units are dispatched. Event-based metrics can help to predict results of implementing solutions found with the remaining scenario sets

Statistical metrics for assessing the quality of wind power scenarios for stochastic unit commitment

In power systems with high penetration of wind generation, probabilistic scenarios are generated for use in stochastic formulations of day-ahead unit commitment problems. To minimize the expected cost, the wind power scenarios should accurately represent the stochastic process for available wind power. We employ some statistical evaluation metrics to assess whether the scenario set possesses desirable properties that are expected to lead to a lower cost in stochastic unit commitment. A new mass transportation distance rank histogram is developed for assessing the reliability of unequally likely scenarios. Energy scores, rank histograms and Brier scores are applied to alternative sets of scenarios that are generated by two very different methods. The mass transportation distance rank histogram is best able to distinguish between sets of scenarios that are more or less calibrated according to their bias, variability and autocorrelation

Short-term operation of the power system and the natural gas system considering uncertainties

Electricity generation increasingly relies on natural gas for fuel. The competing demands for gas by natural gas-fueled generators and other users, the differences in the timing of short-term operations and markets between the natural gas system and the power system, and the deepening penetration of variable renewable energy in the power system cause difficulties in operating the two systems economically and reliably. This dissertation consists of three papers which present different models and methods for the short-term operation of the natural gas and power systems considering different sources of uncertainty. From the viewpoint of a centralized system operator, who can operate the power system and natural gas system simultaneously, we first compare two approaches to addressing the uncertainty in the joint scheduling of a combined natural gas and power system. A stochastic programming model and a deterministic model with reserves are formulated to minimize the daily operational cost and investigate the hourly unit commitment and economic dispatch decisions in the power system as well as the hourly working schedule of the natural gas system while satisfying all the operational constraints. In the deterministic model, the reserves proportional to the wind energy forecast are used to mitigate the effect of the uncertainty in wind energy, whereas in the stochastic programming model the day-ahead decisions are made while explicitly considering the wind energy uncertainty. To tackle the nonlinear constraints on the gas flows in pipelines, we approximately linearize those nonlinear constraints by adding multiple binary variables and constraints. Through numerical experimentation, the number of piecewise linear segments is chosen to balance accuracy and computational efficiency. The simulation results of two case studies indicate that, when the total wind capacity exceeds 15\% of the conventional generation capacity, the stochastic programming model produces schedules with comparable or lower cost and energy shortages than the deterministic model with reserves. The centralized system operator modeled in the first paper does not exist in the real U.S. energy market. From a more realistic viewpoint of the power system operator, in the second paper, we quantify the effect of the uncertainty in the gas spot price on power system dispatch cost in the absence of wind energy. The influence of the natural gas system is considered in terms of fixed or uncertain parameters in the power system daily economic dispatch problem. A benchmark distribution of the dispatch cost is generated by Monte Carlo simulation conducted with the gas price fixed at its expectation while sampling from the marginal distribution for the load. For comparison, another dispatch cost distribution is generated by sampling from a joint distribution for the gas price and the load. The risk from uncertainty in the gas price is quantified by the distance between dispatch cost distributions or, alternatively, by the difference between the values of a risk measure applied to each distribution. We demonstrate that this risk quantification method helps to select from among alternative risk-mitigation strategies, such as providing dual-fuel capability or adding gas storage facilities at the system level. In the third paper, we investigate the use of a reliability unit commitment (RUC) conducted after the day-ahead market unit commitment to manage the natural gas cost in the power system operations, where the operations of the two systems are separately optimized to minimize their own net cost. This separately optimized model incorporates the interruptible contract and the real-time market for gas, where an iterative process between the electricity and gas operations determines the real-time gas flows and prices. An ideal co-optimized model, where a centralized system operator optimizes the two systems simultaneously, is taken as a benchmark for comparison. By numerical studies, we demonstrate the ability of the RUC step to reduce power system cost, maintain a low real-time gas price, and avoid real-time gas supply deficiency

Forecasting: theory and practice

Forecasting has always been in the forefront of decision making and planning. The uncertainty that surrounds the future is both exciting and challenging, with individuals and organisations seeking to minimise risks and maximise utilities. The lack of a free-lunch theorem implies the need for a diverse set of forecasting methods to tackle an array of applications. This unique article provides a non-systematic review of the theory and the practice of forecasting. We offer a wide range of theoretical, state-of-the-art models, methods, principles, and approaches to prepare, produce, organise, and evaluate forecasts. We then demonstrate how such theoretical concepts are applied in a variety of real-life contexts, including operations, economics, finance, energy, environment, and social good. We do not claim that this review is an exhaustive list of methods and applications. The list was compiled based on the expertise and interests of the authors. However, we wish that our encyclopedic presentation will offer a point of reference for the rich work that has been undertaken over the last decades, with some key insights for the future of the forecasting theory and practice

Scenario generation and reduction for long-term and short-term power system generation planning under uncertainties

This dissertation focuses on computational issues of applying two-stage stochastic programming for long-term and short-term generation planning problems from the perspective of scenario generation and reduction. It follows a three-paper format, in which each paper discusses approaches to generating probabilistic scenarios and then reducing the substantial computational burden caused by a huge number of scenarios for different applications in power systems. The first paper investigates a long-term generation expansion planning model with uncertain annual load and natural gas price. A two-stage stochastic program is formulated to minimize the total expected expansion cost, generation cost and penalties on unserved energy while satisfying aggregated operational constraints. A statistical property matching technique is applied to simulate plausible future realizations of annual load and natural gas price over the whole planning horizon. To mitigate the computational complexity of a widely used classic scenario reduction method in this context, we firstly cluster scenarios according to the wait-and-see solution for each scenario and then apply the fast forward selection (FFS) method. The second paper prepares a basis for load scenario generation for the day-ahead reliability unit commitment problem. For the purpose of creating practical load scenarios, epi-splines, based on approximation theory, are employed to approximate the relationship between load and weather forecasts. The epi-spline based short-term load model starts by classifying similar days according to daily forecast temperature as well as monthly and daily load patterns. Parameters of the epi-spline based short-term load model are then estimated by minimizing the fitted errors. The method is tested using day-ahead weather forecast and hourly load data obtained from an Independent System Operator in the U.S. By considering the non-weather dependent load pattern in the short-term load model, the model not only provides accurate load predictions and smaller prediction variances in the validated days, but also preserves similar intraday serial correlations among hourly forecast loads to those from actual load. The last paper in this dissertation proposes a solution-sensitivity based heuristic scenario reduction method, called forward selection in recourse clusters (FSRC), for a two-stage stochastic day-ahead reliability unit commitment model. FSRC alleviates the computational burden of solving the stochastic program by selecting scenarios based on their cost and reliability impacts. In addition, the variant of pre-categorizing scenarios improves the computational efficiency of FSRC by simplifying the clustering procedure. In a case study down-sampled from an Independent System Operator in the U.S., FSRC is shown to provide reliable commitment strategies and preserve solution quality even when the reduction is substantial