Hydroinformatics and diversity in hydrological ensemble prediction systems

Nous abordons la prévision probabiliste des débits à partir de deux perspectives basées sur la complémentarité de multiples modèles hydrologiques (diversité). La première exploite une méthodologie hybride basée sur l’évaluation de plusieurs modèles hydrologiques globaux et d’outils d’apprentissage automatique pour la sélection optimale des prédicteurs, alors que la seconde fait recourt à la construction d’ensembles de réseaux de neurones en forçant la diversité. Cette thèse repose sur le concept de la diversité pour développer des méthodologies différentes autour de deux problèmes pouvant être considérés comme complémentaires. La première approche a pour objet la simplification d’un système complexe de prévisions hydrologiques d’ensemble (dont l’acronyme anglais est HEPS) qui dispose de 800 scénarios quotidiens, correspondant à la combinaison d’un modèle de 50 prédictions météorologiques probabilistes et de 16 modèles hydrologiques globaux. Pour la simplification, nous avons exploré quatre techniques: la Linear Correlation Elimination, la Mutual Information, la Backward Greedy Selection et le Nondominated Sorting Genetic Algorithm II (NSGA-II). Nous avons plus particulièrement développé la notion de participation optimale des modèles hydrologiques qui nous renseigne sur le nombre de membres météorologiques représentatifs à utiliser pour chacun des modèles hydrologiques. La seconde approche consiste principalement en la sélection stratifiée des données qui sont à la base de l’élaboration d’un ensemble de réseaux de neurones qui agissent comme autant de prédicteurs. Ainsi, chacun d’entre eux est entraîné avec des entrées tirées de l’application d’une sélection de variables pour différents échantillons stratifiés. Pour cela, nous utilisons la base de données du deuxième et troisième ateliers du projet international MOdel Parameter Estimation eXperiment (MOPEX). En résumé, nous démontrons par ces deux approches que la diversité implicite est efficace dans la configuration d’un HEPS de haute performance.In this thesis, we tackle the problem of streamflow probabilistic forecasting from two different perspectives based on multiple hydrological models collaboration (diversity). The first one favours a hybrid approach for the evaluation of multiple global hydrological models and tools of machine learning for predictors selection, while the second one constructs Artificial Neural Network (ANN) ensembles, forcing diversity within. This thesis is based on the concept of diversity for developing different methodologies around two complementary problems. The first one focused on simplifying, via members selection, a complex Hydrological Ensemble Prediction System (HEPS) that has 800 daily forecast scenarios originating from the combination of 50 meteorological precipitation members and 16 global hydrological models. We explore in depth four techniques: Linear Correlation Elimination, Mutual Information, Backward Greedy Selection, and Nondominated Sorting Genetic Algorithm II (NSGA-II). We propose the optimal hydrological model participation concept that identifies the number of meteorological representative members to propagate into each hydrological model in the simplified HEPS scheme. The second problem consists in the stratified selection of data patterns that are used for training an ANN ensemble or stack. For instance, taken from the database of the second and third MOdel Parameter Estimation eXperiment (MOPEX) workshops, we promoted an ANN prediction stack in which each predictor is trained on input spaces defined by the Input Variable Selection application on different stratified sub-samples. In summary, we demonstrated that implicit diversity in the configuration of a HEPS is efficient in the search for a HEPS of high performance

Real-time Control and Optimization of Water Supply and Distribution infrastructure

Across North America, water supply and distribution systems (WSDs) are controlled manually by operational staff - who place a heavy reliance on their experience and judgement when rendering operational decisions. These decisions range from scheduling the operation of pumps, valves and chemical dosing in the system. However, due to the uncertainty of demand, stringent water quality regulatory constraints, external forcing (cold/drought climates, fires, bursts) from the environment, and the non-stationarity of climate change, operators have the tendency to control their systems conservatively and reactively. WSDs that are operated in such fashion are said to be 'reactive' because: (i) the operators manually react to changes in the system behaviour, as measured by Supervisory Control and Data Acquisition (SCADA) systems; and (ii) are not always aware of any anomalies in the system until they are reported by consumers and authorities. The net result is that the overall operations of WSDs are suboptimal with respect to energy consumption, water losses, infrastructure damage and water quality. In this research, an intelligent platform, namely the Real-time Dynamically Dimensioned Scheduler (RT-DDS), is developed and quantitatively assessed for the proactive control and optimization of WSD operations. The RT-DDS platform was configured to solve a dynamic control problem at every timestep (hour) of the day. The control problem involved the minimization of energy costs (over the 24-hour period) by recommending 'near-optimal' pump schedules, while satisfying hydraulic reliability constraints. These constraints were predefined by operational staff and regulatory limits and define a tolerance band for pressure and storage levels across the WSD system. The RT-DDS platform includes three essential modules. The first module produces high-resolution forecasts of water demand via ensemble machine learning techniques. A water demand profile for the next 24-hours is predicted based on historical demand, ambient conditions (i.e. temperature, precipitation) and current calendar information. The predicted profile is then fed into the second module, which involves a simulation model of the WSD. The model is used to determine the hydraulic impacts of particular control settings. The results of the simulation model are used to guide the search strategy of the final module - a stochastic single solution optimization algorithm. The optimizer is parallelized for computational efficiency, such that the reporting frequency of the platform is within 15 minutes of execution time. The fidelity of the prediction engine of the RT-DDS platform was evaluated with an Advanced Metering Infrastructure (AMI) driven case study, whereby the short-term water consumption of the residential units in the city were predicted. A Multi-Layer Perceptron (MLP) model alongside ensemble-driven learning techniques (Random forests, Bagging trees and Boosted trees) were built, trained and validated as part of this research. A three-stage validation process was adopted to assess the replicative, predictive and structural validity of the models. Further, the models were assessed in their predictive capacity at two different spatial resolutions: at a single meter and at the city-level. While the models proved to have strong generalization capability, via good performance in the cross-validation testing, the models displayed slight biases when aiming to predict extreme peak events in the single meter dataset. It was concluded that the models performed far better with a lower spatial resolution (at the city or district level) whereby peak events are far more normalized. In general, the models demonstrated the capacity of using machine learning techniques in the context of short term water demand forecasting - particularly for real-time control and optimization. In determining the optimal representation of pump schedules for real-time optimization, multiple control variable formulations were assessed. These included binary control statuses and time-controlled triggers, whereby the pump schedule was represented as a sequence of on/off binary variables and active/idle discrete time periods, respectively. While the time controlled trigger representation systematically outperformed the binary representation in terms of computational efficiency, it was found that both formulations led to conditions whereby the system would violate the predefined maximum number of pump switches per calendar day. This occurred because at each timestep the control variable formulation was unaware of the previously elapsed pump switches in the subsequent hours. Violations in the maximum pump switch limits lead to transient instabilities and thus create hydraulically undesirable conditions. As such, a novel feedback architecture was proposed, such that at every timestep, the number of switches that had elapsed in the previous hours was explicitly encoded into the formulation. In this manner, the maximum number of switches per calendar day was never violated since the optimizer was aware of the current trajectory of the system. Using this novel formulation, daily energy cost savings of up to 25\% were achievable on an average day, leading to cost savings of over 2.3 million dollars over a ten-year period. Moreover, stable hydraulic conditions were produced in the system, thereby changing very little when compared to baseline operations in terms of quality of service and overall condition of assets

Demand Curve Modeling for the Utility of the Future

Electricity systems are undergoing significant changes. Demands are shifting in magnitude and temporal distribution due to developing policies and technologies such as electric vehicles, heat pumps, embedded generation and energy storage, while an increasingly renewable supply is intermittent and less flexible. As such, there is currently great uncertainty in the industry and future business pathways may vary significantly from the current paradigm. This research focused on developing a set of models which can be used by utility companies to leverage their smart meter data and gain insights into possible future impacts and opportunities. The thesis presents a series of novel models, developed and implemented with data provided from a utility in Southern Ontario. First, a regression model was developed to leverage the full value of utility smart meter data by disaggregating residential and commercial sector demands into base, heating and cooling end uses. The use of a variable temperature changepoint only marginally improved prediction accuracy, but significantly shifted disaggregation results, particularly at hourly resolution. This model was also applied for weather normalization, assessment of technology change and projection under different climate scenarios. A second model used this and additional data from literature to project long term utility level average and peak seasonal load curves. A dynamic interface with parameterized controls allowed real-time visualization of technology and policy impacts on the demand curve. A set of eight literature-based scenarios were also projected to demonstrate the extreme range of impacts predicted by different literature. These led to the conclusion that unmanaged technology penetration can lead to significant challenges such as increased peaks, large ramp rates and lower utilization. An analysis was then performed at finer geographic resolution, investigating impacts on representative distribution system transformers. First, the current variation in local technology penetration was examined, showing a significantly skewed distribution with many transformers having up to ten times the average rates. Clustering was then used to identify a set of eight diverse, representative transformer load profiles. Future scenarios were modeled, demonstrating that the impacts of technology and optimal mitigation techniques vary significantly between regions of the distribution system. Finally, the dynamic utility load curve model was also updated to project demands for the representative transformer groups identified. This allows users to simultaneously assess local impacts and mitigation strategies, as well as aggregate effects on the overall system demands. Together these works combine to provide a valuable toolset and significant insight into potential system impacts

Pervasive Data Analytics for Sustainable Energy Systems

With an ever growing population, global energy demand is predicted to keep increasing. Furthermore, the integration of renewable energy sources into the electricity grid (to reduce carbon emission and humanity's dependency on fossil fuels), complicates efforts to balance supply and demand, since their generation is intermittent and unpredictable. Traditionally, it has always been the supply side that has adapted to follow energy demand, however, in order to have a sustainable energy system for the future, the demand side will have to be better managed to match the available energy supply. In the first part of this thesis, we focus on understanding customers' energy consumption behavior (demand analytics). While previously, information about customer's energy consumption could be obtained only with coarse granularity (e.g., monthly or bimonthly), nowadays, using advanced metering infrastructure (or smart meters), utility companies are able to retrieve it in near real-time. By leveraging smart meter data, we then develop a versatile customer segmentation framework, track cluster changes over time, and identify key characteristics that define a cluster. Additionally, although household-level consumption is hard to predict, it can be used to improve aggregate-level forecasting by first segmenting the households into several clusters, forecasting the energy consumption of each cluster, and then aggregating those forecasts. The improvements provided by this strategy depend not only on the number of clusters, but also on the size of the customer base. Furthermore, we develop an approach to model the uncertainty of future demand. In contrast to previous work that used computationally expensive methods, such as simulation, bootstrapping, or ensemble, we construct prediction intervals directly using the time-varying conditional mean and variance of future demand. While analytics on customer energy data are indeed essential to understanding customer behavior, they could also lead to breaches of privacy, with all the attendant risks. The first part of this thesis closes by exploring symbolic representations of smart meter data which still allow learning algorithms to be performed on top of them, thus providing a trade-off between accurate analytics and the protection of customer privacy. In the second part of this thesis, we focus on mechanisms for incentivizing changes in customers' energy usage in order to maintain (electricity) grid stability, i.e., Demand Response (DR). We complement previous work in this area (which typically targeted large, industrial customers) by studying the application of DR to residential customers. We first study the influence of DR baselines, i.e., estimates of what customers would have consumed in the absence of a DR event. While the literature to date has focused on baseline accuracy and bias, we go beyond these concepts by explaining how a baseline affects customer participation in a DR event, and how it affects both the customer and company profit. We then discuss a strategy for matching the demand side with the supply side by using a multiunit auction performed by intelligent agents on behalf of customers. The thesis closes by eliciting behavioral incentives from the crowd of customers for promoting and maintaining customer engagement in DR programs

A Multiscale Analysis of the Factors Controlling Nutrient Dynamics and Cyanobacteria Blooms in Lake Champlain

Cyanobacteria blooms have increased in Lake Champlain due to excessive nutrient loading, resulting in negative impacts on the local economy and environmental health. While climate warming is expected to promote increasingly severe cyanobacteria blooms globally, predicting the impacts of complex climate changes on individual lakes is complicated by the many physical, chemical, and biological processes which mediate nutrient dynamics and cyanobacteria growth across time and space. Furthermore, processes influencing bloom development operate on a variety of temporal scales (hourly, daily, seasonal, decadal, episodic), making it difficult to identify important factors controlling bloom development using traditional methods or coarse temporal resolution datasets. To resolve these inherent problems of scale, I use 4 years of high-frequency biological, hydrodynamic, and biogeochemical data from Missisquoi Bay, Lake Champlain; 23 years of lake-wide monitoring data; and integrated process-based climate-watershed-lake models driven by regional climate projections to answer the following research questions: 1) To what extent do external nutrient inputs or internal nutrient processing control nutrient concentrations and cyanobacteria blooms in Lake Champlain; 2) how do internal and external nutrient inputs interact with meteorological drivers to promote or suppress bloom development; and 3) how is climate change likely to impact these drivers and the risk of cyanobacteria blooms in the future? I find that cyanobacteria blooms are driven by specific combinations of meteorological and biogeochemical conditions in different areas of the lake, and that in the absence of strong management actions cyanobacteria blooms are likely to become more severe in the future due to climate change

Development of Predictive Analytics for Demand Forecasting and Inventory Management in Supply Chain using Machine Learning Techniques

Forecasting demand effectively and managing inventories efficiently are critical components of modern supply chain management. By understanding full scope of demand possibilities, businesses gain ability to fine-tune inventory levels, navigate situations involving stockouts and overstock, and move toward a more resilient and precise supply chain. This thesis focuses on strategies to enhance these critical functions. We start with examining impact of customer segmentation on forecasting precision by introducing a novel cluster-based demand forecasting framework that harnesses ensemble learning techniques. Our results showcase the effectiveness of the clustered-ensembled approach with minimal forecast errors. However, the constraints related to data availability and segmentation indicate areas that warrant further investigation in future research. The significance of demand accuracy becomes most apparent when we consider its impact on safety stock. In second objective, we explore multivariate time series forecasting for optimal safety stock and inventory management, utilizing deep learning models and a cost optimization framework. This strategy outperforms individual models, demonstrating enhanced forecasting accuracy and stability across diverse product domains. Calculating safety stock based on proposed demand prediction framework leads to optimized safety stock levels. This not only prevents costly stockouts but also minimizes surplus inventory, resulting in reduced overall holding costs and improved inventory efficiency. Although the first two objectives provided optimized results, relying on point predictions to calculate safety stock is not ideal. Unlike traditional point forecasting, distribution forecasting aims to cover the entire range of potential demand outcomes, essentially creating a comprehensive map of possibilities. The third objective of this thesis introduces recurrent mixture density networks (RMDNs) for refined distribution demand forecasting and safety stock estimation. These innovative models consistently outperform traditional LSTM models, offering more precise stockout and overstock predictions. This approach not only reduces inventory costs but also enhances supply chain efficiency. In summary, this thesis provides valuable insights and methodologies for businesses aiming to enhance demand forecasting accuracy and optimize inventory management practices in the retail industry. By leveraging customer segmentation, ensemble deep learning, and distribution forecasting techniques, organizations can enhance decision-making processes, reduce operational costs, and thrive in the dynamic landscape of supply chain operations

Lessons Learned and Next Steps in Energy Efficiency Measurement and Attribution: Energy Savings, Net to Gross, Non-Energy Benefits, and Persistence of Energy Efficiency Behavior

This white paper examines four topics addressing evaluation, measurement, and attribution of direct and indirect effects to energy efficiency and behavioral programs: Estimates of program savings (gross); Net savings derivation through free ridership / net to gross analyses; Indirect non-energy benefits / impacts (e.g., comfort, convenience, emissions, jobs); and, Persistence of savings

A Temporal Approach to Characterizing Electrical Peak Demand: Assessment of GHG Emissions at the Supply Side and Identification of Dominant Household Factors at the Demand Side

When electricity demand is at its highest it is most costly to generate and transmit and is usually considered to produce the greatest greenhouse gas (GHG) emissions, so reducing peaks can have a double benefit. In most nations, these peaks in demand occur daily in the mornings and evenings. This PhD research developed a new way of assessing GHG emissions produced by the generation of electricity at different times of the day, including at peak time. It also investigated the dominant factors driving peaks in household demand and developed a new analytical approach to identify these factors so as to assist in developing targeted demand management. Two contrasting countries - New Zealand and Bangladesh - were chosen to apply the research. These two countries have very different climatic conditions, economic conditions, socio-demographic characteristics, electricity generation sector, and emissions from electricity generation, so the research findings could be tested and compared. To assess GHG emissions, an analytical approach was developed - ‘time-varying carbon intensity analysis (TVCIA)’ - to explore the relationships between GHG emissions and peaks in demand. Applied to 2015 data from New Zealand, a country with around 80% renewable generation dominated by hydro, it was found that New Zealand’s carbon intensity was largely uncorrelated with demand. This finding was counter to some perceptions in the electricity sector in New Zealand where it is assumed that peak demand always means higher GHG emissions. In contrast, when the method was applied to Bangladesh, which has an electricity system dominated by fossil fuel generation, it showed that daily peaks in demand had the highest GHG emissions. Therefore, reduction in demand at peak times could be a potential option to reduce GHG emissions in Bangladesh. In New Zealand, seasonal demand management could be beneficial as GHG emissions can increase significantly in a dry year when hydro lakes are low. For the latter part of the research, a methodology called ‘time-segmented regression analysis (TSRA)’ was developed to identify the dominant factors driving peak electricity demand in households. Applied to a New Zealand dataset, the analysis revealed that methods of water and space heating were the dominant factors in determining peak demand in New Zealand’s households. In contrast, the number of occupants and the number of electrical appliances were dominant factors in determining peak demand in Bangladeshi households. Together the new approaches that have been developed can assist nations in determining the GHG emissions from electricity generation at different times (over days, weeks, months or years) and also determining what factors in households are driving peaks in demand. This is important to help design more effective, targeted energy efficiency and demand management strategies. Together these methods can help in devising programmes for reducing GHG emissions from electricity use

Wind Power Forecasting : State-Of-The-Art 2009.

Many countries and regions are introducing policies aimed at reducing the environmental footprint from the energy sector and increasing the use of renewable energy. In the United States, a number of initiatives have been taken at the state level, from renewable portfolio standards (RPSs) and renewable energy certificates (RECs), to regional greenhouse gas emission control schemes. Within the U.S. Federal government, new energy and environmental policies and goals are also being crafted, and these are likely to increase the use of renewable energy substantially. The European Union is pursuing implementation of its ambitious 20/20/20 targets, which aim (by 2020) to reduce greenhouse gas emissions by 20% (as compared to 1990), increase the amount of renewable energy to 20% of the energy supply, and reduce the overall energy consumption by 20% through energy efficiency. With the current focus on energy and the environment, efficient integration of renewable energy into the electric power system is becoming increasingly important. In a recent report, the U.S. Department of Energy (DOE) describes a model-based scenario, in which wind energy provides 20% of the U.S. electricity demand in 2030. The report discusses a set of technical and economic challenges that have to be overcome for this scenario to unfold. In Europe, several countries already have a high penetration of wind power (i.e., in the range of 7 to 20% of electricity consumption in countries such as Germany, Spain, Portugal, and Denmark). The rapid growth in installed wind power capacity is expected to continue in the United States as well as in Europe. A large-scale introduction of wind power causes a number of challenges for electricity market and power system operators who will have to deal with the variability and uncertainty in wind power generation when making their scheduling and dispatch decisions. Wind power forecasting (WPF) is frequently identified as an important tool to address the variability and uncertainty in wind power and to more efficiently operate power systems with large wind power penetrations. Moreover, in a market environment, the wind power contribution to the generation portofolio becomes important in determining the daily and hourly prices, as variations in the estimated wind power will influence the clearing prices for both energy and operating reserves. With the increasing penetration of wind power, WPF is quickly becoming an important topic for the electric power industry. System operators (SOs), generating companies (GENCOs), and regulators all support efforts to develop better, more reliable and accurate forecasting models. Wind farm owners and operators also benefit from better wind power prediction to support competitive participation in electricity markets against more stable and dispatchable energy sources. In general, WPF can be used for a number of purposes, such as: generation and transmission maintenance planning, determination of operating reserve requirements, unit commitment, economic dispatch, energy storage optimization (e.g., pumped hydro storage), and energy trading. The objective of this report is to review and analyze state-of-the-art WPF models and their application to power systems operations. We first give a detailed description of the methodologies underlying state-of-the-art WPF models. We then look at how WPF can be integrated into power system operations, with specific focus on the unit commitment problem