Post-processed data and graphical tools for a CONUS-wide eddy flux evapotranspiration dataset

Large sample datasets of in situ evapotranspiration (ET) measurements with well documented data provenance and quality assurance are critical for water management and many fields of earth science research. We present a post-processed ET oriented dataset at daily and monthly timesteps, from 161 stations, including 148 eddy covariance flux towers, that were chosen based on their data quality from nearly 350 stations across the contiguous United States. In addition to ET, the data includes energy and heat fluxes, meteorological measurements, and reference ET downloaded from grid- MET for each flux station. Data processing techniques were conducted in a reproducible manner using open-source soft- ware. Most data initially came from the public AmeriFlux network, however, several different networks (e.g., the USDA- Agricultural Research Service) and university partners pro- vided data that was not yet public. Initial half-hourly energy balance data were gap-filled and aggregated to daily frequency, and turbulent fluxes were corrected for energy balance closure error using the FLUXNET2015/ONEFlux energy balance ratio approach. Metadata, diagnostics of energy balance, and interactive graphs of time series data are included for each station. Although the dataset was developed primarily to benchmark satellite-based remote sensing ET models of the OpenET initiative, there are many other potential uses, such as validation for a range of regional hydrologic and atmospheric models

Post-processed data and graphical tools for a CONUS-wide eddy flux evapotranspiration dataset

Large sample datasets of in situ evapotranspiration (ET) measurements with well documented data provenance and quality assurance are critical for water management and many fields of earth science research. We present a post-processed ET oriented dataset at daily and monthly timesteps, from 161 stations, including 148 eddy covariance flux towers, that were chosen based on their data quality from nearly 350 stations across the contiguous United States. In addition to ET, the data includes energy and heat fluxes, meteorological measurements, and reference ET downloaded from gridMET for each flux station. Data processing techniques were conducted in a reproducible manner using open-source software. Most data initially came from the public AmeriFlux network, however, several different networks (e.g., the USDA-Agricultural Research Service) and university partners provided data that was not yet public. Initial half-hourly energy balance data were gap-filled and aggregated to daily frequency, and turbulent fluxes were corrected for energy balance closure error using the FLUXNET2015/ONEFlux energy balance ratio approach. Metadata, diagnostics of energy balance, and interactive graphs of time series data are included for each station. Although the dataset was developed primarily to benchmark satellite-based remote sensing ET models of the OpenET initiative, there are many other potential uses, such as validation for a range of regional hydrologic and atmospheric models

GPS water level measurements for Indonesia's Tsunami Early Warning System

On Boxing Day 2004, a severe tsunami was generated by a strong earthquake in Northern Sumatra causing a large number of casualties. At this time, neither an offshore buoy network was in place to measure tsunami waves, nor a system to disseminate tsunami warnings to local governmental entities. Since then, buoys have been developed by Indonesia and Germany, complemented by NOAA's Deep-ocean Assessment and Reporting of Tsunamis (DART) buoys, and have been moored offshore Sumatra and Java. The suite of sensors for offshore tsunami detection in Indonesia has been advanced by adding GPS technology for water level measurements. <br><br> The usage of GPS buoys in tsunami warning systems is a relatively new approach. The concept of the German Indonesian Tsunami Early Warning System (GITEWS) (Rudloff et al., 2009) combines GPS technology and ocean bottom pressure (OBP) measurements. Especially for near-field installations where the seismic noise may deteriorate the OBP data, GPS-derived sea level heights provide additional information. <br><br> The GPS buoy technology is precise enough to detect medium to large tsunamis of amplitudes larger than 10 cm. The analysis presented here suggests that for about 68% of the time, tsunamis larger than 5 cm may be detectable

Machine learning methods for detecting and correcting data errors in water level telemetry systems

Water level data from telemetry stations can be used for early warning to prevent risk situations, such as floods and droughts. However, there is a possibility that the equipment in the telemetry station may fail, which will lead to errors in the data, resulting in false alarms or no warning of true alarms. Manually examining data is time-consuming and require expertise. As a result, the automated system is required. There are several algorithms available for detecting and correcting anomalous data, but the question remains as to which algorithm would be most suitable for telemetry data. To investigate and identify such an algorithm, statistical models, machine learning models, deep learning models, and reinforcement learning models are implemented and evaluated. For anomaly detection, we first evaluated statistical models using our modified sliding window algorithm called Only Normal Sliding Windows (ONSW) to assess their performance. We then proposed Deep Reinforcement Learning (DRL) models and compared them to Deep Learning models to determine their suitability for the task. Additionally, we developed a feature extraction approach that combines the saliency map and nearest neighbor extracted feature (SM+NNFE) to improve model performance. Various ensemble approaches were also implemented and compared to other competitive methods. For data imputation, we developed the Full Subsequence Matching (FSM) technique, which fills in missing values by imitating values from the most similar subsequence. Based on the results, machine learning models with ONSW are the best option for identifying abnormalities in telemetry water level data. Additionally, a deep reinforcement learning model could be used to identify abnormalities in crucial stations requiring further attention. Regarding data imputation, our technique outperforms other competitive approaches when dealing with water level data influenced by tides. However, relying solely on a single or limited number of models may be risky, as their performance could deteriorate in the future without being realized. Therefore, building models using ensemble techniques is a viable option for reducing errors caused by this issue

Data-driven modelling, forecasting and uncertainty analysis of disaggregated demands and wind farm power outputs

Correct analysis of modern power supply systems requires to evaluate much wider ranges of uncertainties introduced by the implementation of new technologies on both supply and demand sides. On the supply side, these uncertainties are due to the increased contributions of renewable generation sources (e.g., wind and PV), whose stochastic output variations are difficult to predict and control, as well as due to the significant changes in system operating conditions, coming from the implementation of various control and balancing actions, increased automation and switching functionalities, and frequent network reconfiguration. On the demand side, these uncertainties are due to the installation of new types of loads, featuring strong spatio-temporal variations of demands (e.g., EV charging), as well as due to the deployment of different demand-side management schemes. Modern power supply systems are also characterised by much higher availability of measurements and recordings, coming from a number of recently deployed advanced monitoring, data acquisition and control systems, and providing valuable information on system operating and loading conditions, state and status of network components and details on various system events, transients and disturbances. Although the processing of large amounts of measured data brings its own challenges (e.g., data quality, performance, and incorporation of domain knowledge), these data open new opportunities for a more accurate and comprehensive evaluation of the overall system performance, which, however, require new data-driven analytical approaches and modelling tools. This PhD research is aimed at developing and evaluating novel and improved data-driven methodologies for modelling renewable generation and demand, in general, and for assessing the corresponding uncertainties and forecasting, in particular. The research and methods developed in this thesis use actual field measurements of several onshore and offshore wind farms, as well as measured active and reactive power demands at several low voltage (LV) individual household levels, up to the demands at medium voltage (MV) substation level. The models are specifically built to be implemented for power system analysis and are actually used by a number of researchers and PhD students in Edinburgh and elsewhere (e.g., collaborations with colleagues from Italy and Croatia), which is discussed and illustrated in the thesis through the selected study cases taken from this joint research efforts. After literature review and discussion of basic concepts and definitions, the first part of the thesis presents data-driven analysis, modelling, uncertainty evaluation and forecasting of (predominantly residential) demands and load profiles at LV and MV levels. The analysis includes both aggregation and disaggregation of measured demands, where the latter is considered in the context of identifying demand-manageable loads (e.g., heating). For that purpose, periodical changes in demands, e.g., half-daily, daily, weekly, seasonal and annual, are represented with Fourier/frequency components and correlated with the corresponding exploratory meteorological variables (e.g., temperature, solar irradiance), allowing to select the combination of components maximising the positive or negative correlations as an additional predictor variable. Convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) are then used to represent dependencies among multiple dimensions and to output the estimated disaggregated time series of specific load types (with Bayesian optimisation applied to select appropriate CNN-BiLSTM hyperparameters). In terms of load forecasting, both tree-based and neural network-based models are analysed and compared for the day-ahead and week-ahead forecasting of demands at MV substation level, which are also correlated with meteorological data. Importantly, the presented load forecasting methodologies allow, for the first time, to forecast both total/aggregate demands and corresponding disaggregated demands of specific load types. In terms of the supply side analysis, the thesis presents data-driven evaluation, modelling, uncertainty evaluation and forecasting of wind-based electricity generation systems. The available measurements from both the individual wind turbines (WTs) and the whole wind farms (WFs) are used to formulate simple yet accurate operational models of WTs and WFs. First, available measurements are preprocessed, to remove outliers, as otherwise obtained WT/WF models may be biased, or even inaccurate. A novel simulation-based approach that builds on a procedure recommended in a standard is presented for processing all outliers due to applied averaging window (typically 10 minutes) and WT hysteresis effects (around the cut-in and cut-out wind speeds). Afterwards, the importance of distinguishing between WT-level and WF-level analysis is discussed and a new six-parameter power curve model is introduced for accurate modelling of both cut-in and cut-out regions and for taking into account operating regimes of a WF (WTs in normal/curtailed operation, or outage/fault). The modelling framework in the thesis starts with deterministic models (e.g., CNN-BiLSTM and power curve models) and is then extended to include probabilistic models, building on the Bayesian inference and Copula theory. In that context, the thesis presents a set of innovative data-driven WT and WF probabilistic models, which can accurately model cross-correlations between the WT/WF power output (Pout), wind speed (WS), air density (AD) and wind direction (WD). Vine Copula and Gaussian mixture Copula model (GMCM) are combined, for the first time, to evaluate the uncertainty of Pout values, conditioning on other explanatory variables (which may be either deterministic, or also uncertain). In terms of probabilistic wind energy forecasting, Bayesian CNN-BiLSTM model is used to analyse and efficiently handle high dimensionality of both input meteorological variables (WS, AD and WD) and additional uncertainties due to WF operating regimes. The presented results demonstrate that the developed Vine-GMCM and operational WF model can accurately integrate and effectively correlate all propagated uncertainties, ultimately resulting in much higher confidence levels of the forecasted WF power outputs than in the existing literature

Remote sensing of aerosols at night with the CoSQM sky brightness data

Aerosol optical depth is an important indicator of aerosol particle properties and their associated radiative impacts. AOD determination is very important to achieve relevant climate modelling. Most remote sensing techniques to retrieve aerosol optical depth are applicable to daytime given the high level of light available. The night represents half of the time but in such conditions only a few remote sensing methods are available. Among these approaches, the most reliable are moon photometers and star photometers. In this paper, we attempt to fill gaps in the aerosol detection performed with the aforementioned techniques using night sky brightness measurements during moonless nights with the novel CoSQM, a portable, low-cost and open-source multispectral photometer. In this paper, we present an innovative method for estimating the aerosol optical depth using an empirical relationship between the zenith night sky brightness measured at night with the CoSQM and the aerosol optical depth retrieved during daytime from the AErosol Robotic NETwork

Stacked LSTM for wind turbine yaw fault forecasting based on SCADA data analysis

With the final change of our society from the fossil resources reliant only to the one with higher use of renewable resources, the necessity of improving the efficiency and profitability of renewable resources is of primary importance. The performance of a wind turbine depends on the wind conditions as well as on the optimal extraction of kinetic energy and its transformation in the electricity. A wind turbine is a complex system and the coordination of all subsystems should be carefully orchestrated. The focus of this thesis is improvement of predict yaw faults relying on data collected via SCADA system. The data used in the experiment is kindly provided by Statkraft. A possibility to forecast the on-start of a fault alarm some time in advance gives possibility to implement required measures for eliminating the fault. Remote and automatic forecasting of such faults is utter importance for offshore wind parks that are emerging now all around the world. The goal of this thesis is to improve the algorithms implemented by fellow student \citeA{Tallaksrud}. The employed strategy focused on expanding the amount of information in the dataset. The study of influence of such parameters as rolling window length, a method used for handling the missing data, the number of features in the dataset and the number of time sequences in probability of a yaw alarm on-start. The preferential choice is a longer period in order to give time to fix the fault without risking long downtime and failure of other subsystems. The results for 12 models are presented together with a single-layer LSTM model for comparison of stacked models predicting better than a simple one. The best results was produced with a single-layer model with the with MAE score of 0.001. The model presents varying forecasting behaviour as a result of randomness and instability. The conclusion is that the stacked LSTM models do not cope with solving the problem in the thesisMed den endelige skiften av våres samfunn fra å være bare fossile ressursene basert til en med høyere bruk av fornybare ressurser, behovet for forbedring av effektivitet og lønsomhet av fornybære ressurser er av primær betydning. Ytelsen til en vindturbin avhenger av vindforholdene samtidig som optimal utvinning av kinetisk energi og dens transformasjon i elektrisiteten. En vindturbin er et komplekst system og koordineringen av alle delsystemer bør være nøye organisert. Fokuset i denne oppgaven er forbedring av yaw feil prediksjon basert på data samlet med SCADA-systemet. Data som ble brukt i forsøket er levert av Statkraft. En mulighet til å forutsi start av en feilalarm i forveien gir mulighet til å iverksette nødvendige tiltak for å eliminere feilen. Automatisk og mennesker-uavhengig varsling av slike feil er ekstremt viktig for offshore vindparker som dukker opp nå over hele verden. Målet med denne oppgaven er å forbedre algoritmene implementert av medstudent Tallaksrud (2021). Den anvendte strategien fokuserte på å utvide mengden informasjon i datasettet. Undersøkelse av påvirkning av parametere som rullende vindulengde, en metode som brukes for å håndtere de manglende dataene, antall parametrene i datasettet og antall tidssekvenser med sannsynlighet for en yaw ralarm ved start. Det foretrukne valget er en lengre periode for å gi tid til å fikse feilen uten å risikere lang nedetid og svikt i andre delsystemer. Resultatene for 12 modeller presenteres sammen med en enkeltlags LSTM-modell for sammenligning om stablede modeller klarer å predikere bedre enn en enkel model. De beste resultatene ble produsert med en enkeltlagsmodell med MAE-score på 0,001. Modellen presenterer varierende prognoseatferd som følge av tilfeldighet og ustabilitet. Derfor konklusjonen er at stablede LSTM modeller passer ikke til denne problmestillingen.M-M

Quantitative estimates of velocity sensitivity to surface melt variations at a large Greenland outlet glacier

This is the publisher's version, also available electronically from "http://www.ingentaconnect.com".The flow speed of Greenland outlet glaciers is governed by several factors, the relative importance of which is poorly understood. The delivery of surface-generated meltwater to the bed of alpine glaciers has been shown to influence glacier flow speed when the volume of water is sufficient to increase basal fluid pressure and hence basal lubrication. While this effect has also been demonstrated on the Greenland ice-sheet margin, little is known about the influence of surface melting on the large, marine-terminating outlet glaciers that drain the ice sheet. We use a validated model of meltwater input and GPS-derived surface velocities to quantify the sensitivity of glacier flow speed to changes in surface melt at Helheim Glacier during two summer seasons (2007–08). Our observations span ∼55 days near the middle of each melt season. We find that relative changes in glacier speed due to meltwater input are small, with variations of ∼45% in melt producing changes in velocity of ∼2–4%. These velocity variations are, however, of similar absolute magnitude to those observed at smaller glaciers and on the ice-sheet margin. We find that the glacier's sensitivity to variations in meltwater input decreases approximately exponentially with distance from the calving front. Sensitivity to melt varies with time, but generally increases as the melt season progresses. We interpret the time-varying sensitivity of glacier flow to meltwater input as resulting from changes in subglacial hydraulic routing caused by the changing volume of meltwater input

Monitoring the Quality of Information (QoI) for low-cost sensor networks

Mantaining a low cost sensor network calibrated for a certain time having trustable information is a highly complicated task. Thanks to redundant sensors in sensing devices and a sensor network, statistical tests can be applied in order to know whenever a calibration or a replacement should be done