Machine Learning Applications for Load Predictions in Electrical Energy Network

In this work collected operational data of typical urban and rural energy network are analysed for predictions of energy consumption, as well as for selected region of Nordpool electricity markets. The regression techniques are systematically investigated for electrical energy prediction and correlating other impacting parameters. The k-Nearest Neighbour (kNN), Random Forest (RF) and Linear Regression (LR) are analysed and evaluated both by using continuous and vertical time approach. It is observed that for 30 minutes predictions the RF Regression has the best results, shown by a mean absolute percentage error (MAPE) in the range of 1-2 %. kNN show best results for the day-ahead forecasting with MAPE of 2.61 %. The presented vertical time approach outperforms the continuous time approach. To enhance pre-processing stage, refined techniques from the domain of statistics and time series are adopted in the modelling. Reducing the dimensionality through principal components analysis improves the predictive performance of Recurrent Neural Networks (RNN). In the case of Gated Recurrent Units (GRU) networks, the results for all the seasons are improved through principal components analysis (PCA). This work also considers abnormal operation due to various instances (e.g. random effect, intrusion, abnormal operation of smart devices, cyber-threats, etc.). In the results of kNN, iforest and Local Outlier Factor (LOF) on urban area data and from rural region data, it is observed that the anomaly detection for the scenarios are different. For the rural region, most of the anomalies are observed in the latter timeline of the data concentrated in the last year of the collected data. For the urban area data, the anomalies are spread out over the entire timeline. The frequency of detected anomalies where considerably higher for the rural area load demand than for the urban area load demand. Observing from considered case scenarios, the incidents of detected anomalies are more data driven, than exceptions in the algorithms. It is observed that from the domain knowledge of smart energy systems the LOF is able to detect observations that could not have detected by visual inspection alone, in contrast to kNN and iforest. Whereas kNN and iforest excludes an upper and lower bound, the LOF is density based and separates out anomalies amidst in the data. The capability that LOF has to identify anomalies amidst the data together with the deep domain knowledge is an advantage, when detecting anomalies in smart meter data. This work has shown that the instance based models can compete with models of higher complexity, yet some methods in preprocessing (such as circular coding) does not function for an instance based learner such as k-Nearest Neighbor, and hence kNN can not option for this kind of complexity even in the feature engineering of the model. It will be interesting for the future work of electrical load forecasting to develop solution that combines a high complexity in the feature engineering and have the explainability of instance based models.publishedVersio

Earth Survey Applications Division: Research leading to the effective use of space technology in applications relating to the Earth's surface and interior

There are no author-identified significant results in this report

Optic Nerve Head Image Analysis for Glaucoma Progression Detection

Glaucoma is a leading cause of visual disability across the world and when diagnosed the glaucoma patient will spend the rest of their life receiving treatment in managed clinical care. In the glaucoma clinic, retinal and optic nerve head (ONH) imaging can be used to help the clinician to manage patient treatment appropriately. By providing high resolution images of the optic nerve head structures and identifying changes therein related to disease onset and progression, an objective measure can be obtained as to how well or badly treatment is preventing further disease damage. This thesis contributes to the field of glaucoma progression detection by the analysis of clinical imaging data using confocal scanning laser tomography (CSLT). Primarily it is an investigation of how best to appraise and optimise current algorithms which aim to detect these glaucomatous structural changes in the optic nerve head. This is done by addressing how the performance of these methods can be best assessed in the absence of a gold standard for glaucomatous structural progression. Glaucoma expert assessment of photographs of the optic disc is the current clinical standard of assessing glaucomatous damage evident in the ONH. This is used in this thesis to act as a reference standard by which these algorithms can be compared. In addition, the statistical principles underpinning trend detection techniques are also investigated along with the performance of these techniques to detect trends in CSLT data in the presence of different types of measurement noise and image quality. A new computer model is developed and validated to simulate stable series of CSLT images, with realistic variability, which can be used to benchmark the false-positive rates of current and future progression algorithms. In conclusion, the main results reported in this thesis show that uncertainties involved in expert assessment of change in ONH photographs limits this as a reference standard for structural change in glaucoma. In addition, since stability in clinical datasets is uncertain, simulation using modelled series is shown to provide a new benchmark for comparing methods of progression detection

Remote Sensing of the Aquatic Environments

The book highlights recent research efforts in the monitoring of aquatic districts with remote sensing observations and proximal sensing technology integrated with laboratory measurements. Optical satellite imagery gathered at spatial resolutions down to few meters has been used for quantitative estimations of harmful algal bloom extent and Chl-a mapping, as well as winds and currents from SAR acquisitions. The knowledge and understanding gained from this book can be used for the sustainable management of bodies of water across our planet

Regional application of the Pitman monthly rainfall-runoff model in Southern Africa incorporating uncertainty

Climate change and a growing demand for freshwater resources due to population increases and socio-economic changes will make water a limiting factor (in terms of both quantity and quality) in development. The need for reliable quantitative estimates of water availability cannot be over-emphasised. However, there is frequently a paucity of the data required for this quantification as many basins, especially in the developing world, are inadequately equipped with monitoring networks. Existing networks are also shrinking due mainly to shortages in human and financial resources. Over the past few decades mathematical models have been used to bridge the data gap by generating datasets for use in management and policy making. In southern Africa, the Pitman monthly rainfall-runoff model has enjoyed relatively popular use as a water resources estimation tool. However, it is acknowledged that models are abstractions of reality and the data used to drive them is imperfect, making the model outputs uncertain. While there is acknowledgement of the limitations of modelled data in the southern African region among water practitioners, there has been little effort to explicitly quantify and account for this uncertainty in water resources estimation tools and explore how it affects the decision making process. Uncertainty manifests itself in three major areas of the modelling chain; the input data used to force the model, the parameter estimation process and the model structural errors. A previous study concluded that the parameter estimation process for the Pitman model contributed more to the global uncertainty of the model than other sources. While the literature abounds with uncertainty estimation techniques, many of these are dependent on observations and are therefore unlikely to be easily applicable to the southern African region where there is an acute shortage of such data. This study focuses on two aspects of making hydrologic predictions in ungauged basins. Firstly, the study advocates the development of an a priori parameter estimation process for the Pitman model and secondly, uses indices of hydrological functional behaviour to condition and reduce predictive uncertainty in both gauged and ungauged basins. In this approach all the basins are treated as ungauged, while the historical records in the gauged basins are used to develop regional indices of expected hydrological behaviour and assess the applicability of these methods. Incorporating uncertainty into the hydrologic estimation tools used in southern Africa entails rethinking the way the uncertain results can be used in further analysis and how they will be interpreted by stakeholders. An uncertainty framework is proposed. The framework is made up of a number of components related to the estimation of the prior distribution of the parameters, used to generate output ensembles which are then assessed and constrained using regionalised indices of basin behavioural responses. This is premised on such indices being based on the best available knowledge covering different regions. This framework is flexible enough to be used with any model structure to ensure consistent and comparable results. While the aim is to eventually apply the uncertainty framework in the southern African region, this study reports on the preliminary work on the development and testing of the framework components based on South African basins. This is necessitated by the variations in the availability and quality of the data across the region. Uncertainty in the parameter estimation process was incorporated by assuming uncertainty in the physical and hydro-meteorological data used to directly quantify the parameter. This uncertainty was represented by the range of variability of these basin characteristics and probability distribution functions were developed to account for this uncertainty and propagate it through the estimation process to generate posterior distributions for the parameters. The results show that the framework has a great deal of potential but can still be improved. In general, the estimated uncertain parameters managed to produce hydrologically realistic model outputs capturing the expected regimes across the different hydro-climatic and geo-physical gradients examined. The regional relationships for the three indices developed and tested in this study were in general agreement with existing knowledge and managed to successfully provide a multi-criteria conditioning of the model output ensembles. The feedback loop included in the framework enabled a systematic re-examination of the estimation procedures for both the parameters and the indices when inconsistencies in the results were identified. This improved results. However, there is need to carefully examine the issues and problems that may arise within other basins outside South Africa and develop guidelines for the use of the framework.iText 1.4.6 (by lowagie.com

The use of natural site derived materials as concrete aggregate.

Includes abstract.Includes bibliographical references.This thesis focuses on the use of site-derived fine material, in its natural form, as aggregate in concrete construction. It is proposed that the utilisation of this type of concrete aggregate will lead to; the preservation of natural materials that would otherwise have to be beneficiated off site, the reduction of waste material produced on a construction site, and an overall energy saving

Evaluating uncertainty in water resources estimation in Southern Africa : a case study of South Africa

Hydrological models are widely used tools in water resources estimation, but they are simple representations of reality and are frequently based on inadequate input data and uncertainties in parameter values. Data observation networks are expensive to establish and maintain and often beyond the resources of most developing countries. Consequently, measurements are difficult to obtain and observation networks in many countries are shrinking, hence obtaining representative observations in space and time remains a challenge. This study presents some guidelines on the identification, quantification and reduction of sources of uncertainty in water resources estimation in southern Africa, a data scarce region. The analyses are based on example sub-basins drawn from South Africa and the application of the Pitman hydrological model. While it has always been recognised that estimates of water resources availability for the region are subject to possible errors, the quantification of these uncertainties has never been explicitly incorporated into the methods used in the region. The motivation for this study was therefore to contribute to the future development of a revised framework for water resources estimation that does include uncertainty. The focus was on uncertainties associated with climate input data, parameter estimation (and recognizing the uncertainty due model structure deficiencies) methods and water use data. In addition to variance based measures of uncertainty, this study also used a reservoir yield based statistic to evaluate model output uncertainty, which represents an integrated measure of flow regime variations and one that can be more easily understood by water resources managers. Through a sensitivity analysis approach, the results of the individual contribution of each source of uncertainty suggest regional differences and that clear statements about which source of uncertainty is likely to dominate are not generally possible. Parameter sensitivity analysis was used in identifying parameters which are important withinspecific sub-basins and therefore those to focus on in uncertainty analysis. The study used a simple framework for evaluating the combined contribution of uncertainty sources to model outputs that is consistent with the model limitations and data available, and that allows direct quantitative comparison between model outputs obtained by using different sources of information and methods within Spatial and Time Series Information Modelling (SPATSIM) software. The results from combining the sources of uncertainties showed that parameter uncertainty dominates the contribution to model output uncertainty. However, in some parts of the country especially those with complex topography, which tend to experience high rainfall spatial variability, rainfall uncertainty is equally dominant, while the contributions of evaporation and water use data uncertainty are relatively small. While the results of this study are encouraging, the weaknesses of the methods used to quantify uncertainty (especially subjectivity involved in evaluating parameter uncertainty) should not be neglected and require further evaluations. An effort to reduce data and parameter uncertainty shows that this can only be achieved if data access at appropriate scale and quality improves. Perhaps the focus should be on maintaining existing networks and concentrating research efforts on making the most out of the emerging data products derived from remote sensing platforms. While this study presents some initial guidelines for evaluating uncertainty in South Africa, there is need to overcome several constraints which are related to data availability and accuracy, the models used and the capacity or willingness to adopt new methods that incorporate uncertainty. The study has provided a starting point for the development of new approaches to modelling water resources in the region that include uncertain estimates

Factors influencing wetland distribution and structure, including ecosystem function of ephemeral wetlands, in Nelson Mandela Bay Municipality (NMBM), South Africa

The Nelson Mandela Bay Municipality (NMBM) is a semi-arid area along the southern coastline of South Africa (SA). Until recently, there was no systematic approach to research on wetland systems in the NMBM. The systematic identification of wetlands was made more difficult by the relatively large number of small, ephemeral systems that can be difficult to delineate. This has meant that fundamental knowledge on wetland distribution, structure and function has been limited and, consequently, management and conservation strategies have been based on knowledge on systems from other regions of the country. Environmental processes occur at different spatial and temporal scales. These processes have an effect on the abiotic factors and biotic structure of wetlands, resulting in inherently complex systems. The location of the NMBM provides a good study area to research some of these environmental and biological attributes at different spatial scales, due to the variability in the underlying geology, geomorphology, vegetation types and the spatial and temporal variability in rainfall, within a relatively small area of 1951 km2. Thus, the aim of this study was to determine the factors influencing wetland distribution, structure and ecosystem functioning within the NMBM. The first Research Objective of work presented here was to identify wetlands using visual interpretation of aerial photographs. A total of 1712 wetlands were identified within the NMBM using aerial photographs, covering an area of 17.88 km2 (Chapter 5). The majority of these wetlands were depressions, seeps and wetland flats. Valley bottom wetlands (channelled and unchannelled) and floodplain wetlands were also identified. A range of wetland sizes was recorded, with 86% of the wetlands being less than 1 ha in size and the largest natural wetland being a floodplain wetland of 57 ha, located south of the Swartkops River. The identified wetlands were used to create a wetland occurrence model using logistic regression (LR) techniques (Chapter 5), in accordance with Objective 2 of the study. An accuracy of 66% was obtained, which was considered acceptable for a semi-arid climate with a relatively high degree of spatial and temporal rainfall variability. The model also highlighted several key environmental variables that are associated with wetland occurrence and distribution at various spatial scales. Some of the important variables included precipitation, evapotranspiration, temperature, flow accumulation and groundwater occurrence. Wetland distribution patterns were described in Chapter 6. Spatial statistics were used to identify whether wetlands are clustered and, therefore, form mosaics within the surrounding landscape (Objective 3). Systems were found to be highly clustered, with 43% of wetlands located within 200 m of another system. Clustering and wetland presence was especially prominent in the southern portion of the Municipality, which is also associated with a higher mean annual precipitation. Smaller wetlands were also significantly more clustered than larger systems (Average Nearest Neighbour statistic, p-value < 0.0001). Average distances also significantly varied according to HGM type, with depressions being the most geographically isolated wetland type compared to the other HGM types. Overall, distances between wetlands indicated good proximal connectivity. Potentially vulnerable areas associated with wetland systems were identified successfully using landscape variables, in accordance with Objective 4. These variables were: land cover, slope gradient, flow accumulation, APAN evaporation, mean annual precipitation (MAP) and annual heat units. The existing Critical Biodiversity Network was also used in connection with these variables to further identify potentially vulnerable areas. The abiotic and biotic characteristics were decribed for three hydrogeomorphic (HGM) types at a total of 46 wetland sites (Chapter 7), as per Objective 5. Depressions, seeps and wetland flats were sampled across the different geological, vegetation and rainfall zones within the NMBM. The wetland sites were delineated up to Level 6 of the Classification System used in SA, and the various abiotic and biotic characteristics of these systems were defined. A total of 307 plant, 144 aquatic macroinvertebrate and 10 tadpole species were identified. Of these species, over 90 species were Eastern Cape and SA endemic species, as well as three threatened species on the IUCN Red List. Multivariate analyses (including Bray-Curtis similarity resemblance analyses, distance-based redundancy analyses, SIMPER analyses and BIOENV analysis in Primer), together with environmental data, were used to define community structure at an HGM level, in accordance with Objective 5. The importance of the spatial scale of the environmental data used to define plant and macroinvertebrate community structure was described in Chapter 7, to address Objective 6. The results showed that both broad-scale and site-level characteristics were important in distinguishing community structure within the HGM types that superseded general location, the sample timing or the stage of inundation. These results also indicated that a combination of both landscape and site-level data are important in defining the community structure in the various HGM types. Some of the important environmental variables that explained some of species assemblages were similar to those in the wetland occurrence model (Chapter 5), with some additional hydrological and soil physico-chemical parameters (e.g. soil electrical conductivity, soil pH, and surface and subsurface water nutrients). These significant variables indicate the complex, multi-scalar role of environmental attributes on wetland distribution, structure and function

Semi-automatic mapping of shallow landslides using free Sentinel-2 images and Google Earth Engine

The global availability of Sentinel-2 data and the widespread coverage of cost-free and high-resolution images nowadays give opportunities to map, at a low cost, shallow landslides triggered by extreme events (e.g. rainfall, earthquakes). Rapid and low-cost shallow landslide mapping could improve damage estimations, susceptibility models and land management. This work presents a two-phase procedure to detect and map shallow landslides. The first is a semi-automatic methodology allowing for mapping potential shallow landslides (PLs) using Sentinel-2 images. The PL aims to detect the most affected areas and to focus on them an high-resolution mapping and further investigations. We create a GIS-based and user-friendly methodology to extract PL based on pre- and post-event normalised difference vegetation index (NDVI) variation and geomorphological filtering. In the second phase, the semi-automatic inventory was compared with a benchmark landslide inventory drawn on high-resolution images. We also used Google Earth Engine scripts to extract the NDVI time series and to make a multi-temporal analysis. We apply this procedure to two study areas in NW Italy, hit in 2016 and 2019 by extreme rainfall events. The results show that the semi-automatic mapping based on Sentinel-2 allows for detecting the majority of shallow landslides larger than satellite ground pixel (100 m2). PL density and distribution match well with the benchmark. However, the false positives (30 % to 50 % of cases) are challenging to filter, especially when they correspond to riverbank erosions or cultivated land.</p