Optimal energy management for a grid-tied solar PV-battery microgrid: A reinforcement learning approach

There has been a shift towards energy sustainability in recent years, and this shift should continue. The steady growth of energy demand because of population growth, as well as heightened worries about the number of anthropogenic gases released into the atmosphere and deployment of advanced grid technologies, has spurred the penetration of renewable energy resources (RERs) at different locations and scales in the power grid. As a result, the energy system is moving away from the centralized paradigm of large, controllable power plants and toward a decentralized network based on renewables. Microgrids, either grid-connected or islanded, provide a key solution for integrating RERs, load demand flexibility, and energy storage systems within this framework. Nonetheless, renewable energy resources, such as solar and wind energy, can be extremely stochastic as they are weather dependent. These resources coupled with load demand uncertainties lead to random variations on both the generation and load sides, thus challenging optimal energy management. This thesis develops an optimal energy management system (EMS) for a grid-tied solar PV-battery microgrid. The goal of the EMS is to obtain the minimum operational costs (cost of power exchange with the utility and battery wear cost) while still considering network constraints, which ensure grid violations are avoided. A reinforcement learning (RL) approach is proposed to minimize the operational cost of the microgrid under this stochastic setting. RL is a reward-motivated optimization technique derived from how animals learn to optimize their behaviour in new environments. Unlike other conventional model-based optimization approaches, RL doesn't need an explicit model of the optimization system to get optimal solutions. The EMS is modelled as a Markov Decision Process (MDP) to achieve optimality considering the state, action, and reward function. The feasibility of two RL algorithms, namely, conventional Q-learning algorithm and deep Q network algorithm, are developed, and their efficacy in performing optimal energy management for the designed system is evaluated in this thesis. First, the energy management problem is expressed as a sequential decision-making process, after which two algorithms, trading, and non-trading algorithm, are developed. In the trading algorithm case, excess microgrid's energy can be sold back to the utility to increase revenue, while in the latter case constraining rules are embedded in the designed EMS to ensure that no excess energy is sold back to the utility. Then a Q-learning algorithm is developed to minimize the operational cost of the microgrid under unknown future information. Finally, to evaluate the performance of the proposed EMS, a comparison study between a trading case EMS model and a non-trading case is performed using a typical commercial load curve and PV generation profile over a 24- hour horizon. Numerical simulation results indicated that the algorithm learned to select an optimized energy schedule that minimizes energy cost (cost of power purchased from the utility based on the time-varying tariff and battery wear cost) in both summer and winter case studies. However, comparing the non-trading EMS to the trading EMS model operational costs, the latter one decreased cost by 4.033% in the summer season and 2.199% in the winter season. Secondly, a deep Q network (DQN) method that uses recent learning algorithm enhancements, including experience replay and target network, is developed to learn the system uncertainties, including load demand, grid prices and volatile power supply from the renewables solve the optimal energy management problem. Unlike the Q-learning method, which updates the Q-function using a lookup table (which limits its scalability and overall performance in stochastic optimization), the DQN method uses a deep neural network that approximates the Q- function via statistical regression. The performance of the proposed method is evaluated with differently fluctuating load profiles, i.e., slow, medium, and fast. Simulation results substantiated the efficacy of the proposed method as the algorithm was established to learn from experience to raise the battery state of charge and optimally shift loads from a one-time instance, thus supporting the utility grid in reducing aggregate peak load. Furthermore, the performance of the proposed DQN approach was compared to the conventional Q-learning algorithm in terms of achieving a minimum global cost. Simulation results showed that the DQN algorithm outperformed the conventional Q-learning approach, reducing system operational costs by 15%, 24%, and 26% for the slow, medium, and fast fluctuating load profiles in the studied cases

Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach

In the near future, microgrids will become more prevalent as they play a critical role in integrating distributed renewable energy resources into the main grid. Nevertheless, renewable energy sources, such as solar and wind energy can be extremely volatile as they are weather dependent. These resources coupled with demand can lead to random variations on both the generation and load sides, thus complicating optimal energy management. In this article, a reinforcement learning approach has been proposed to deal with this non-stationary scenario, in which the energy management system (EMS) is modelled as a Markov decision process (MDP). A novel modification of the control problem has been presented that improves the use of energy stored in the battery such that the dynamic demand is not subjected to future high grid tariffs. A comprehensive reward function has also been developed which decreases infeasible action explorations thus improving the performance of the data-driven technique. A Q-learning algorithm is then proposed to minimize the operational cost of the microgrid under unknown future information. To assess the performance of the proposed EMS, a comparison study between a trading EMS model and a non-trading case is performed using a typical commercial load curve and PV profile over a 24-h horizon. Numerical simulation results indicate that the agent learns to select an optimized energy schedule that minimizes energy cost (cost of power purchased from the utility and battery wear cost) in all the studied cases. However, comparing the non-trading EMS to the trading EMS model operational costs, the latter one was found to decrease costs by 4.033% in summer season and 2.199% in winter season

CO2 sequestration using brine impacted fly fish

Magister Scientiae - MScIt was hypothesized that South African FA and brine could sequester CO2 through mineral carbonation. A statistical approach was undertaken to optimize the % CaCO3 formed from FA/brine/CO2 interaction with input parameters of temperature, pressure, particle size and solid/liquid ratio (S/L) being varied. The ranges adopted for the input parameters were: temperature of 30 ºC or 90 ºC; pressure of 1 Mpa or 4 Mpa; four particle sizes namely bulk ash, > 150 μm, < 20 μm and 20 μm- 150 μm particle size range; S/L ratios of 0.1, 0.5 or 1. The FA/ brine dispersions were carbonated in a high pressure reactor varying the above mentioned input parameters. The fresh Secunda FA of various size fractions was characterized morphologically using scanning electron microscopy, chemically using X-ray fluorescence and mineralogically using qualitative X-ray diffraction. The carbonated solid residues on the other hand were characterized using quantitative X-ray diffraction, scanning electron microscopy, thermal gravimetic analysis and Chittick tests. The raw brine from Tutuka together with the carbonation leachates were characterized using inductively coupled mass spectrometry and ion chromatography. Total acid digestion was carried out to evaluate the differences in the total elemental content in both the fresh ash and the carbonated solid residues. The results suggested that South African FA from Secunda belongs to class F based on the CaO content as well as the total alumina, silica and ferric oxide content, while the RO brine from Tutuka were classified as NaSO4 waters.South Afric

Carbonation of brine impacted fractionated coal fly ash: Implications for CO2 sequestration

Coal combustion by-products such as fly ash (FA), brine and CO2 from coal fired power plants have the potential to impact negatively on the environment. FA and brine can contaminate the soil, surface and ground water through leaching of toxic elements present in their matrices while CO2 has been identified as a green house gas that contributes significantly towards the global warming effect. Reaction of CO2 with FA/brine slurry can potentially provide a viable route for CO2 sequestration via formation of mineral carbonates. Fractionated FA has varying amounts of CaO which not only increases the brine pH but can also be converted into an environmentally benign calcite. Carbonation efficiency of fractionated and brine impacted FA was investigated in this study. Controlled carbonation reactions were carried out in a reactor set-up to evaluate the effect of fractionation on the carbonation efficiency of FA. Chemical and mineralogical characteristics of fresh and carbonated ash were evaluated using XRF, SEM, and XRD. Brine effluents were characterized using ICP-MS and IC. A factorial experimental approach was employed in testing the variables. The 20–150 μm size fraction was observed to have the highest CO2 sequestration potential of 71.84 kg of CO2 per ton of FA while the >150 μm particles had the lowest potential of 36.47 kg of CO2 per ton of FA. Carbonation using brine resulted in higher degree of calcite formation compared to the ultra-pure water carbonated residues.Web of Scienc

CO2 sequestration using brine impacted fly ash

>Magister Scientiae - MScCoal combustion accounts for over 40 % of the world's energy production and this figure is projected to increase with increasing human population and industrialization. The combustion of coal leads to the generation of waste products such as fly ash (FA), brine from water treatment, bottom ash, slag, flue gas desulphurization products (FGD) and gas emissions such as N20, and C02. The emissions contribute to air pollution and global warming, while FA, brines, and FGD are possible soil and water pollutants. In order to minimize the environmental impact of coal combustion, mitigation of the effects of coal burning processes such as the waste products (FA, brine, bottom ash, slag and FGD) and gas emissions is required. This study investigated utilization of the Secunda FA (class F) and reverse osmosis (RO) Tutuka brine to sequester C02 in an attempt to make coal power production more environmentally sustainable. It was hypothesized that South African FA and brine could sequester C02 through mineral carbonation. A statistical approach was undertaken to optimize the % CaC03 formed from FAlbrine/C02 interaction with input parameters of temperature, pressure, particle size and solid/liquid ratio (S/L) being varied. The ranges adopted for the input parameters were: temperature of 30°C or 90 °C; pressure of 1 Mpa or 4 Mpa; four particle sizes namely bulk ash, > 150 11m, < 20 11m and 20 urn- 150 11m particle size range; S/L ratios ofO.1, 0.5 or 1. The FA! brine dispersions were carbonated in a high pressure reactor varying the above mentioned input parameters. The fresh Secunda FA of various size fractions was characterized morphologically using scanning electron microscopy, chemically using X-ray fluorescence and mineralogically using qualitative X-ray diffraction. The carbonated solid residues on the other hand were characterized using quantitative X-ray diffraction, scanning electron microscopy, thermal gravimetic analysis and Chittick tests. The raw brine from Tutuka together with the carbonation leachates were characterized using inductively coupled mass spectrometry and ion chromatography. Total acid digestion was carried out to evaluate the differences in the total elemental content in both the fresh ash and the carbonated solid residues. The results suggested that South African FA from Secunda belongs to class F based on the CaO content as well as the total alumina, silica and ferric oxide content, while the RO brine from Tutuka were classified as NaS04 waters. Mineral carbonation occurred and ranged between 2.75 % and 6.5 % of CaC03 depending on the input parameters. Two polymorphs of CaC03 were identified in the carbonated residues i.e. calcite and aragonite. The carbonated ash/brine leachates were cleaner with respect to major and trace element concentration compared to raw brine thus the carbonation process could be used to improve the quality of brines generated in the power industry. Removal of the major elements from brine was as follows Ca-74.8 %, Na- 28.7 %, Mg- 98 %, K- 82.9 %, S04- 20.8 %. Hundred percent removal was observed for traces of Fe, Al, Mn, Cu, Zn, Pb, Ni, As, Ti, Sr, Se, Si and N03. However Mo, V, B, and Cl concentrations increased by 72.5 %, 94 %,48.2 % and 7.2 % respectively after carbonation at 90°C, 4 Mpa, S/L ratio of 1 using the bulk ash. The parameters found to be of most significance in the carbonation process were the main effects of temperature, particle size and S/L ratio while the interactions of temperature and particle size as well as the interaction of temperature with S/L ratio were also found to be significant. The statistical approach led to a clear understanding of the effect of each input parameter as well as the ansmg interactions. The conditions of 90°C, 4 Mpa, using bulk ash at a S/L ratio of 1 resulted in the highest yield of % CaC03 with a value of 6.5 %. Theoretically one ton of Secunda FA containing 9.2 % of CaO could sequester 0.083 tons of C02. With the optimized protocol developed in this study bearing in mind that the carbonation efficiency is 75.54%, 1 ton of Secunda FA could sequester 0.062 tons of CO2. This translates to 0.65 % of CO2 produced annually at Secunda plant being sequestered in the FAlbrine dispersions. In other words, 16 tons of FA are required to sequester a ton of C02 annually. It was also observed that carbonation using brine resulted in higher carbonation efficiency than carbonation using water as the Ca2+ component in the brine contributed towards the Ca 2+concentration

Comparison of CO2 capture by ex-situ accelerated carbonation and in in-situ naturally weathered coal fly ash

Natural weathering at coal power plants ash dams occurs via processes such as carbonation, dissolution, co-precipitation and fluid transport mechanisms which are responsible for the long-term chemical, physical and geochemical changes in the ash. Very little information is available on the natural carbon capture potential of wet or dry ash dams. This study investigated the extent of carbon capture in a wet-dumped ash dam and the mineralogical changes promoting CO2 capture, comparing this natural phenomenon with accelerated ex-situ mineral carbonation of fresh fly ash (FA). Significant levels of trace elements of Sr, Ba and Zr were present in both fresh and weathered ash. However Nb, Y, Sr, Th and Ba were found to be enriched in weathered ash compared to fresh ash. Mineralogically, fresh ash is made up of quartz, mullite, hematite, magnetite and lime while weathered and carbonated ashes contained additional phases such as calcite and aragonite. Up to 6.5 wt % CO2 was captured by the fresh FA with a 60% conversion of calcium to CaCO3 via accelerated carbonation (carried out at 2 h, 4Mpa, 90 o C, bulk ash and a S/L ratio of 1). On the other hand 6.8 wt % CO2 was found to have been captured by natural carbonation over a period of 20 years of wet disposed ash. Thus natural carbonation in the ash dumps is significant and may be effective in capturing CO2.Web of Scienc

The Effectiveness of Strategic Technology Forecasting In the Telecommunication Industry in Africa: A Case of West Indian Ocean Cable Company

A Research Project Report Submitted to the Chandaria School of Business in Partial Fulfilment of the Requirement for the Award of the Degree of Master of Business Administration (MBA)The purpose of this study was to evaluate the effectiveness of the strategic technology forecasting techniques used at WIOCC and their impact on the delivery of robust and creative technology solutions across Africa. The study specifically sought to establish how expert opinion as a technology forecasting method affects the technology solutions provided at WIOCC, how technology monitoring and market intelligence affects the technology solutions provided at WIOCC and what other technology forecasting techniques or methods can be used to enhance the technology solutions provided at WIOCC. A descriptive research was used and the study used questionnaires to collect data. The target population was determined by the case study organisation, WIOCC and was the WIOCC employees in a management position. The target population was 54 but a sample of 48 respondents was used. 48 questionnaires were distributed and only 40 were filled and returned. Data was analysed using descriptive statistics where the percentages and means of the variables was computed. For the inferential statistics a correlation analysis was done for the dependent and independent variable by using SPSS and the results were presented in figures and tables. The findings revealed that monitoring and intelligence methods had the highest rating followed by expert opinion, trend analysis and creativity. It was also established that technology forecasting has enabled WIOCC to have competitive products and the firm has had improved performance attributed to technology forecasting. It was also revealed that technology forecasting has resulted into effectiveness. To establish the role of expert opinion in technology forecasting the study revealed that WIOCC makes use of varied technologies in the delivery of its technology solution and engages. The findings revealed that technology monitoring is considered vital in technology and innovation management at WIOCC, and technology monitoring has enabled WIOCC strategically position itself against competitors. There was uncertainty on whether the firm has allocated necessary resources to effectively detect and monitor technology trends or if it continuously observes technologies and markets over a period of time to determine trends. It was also revealed that the firm undertakes constant evaluations on its technological core competencies and external developments and it considers insights from internal and external v customers who may have information on potential opportunities and threats. The findings also showed that through monitoring and intelligence, WIOCC has minimised risk in strategic planning. The findings revealed that additional technology forecasting techniques have enabled researchers to document their findings in relation to the phenomenon of interest. Additional technology forecasting techniques applied have also resulted in timely and technology-specific solutions, despite having additional technology forecasting techniques many respondents were not sure if firms’ technology innovation at WIOCC is still unstructured and unsystematic. In addition, technology forecasting techniques have created an opportunity for improvement. As per the findings, at WIOCC, monitoring and intelligence methods are the most pronounced followed by expert opinion, trend analysis and creativity. Use of technology forecasting has enabled the firm to have very competitive products that have resulted in overall improved performance and organizational effectiveness. At WIOCC, technology monitoring is considered very vital as the nature of the industry demands use of technology to remain relevant in the market and to guarantee this, the firm has to pursue continuous evaluations on its technological core competencies to deter potential threats. There is awareness on access to a variety of techniques that can be developed for technological forecasting and use of past events to predict the future. This has been made possible by the use of additional technology forecasting techniques that have enabled researchers to document their findings in relation to the phenomenon of interest. The study recommended that WIOCC needs to undertake a thorough evaluation of its system to establish whether its efficiency is attributed to technology forecasting. Also, it is necessary for WIOCC to have in place policies to guarantee the same. There is a need of setting up a clear policy guideline on technology forecasting techniques and the firm needs to gain access to a variety of techniques that can be developed for technological forecasting. More studies need to be done on related firms in order to be able to generalize the findings, there is also a need to undertake further studies to determine the challenges facing technology forecasting at WIOCC

Re-use of South African fly ash for CO2 capture and brine remediation.

Philosophiae Doctor - PhDCoal combustion accounts for 95% of electricity generation in South Africa while globally coal combustion for energy generation stands at 42%. It has been predicted that coal utilization for energy generation will continue due to its low cost and availability in huge quantities in different parts of the world. Additionally brine and gaseous emissions are produced in the power generation and coal combustion processes. In fact, it has been established that CO2 emissions from power plants are the main cause of the green-house effect leading to global warming. Mitigation of the effects of disposal of fly ash, brine and CO2 emissions is critical for sustainable energy generation from coal and environmental protection. The study investigated whether South African coal fly ash could be used for brine remediation and CO2 capture using fly ash based hydrotalcites and zeolites. Four main objectives were investigated. These were; firstly, to compare the natural CO2 capture potential of a power station ash dam with an accelerated ex-situ mineral carbonation process. Secondly, to probe the effect of accelerated ex-situ mineral carbonation on brine quality with regards to major, minor and trace elements concentration. Furthermore, the study investigated the feasibility of synthesizing hydrotalcites from fly ash by optimizing the synthesis parameters such as acid concentration, aging time, aging temperature, pH during aging, crystallization time and crystallization temperature. Finally the study compared the CO2 adsorption capacities of the fly ash based hydrotalcites with fly ash based zeolites NaA, and NaX. The natural carbonation potential of the wet disposed ash dam at Secunda was investigated by coring a 20 year old dam. Three cores (SI, S2 and S3) were obtained by air flush coring the dam along a geophysical line and establishing the geophysical profile of the three cores. The surface of the three cores was of medium resistivity with values between 9.3 and 12.2 nm while the midsections were of low resistivity with values ranging between 4 and 7 nm. The base section of core SI had a resistivity of 28.3 nm, that of S2 was between 16.2 and 21.4 nm and that of S3 between 12.2 and 16.2 nm; implying that SI had the lowest salt load while S3 had the highest salt content. Moisture content was observed to be high deeper down the profiles of S2 and S3 with samples appearing water logged while SI had the highest moisture content at the surface showing the inhomogeneity of the ash dam. The morphology of fresh fly ash taken from the ash collection hoppers at Secunda was observed to be spherical. Weathered ash from the ash dam showed irregularly agglomerated particles while accelerated ex-situ mineral carbonation resulted in the formation of acicular particles of calcite. Fresh ash, weathered ash and the accelerated carbonated ash were all class F with a sum total of silica, alumina and iron oxide totaling more than 70%. A reduction in silica and alumina content with instability of fly ash. Dumping of spent iron catalyst (resulting from the petrochemical operations as Sasol) on the ash dam led to an increase in Fe203 content of the weathered ash. Enrichment of Nb, Sr, Y, Th, Na, Cl, S04, K and S with natural carbonation as well as during accelerated ex-situ mineral carbonation was observed and was due to the contact of ash with brine during these two processes. Reduction of Zr, Rb, Pb, Ni, Co and V content of ash was observed with weathering. Mineralogically, all the ash samples had main phases of mullite, quartz, magnetite and hematite, with weathered and accelerated carbonated ash having additional phases of calcite. The aluminosilicious nature of the three ashes was identified by structural evaluation using Fourier transform infrared analysis which revealed that, bands associated with C-O in-plane and out of plane bending of carbonates was only visible in weathered and carbonated ash

Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach

In the near future, microgrids will become more prevalent as they play a critical role in integrating distributed renewable energy resources into the main grid. Nevertheless, renewable energy sources, such as solar and wind energy can be extremely volatile as they are weather dependent. These resources coupled with demand can lead to random variations on both the generation and load sides, thus complicating optimal energy management. In this article, a reinforcement learning approach has been proposed to deal with this non-stationary scenario, in which the energy management system (EMS) is modelled as a Markov decision process (MDP). A novel modification of the control problem has been presented that improves the use of energy stored in the battery such that the dynamic demand is not subjected to future high grid tariffs. A comprehensive reward function has also been developed which decreases infeasible action explorations thus improving the performance of the data-driven technique. A Q-learning algorithm is then proposed to minimize the operational cost of the microgrid under unknown future information. To assess the performance of the proposed EMS, a comparison study between a trading EMS model and a non-trading case is performed using a typical commercial load curve and PV profile over a 24-h horizon. Numerical simulation results indicate that the agent learns to select an optimized energy schedule that minimizes energy cost (cost of power purchased from the utility and battery wear cost) in all the studied cases. However, comparing the non-trading EMS to the trading EMS model operational costs, the latter one was found to decrease costs by 4.033% in summer season and 2.199% in winter season