Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation

Grid-connected microgrids consisting of renewable energy sources, battery storage, and load require an appropriate energy management system that controls the battery operation. Traditionally, the operation of the battery is optimised using 24 h of forecasted data of load demand and renewable energy sources (RES) generation using offline optimisation techniques, where the battery actions (charge/discharge/idle) are determined before the start of the day. Reinforcement Learning (RL) has recently been suggested as an alternative to these traditional techniques due to its ability to learn optimal policy online using real data. Two approaches of RL have been suggested in the literature viz. offline and online. In offline RL, the agent learns the optimum policy using predicted generation and load data. Once convergence is achieved, battery commands are dispatched in real time. This method is similar to traditional methods because it relies on forecasted data. In online RL, on the other hand, the agent learns the optimum policy by interacting with the system in real time using real data. This paper investigates the effectiveness of both the approaches. White Gaussian noise with different standard deviations was added to real data to create synthetic predicted data to validate the method. In the first approach, the predicted data were used by an offline RL algorithm. In the second approach, the online RL algorithm interacted with real streaming data in real time, and the agent was trained using real data. When the energy costs of the two approaches were compared, it was found that the online RL provides better results than the offline approach if the difference between real and predicted data is greater than 1.6%

Unbiased Shape Compactness for Segmentation

We propose to constrain segmentation functionals with a dimensionless, unbiased and position-independent shape compactness prior, which we solve efficiently with an alternating direction method of multipliers (ADMM). Involving a squared sum of pairwise potentials, our prior results in a challenging high-order optimization problem, which involves dense (fully connected) graphs. We split the problem into a sequence of easier sub-problems, each performed efficiently at each iteration: (i) a sparse-matrix inversion based on Woodbury identity, (ii) a closed-form solution of a cubic equation and (iii) a graph-cut update of a sub-modular pairwise sub-problem with a sparse graph. We deploy our prior in an energy minimization, in conjunction with a supervised classifier term based on CNNs and standard regularization constraints. We demonstrate the usefulness of our energy in several medical applications. In particular, we report comprehensive evaluations of our fully automated algorithm over 40 subjects, showing a competitive performance for the challenging task of abdominal aorta segmentation in MRI.Comment: Accepted at MICCAI 201

Dual-Layer Q-Learning Strategy for Energy Management of Battery Storage in Grid-Connected Microgrids

This is the final version. Available on open access from MDPI via the DOI in this recordData Availability Statement: The data are available from the lead or the corresponding author upon reasonable requestsReal-time energy management of battery storage in grid-connected microgrids can be very challenging due to the intermittent nature of renewable energy sources (RES), load variations, and variable grid tariffs. Two reinforcement learning (RL)–based energy management systems have been previously used, namely, offline and online methods. In offline RL, the agent learns the optimum policy using forecasted generation and load data. Once the convergence is achieved, battery commands are dispatched in real time. The performance of this strategy highly depends on the accuracy of the forecasted data. An agent in online RL learns the best policy by interacting with the system in real time using real data. Online RL deals better with the forecasted error but can take a longer time to converge. This paper proposes a novel dual layer Q-learning strategy to address this challenge. The first (upper) layer is conducted offline to produce directive commands for the battery system for a 24 h horizon. It uses forecasted data for generation and load. The second (lower) Q-learning-based layer refines these battery commands every 15 min by considering the changes happening in the RES and load demand in real time. This decreases the overall operating cost of the microgrid as compared with online RL by reducing the convergence time. The superiority of the proposed strategy (dual-layer RL) has been verified by simulation results after comparing it with individual offline and online RL algorithms.Engineering and Physical Sciences Research Council (EPSRC

Weinberg like sum rules revisited

The generalized Weinberg sum rules containing the difference of isovector vector and axial-vector spectral functions saturated by both finite and infinite number of narrow resonances are considered. We summarize the status of these sum rules and analyze their overall agreement with phenomenological Lagrangians, low-energy relations, parity doubling, hadron string models, and experimental data.Comment: 31 pages, noticed misprints are corrected, references are added, and other minor corrections are mad

Fragment Flow and the Nuclear Equation of State

We use the Boltzmann-Uehling-Uhlenbeck model with a momentum-dependent nuclear mean field to simulate the dynamical evolution of heavy ion collisions. We re-examine the azimuthal anisotropy observable, proposed as sensitive to the equation of state of nuclear matter. We obtain that this sensitivity is maximal when the azimuthal anisotropy is calculated for nuclear composite fragments, in agreement with some previous calculations. As a test case we concentrate on semi-central $^{197}{\rm Au}\ +\ ^{197}{\rm Au}$ collisions at 400 $A$ MeV.Comment: 12 pages, ReVTeX 3.0. 12 Postscript figures, uuencoded and appende

Solutions of Several Coupled Discrete Models in terms of Lame Polynomials of Order One and Two

Coupled discrete models abound in several areas of physics. Here we provide an extensive set of exact quasiperiodic solutions of a number of coupled discrete models in terms of Lame polynomials of order one and two. Some of the models discussed are (i) coupled Salerno model, (ii) coupled Ablowitz-Ladik model, (iii) coupled saturated discrete nonlinear Schrodinger equation, (iv) coupled phi4 model, and (v) coupled phi6 model. Furthermore, we show that most of these coupled models in fact also possess an even broader class of exact solutions.Comment: 31 pages, to appear in Pramana (Journal of Physics) 201

Non Abelian Geometrical Tachyon

We investigate the dynamics of a pair of coincident D5 branes in the background of $k$ NS5 branes. It has been proposed by Kutasov that the system with a single probing D-brane moving radially in this background is dual to the tachyonic DBI action for a non-BPS Dp brane. We extend this proposal to the non-abelian case and find that the duality still holds provided one promotes the radial direction to a matrix valued field associated with a non-abelian geometric tachyon and a particular parametrization for the transverse scalar fields is chosen. The equations of motion of a pair of coincident D5 branes moving in the NS5 background are determined. Analytic and numerical solutions for the pair are found in certain simplified cases in which the U(2) symmetry is broken to $U(1) \times U(1)$ corresponding to a small transverse separation of the pair. For certain range of parameters these solutions describe periodic motion of the centre of mass of the pair 'bouncing off' a finite sized throat whose minimum size is limited by the D5 branes separation.Comment: 18 pages, 2 figures, PdfLatex: references added.accepted for publication in JHE

Supersymmetric Intersecting Branes on the Waves

We construct a general family of supersymmetric solutions in time- and space-dependent wave backgrounds in general supergravity theories describing single and intersecting p-branes embedded into time-dependent dilaton-gravity plane waves of an arbitrary (isotropic) profile, with the brane world-volume aligned parallel to the propagation direction of the wave. We discuss how many degrees of freedom we have in the solutions. We also propose that these solutions can be used to describe higher-dimensional time-dependent "black holes", and discuss their property briefly.Comment: 12 pages, LaTe

Reminder Care System: An Activity-Aware Cross-Device Recommendation System

© 2019, Springer Nature Switzerland AG. Alzheimer’s disease (AD) affects large numbers of elderly people worldwide and represents a significant social and economic burden on society, particularly in relation to the need for long term care facilities. These costs can be reduced by enabling people with AD to live independently at home for a longer time. The use of recommendation systems for the Internet of Things (IoT) in the context of smart homes can contribute to this goal. In this paper, we present the Reminder Care System (RCS), a research prototype of a recommendation system for the IoT for elderly people with cognitive disabilities. RCS exploits daily activities that are captured and learned from IoT devices to provide personalised recommendations. The experimental results indicate that RCS can inform the development of real-world IoT applications