Load Frequency Control: A Deep Multi-Agent Reinforcement Learning Approach

The paradigm shift in energy generation towards microgrid-based architectures is changing the landscape of the energy control structure heavily in distribution systems. More specifically, distributed generation is deployed in the network demanding decentralised control mechanisms to ensure reliable power system operations. In this work, a Multi-Agent Reinforcement Learning approach is proposed to deliver an agentbased solution to implement load frequency control without the need of a centralised authority. Multi-Agent Deep Deterministic Policy Gradient is used to approximate the frequency control at the primary and the secondary levels. Each generation unit is represented as an agent that is modelled by a Recurrent Neural Network. Agents learn the optimal way of acting and interacting with the environment to maximise their long term performance and to balance generation and load, thus restoring frequency. In this paper we prove using three test systems, with two, four and eight generators, that our Multi-Agent Reinforcement Learning approach can efficiently be used to perform frequency control in a decentralised way

Towards active microfluidics: Interface turbulence in thin liquid films with floating molecular machines

Thin liquid films with floating active protein machines are considered. Cyclic mechanical motions within the machines, representing microscopic swimmers, lead to molecular propulsion forces applied to the air-liquid interface. We show that, when the rate of energy supply to the machines exceeds a threshold, the flat interface becomes linearly unstable. As the result of this instability, the regime of interface turbulence, characterized by irregular traveling waves and propagating machine clusters, is established. Numerical investigations of this nonlinear regime are performed. Conditions for the experimental observation of the instability are discussed.Comment: 9 pages, 8 figures, RevTeX, submitted to Physical Review

Typical AGN at intermediate redshifts

We study the host galaxies and black holes of typical X-ray selected AGN at intermediate redshifts (z~0.5-1.4). The AGN are selected such that their spectral energy distributions are dominated by stellar emission, i.e., they show a prominent 1.6micron bump thus minimizing the AGN emission contamination. This AGN population comprises approximately 50% of the X-ray selected AGN at these redshifts. AGN reside in the most massive galaxies at the redshifts probed here, with characteristic stellar masses that are intermediate between those of local type 2 AGN and high redshift (z~2) AGN. The inferred black hole masses of typical AGN are similar to those of optically identified quasars at similar redshifts. Since the AGN in our sample are much less luminous than quasars, typical AGN have low Eddington ratios. This suggests that, at least at intermediate redshifts, the cosmic AGN 'downsizing' is due to both a decrease in the characteristic stellar mass of the host galaxies, and less efficient accretion. Finally there is no strong evidence in AGN host galaxies for either highly suppressed star formation, expected if AGN played a role in quenching star formation, or elevated star formation when compared to mass selected galaxies of similar stellar masses and redshifts.Comment: Conference proceedings of the meeting "Observational Evidence for Black Holes" held in Calcutta, Feb 2008. Paper will be published by AI

Reply to Comment on "Magnetization Process of Single Molecule Magnets at Low Temperatures"

This is the reply to a Comment by I.S.Tupitsyn and P.C.E. Stamp (PRL v92,119701 (2004)) on a letter of ours (J.F.Fernandez and J.J.Alonso, PRL v91, 047202 (2003)).Comment: 2 LaTeX pages, 1 eps figure. Submitted to PRL on 20 October 200

The thermodynamics of metabolism, cardiovascular performance and exercise, in health and diabetes: The objective of clinical markers

Extensive experience in UK National Health Service metabolic syndrome/type 2 diabetes clinics highlights the need for convenient clinical marker(s) which can be readily used to indicate the success or otherwise of alternative therapies. In this paper we study the metabolic context of the healthy and diseased states, which points to the haemodynamics being a possible key in identifying candidate markers. Human metabolism relates to two elemental thermodynamic systems, the individual cell and the human body in its entirety. The fundamental laws of thermodynamics apply to humans, animals, and their individual cells for both healthy and diseased conditions. as they are to classic heat engines. In compliance with the second law enhanced levels of heat are generated under exercise, heat itself being another factor modulating the cardiovascular response to physical exercise. Nutrients and oxygen uptake occurs via the digestive system and lungs, respectively, leading to ATP production by the established metabolic pathways: this is controlled by insulin. These are then delivered to the cells via the haemodynamic system to satisfy local metabolic need. The supply and demand of oxygen are finely regulated, in part, via oxygen-dependent release of ATP from the circulating erythrocytes. Energy supply and demand are regulated to sustain muscle activity resulting in the body’s output of measurable thermodynamic work—i.e. exercise. Recently a dynamic pathway model allowing quantification of ATP release from the erythrocytes and its contribution to oxygen supply regulation has been published. However, metabolic uptake is well known to be greatly affected by disease such as the highly prevalent diabetes type 2 with insulin resistance and beta cell dysfunction having mechanistic roles. In 2010, over 25% of residents above 65 in the USA had diabetes 2. The complexity of the metabolic pathways means that monitoring of patient-specific treatment would be beneficial from a diabetic marker which may be haemodynamic-related and traceable via the local fluid dynamics

Wheel–rail contact: experimental study of the creep forces–creepage relationships

The wheel–rail contact problem plays an important role in the simulation methods used to solve railway dynamics problems. As a consequence, many different mathematical models have been developed to calculate wheel–rail contact forces. However, most of them tackle this problem purely from a theoretical point of view and need to be experimentally validated. Such validation could also reveal the influence of certain parameters not taken into account in the mathematical developments. This paper presents the steps followed in building a scaled test-bench to experimentally characterise the wheel–rail contact problem. The results of the longitudinal contact force as a function of the longitudinal creepage are obtained and the divergences with respect to Kalker's simplified theory are analysed. The influence of lateral creepage, angular velocity and certain contaminants such as cutting fluid or high positive friction modifier is also discussed

Isospin breaking in the nucleon mass and the sensitivity of β decays to new physics

We discuss the consequences of the approximate conservation of the vector and axial currents for the hadronic matrix elements appearing in β decay if nonstandard interactions are present. In particular, the isovector (pseudo)scalar charge gS(P) of the nucleon can be related to the difference (sum) of the nucleon masses in the absence of electromagnetic effects. Using recent determinations of these quantities from phenomenological and lattice QCD studies we obtain the accurate values gS=1.02(11) and gP=349(9) in the modified minimal subtraction scheme at μ=2  GeV. The consequences for searches of nonstandard scalar interactions in nuclear β decays are studied, finding for the corresponding Wilson coefficient εS=0.0012(24) at 90% C.L., which is significantly more stringent than current LHC bounds and previous low-energy bounds using less precise gS values. We argue that our results could be rapidly improved with updated computations and the direct calculation of certain ratios in lattice QCD. Finally, we discuss the pion-pole enhancement of gP, which makes β decays much more sensitive to nonstandard pseudoscalar interactions than previously thought