Load Frequency Control: A Deep Multi-Agent Reinforcement Learning Approach

The paradigm shift in energy generation towards microgrid-based architectures is changing the landscape of the energy control structure heavily in distribution systems. More specifically, distributed generation is deployed in the network demanding decentralised control mechanisms to ensure reliable power system operations. In this work, a Multi-Agent Reinforcement Learning approach is proposed to deliver an agentbased solution to implement load frequency control without the need of a centralised authority. Multi-Agent Deep Deterministic Policy Gradient is used to approximate the frequency control at the primary and the secondary levels. Each generation unit is represented as an agent that is modelled by a Recurrent Neural Network. Agents learn the optimal way of acting and interacting with the environment to maximise their long term performance and to balance generation and load, thus restoring frequency. In this paper we prove using three test systems, with two, four and eight generators, that our Multi-Agent Reinforcement Learning approach can efficiently be used to perform frequency control in a decentralised way

Geometrical and spectral study of beta-skeleton graphs

We perform an extensive numerical analysis of beta-skeleton graphs, a particular type of proximity graphs. In beta-skeleton graph (BSG) two vertices are connected if a proximity rule, that depends of the parameter beta is an element of (0, infinity), is satisfied. Moreover, for beta > 1 there exist two different proximity rules, leading to lune-based and circle-based BSGs. First, by computing the average degree of large ensembles of BSGs we detect differences, which increase with the increase of beta, between lune-based and circle-based BSGs. Then, within a random matrix theory (RMT) approach, we explore spectral and eigenvector properties of random BSGs by the use of the nearest-neighbor energy-level spacing distribution and the entropic eigenvector localization length, respectively. The RMT analysis allows us to conclude that a localization transition occurs at beta = 1

Near-infrared K-band Spectroscopic Investigation of Seyfert 2 Nuclei in the CfA and 12 Micron Samples

We present near-infrared K-band slit spectra of the nuclei of 25 Seyfert 2 galaxies in the CfA and 12 micron samples. The strength of the CO absorption features at 2.3-2.4 micron produced by stars is measured in terms of a spectroscopic CO index. A clear anti-correlation between the observed CO index and the nuclear K-L color is present, suggesting that a featureless hot dust continuum heated by an AGN contributes significantly to the observed K-band fluxes in the nuclei of Seyfert 2 galaxies. After correction for this AGN contribution, we estimate nuclear stellar K-band luminosities for all sources, and CO indices for sources with modestly large observed CO indices. The corrected CO indices for 10 (=40%) Seyfert 2 nuclei are found to be as high as those observed in star-forming or elliptical (=spheroidal) galaxies. We combine the K-band data with measurements of the L-band 3.3 micron polycyclic aromatic hydrocarbon (PAH) emission feature, another powerful indicator for star-formation, and find that the 3.3 micron PAH to K-band stellar luminosity ratios are substantially smaller than those of starburst galaxies. Our results suggest that the 3.3 micron PAH emission originates in the putative nuclear starbursts in the dusty tori surrounding the AGNs, because of its high surface brightness, whereas the K-band CO absorption features detected at the nuclei are dominated by old bulge (=spheroid) stars, and thus may not be a powerful indicator for the nuclear starbursts. We see no clear difference in the strength of the CO absorption and PAH emission features between the CfA and 12 micron Seyfert 2s.Comment: 28 pages, 6 figures, accepted for publication in ApJ (10 October 2004, v614 issue

A Double Error Dynamic Asymptote Model of Associative Learning

In this paper a formal model of associative learning is presented which incorporates representational and computational mechanisms that, as a coherent corpus, empower it to make accurate predictions of a wide variety of phenomena that so far have eluded a unified account in learning theory. In particular, the Double Error Dynamic Asymptote (DDA) model introduces: 1) a fully-connected network architecture in which stimuli are represented as temporally clustered elements that associate to each other, so that elements of one cluster engender activity on other clusters, which naturally implements neutral stimuli associations and mediated learning; 2) a predictor error term within the traditional error correction rule (the double error), which reduces the rate of learning for expected predictors; 3) a revaluation associability rate that operates on the assumption that the outcome predictiveness is tracked over time so that prolonged uncertainty is learned, reducing the levels of attention to initially surprising outcomes; and critically 4) a biologically plausible variable asymptote, which encapsulates the principle of Hebbian learning, leading to stronger associations for similar levels of cluster activity. The outputs of a set of simulations of the DDA model are presented along with empirical results from the literature. Finally, the predictive scope of the model is discussed

Cooler and bigger than thought? Planetary host stellar parameters from the InfraRed Flux Method

Effective temperatures and radii for 92 planet-hosting stars as determined from the InfraRed Flux Method (IRFM) are presented and compared with those given by other authors using different approaches. The IRFM temperatures we have derived are systematically lower than those determined from the spectroscopic condition of excitation equilibrium, the mean difference being as large as 110 K. They are, however, consistent with previous IRFM studies and with the colors derived from Kurucz and MARCS model atmospheres. Comparison with direct measurements of stellar diameters for 7 dwarf stars, which approximately cover the range of temperatures of the planet-hosting stars, suggest that the IRFM radii and temperatures are reliable in an absolute scale. A better understanding of the fundamental properties of the stars with planets will be achieved once this discrepancy between the IRFM and the spectroscopic temperature scales is resolved.Comment: 15 pages, 4 figures. Accepted for publication in Ap

Encoding algebraic power series

Algebraic power series are formal power series which satisfy a univariate polynomial equation over the polynomial ring in n variables. This relation determines the series only up to conjugacy. Via the Artin-Mazur theorem and the implicit function theorem it is possible to describe algebraic series completely by a vector of polynomials in n+p variables. This vector will be the code of the series. In the paper, it is then shown how to manipulate algebraic series through their code. In particular, the Weierstrass division and the Grauert-Hironaka-Galligo division will be performed on the level of codes, thus providing a finite algorithm to compute the quotients and the remainder of the division.Comment: 35 page

Continuum discretized BCS approach for weakly bound nuclei

The Bardeen-Cooper-Schrieffer (BCS) formalism is extended by including the single-particle continuum in order to analyse the evolution of pairing in an isotopic chain from stability up to the drip line. We propose a continuum discretized generalized BCS based on single-particle pseudostates (PS). These PS are generated from the diagonalization of the single-particle Hamiltonian within a Transformed Harmonic Oscillator (THO) basis. The consistency of the results versus the size of the basis is studied. The method is applied to neutron rich Oxygen and Carbon isotopes and compared with similar previous works and available experimental data. We make use of the flexibility of the proposed model in order to study the evolution of the occupation of the low-energy continuum when the system becomes weakly bound. We find a larger influence of the non-resonant continuum as long as the Fermi level approaches zero.Comment: 20 pages, 16 figures, to be submitte

Smart Computer Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns new attack sequences and improves its own internal penetration testing logic. As a result, this agent (AgentPen for simplicity) is able to successfully penetrate hosts it has never interacted with before. A computer security administrator using this tool would receive a comprehensive, automated sequence of actions leading to a security breach, highlighting potential vulnerabilities, and reducing the amount of menial tasks a typical penetration tester would need to execute. To achieve autonomy, we apply an unsupervised machine learning algorithm, Q-learning, with an approximator that incorporates a deep neural network architecture. The security audit itself is modelled as a Markov Decision Process in order to test a number of decisionmaking strategies and compare their convergence to optimality. A series of experimental results is presented to show how this approach can be effectively used to automate penetration testing using a scalable, i.e. not exhaustive, and adaptive approach