Smart Computer Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns new attack sequences and improves its own internal penetration testing logic. As a result, this agent (AgentPen for simplicity) is able to successfully penetrate hosts it has never interacted with before. A computer security administrator using this tool would receive a comprehensive, automated sequence of actions leading to a security breach, highlighting potential vulnerabilities, and reducing the amount of menial tasks a typical penetration tester would need to execute. To achieve autonomy, we apply an unsupervised machine learning algorithm, Q-learning, with an approximator that incorporates a deep neural network architecture. The security audit itself is modelled as a Markov Decision Process in order to test a number of decisionmaking strategies and compare their convergence to optimality. A series of experimental results is presented to show how this approach can be effectively used to automate penetration testing using a scalable, i.e. not exhaustive, and adaptive approach

Typical AGN at intermediate redshifts

We study the host galaxies and black holes of typical X-ray selected AGN at intermediate redshifts (z~0.5-1.4). The AGN are selected such that their spectral energy distributions are dominated by stellar emission, i.e., they show a prominent 1.6micron bump thus minimizing the AGN emission contamination. This AGN population comprises approximately 50% of the X-ray selected AGN at these redshifts. AGN reside in the most massive galaxies at the redshifts probed here, with characteristic stellar masses that are intermediate between those of local type 2 AGN and high redshift (z~2) AGN. The inferred black hole masses of typical AGN are similar to those of optically identified quasars at similar redshifts. Since the AGN in our sample are much less luminous than quasars, typical AGN have low Eddington ratios. This suggests that, at least at intermediate redshifts, the cosmic AGN 'downsizing' is due to both a decrease in the characteristic stellar mass of the host galaxies, and less efficient accretion. Finally there is no strong evidence in AGN host galaxies for either highly suppressed star formation, expected if AGN played a role in quenching star formation, or elevated star formation when compared to mass selected galaxies of similar stellar masses and redshifts.Comment: Conference proceedings of the meeting "Observational Evidence for Black Holes" held in Calcutta, Feb 2008. Paper will be published by AI

Near-infrared K-band Spectroscopic Investigation of Seyfert 2 Nuclei in the CfA and 12 Micron Samples

We present near-infrared K-band slit spectra of the nuclei of 25 Seyfert 2 galaxies in the CfA and 12 micron samples. The strength of the CO absorption features at 2.3-2.4 micron produced by stars is measured in terms of a spectroscopic CO index. A clear anti-correlation between the observed CO index and the nuclear K-L color is present, suggesting that a featureless hot dust continuum heated by an AGN contributes significantly to the observed K-band fluxes in the nuclei of Seyfert 2 galaxies. After correction for this AGN contribution, we estimate nuclear stellar K-band luminosities for all sources, and CO indices for sources with modestly large observed CO indices. The corrected CO indices for 10 (=40%) Seyfert 2 nuclei are found to be as high as those observed in star-forming or elliptical (=spheroidal) galaxies. We combine the K-band data with measurements of the L-band 3.3 micron polycyclic aromatic hydrocarbon (PAH) emission feature, another powerful indicator for star-formation, and find that the 3.3 micron PAH to K-band stellar luminosity ratios are substantially smaller than those of starburst galaxies. Our results suggest that the 3.3 micron PAH emission originates in the putative nuclear starbursts in the dusty tori surrounding the AGNs, because of its high surface brightness, whereas the K-band CO absorption features detected at the nuclei are dominated by old bulge (=spheroid) stars, and thus may not be a powerful indicator for the nuclear starbursts. We see no clear difference in the strength of the CO absorption and PAH emission features between the CfA and 12 micron Seyfert 2s.Comment: 28 pages, 6 figures, accepted for publication in ApJ (10 October 2004, v614 issue

Angular Momenta and Spin-Orbit Interaction of Nonparaxial Light in Free Space

We give an exact self-consistent operator description of the spin and orbital angular momenta, position, and spin-orbit interactions of nonparaxial light in free space. Both quantum-operator formalism and classical energy-flow approach are presented. We apply the general theory to symmetric and asymmetric Bessel beams exhibiting spin- and orbital-dependent intensity profiles. The exact wave solutions are clearly interpreted in terms of the Berry phases, quantization of caustics, and Hall effects of light, which can be readily observed experimentally.Comment: 8 pages, 3 figure

Kink fluctuation asymptotics and zero modes

In this paper we propose a refinement of the heat kernel/zeta function treatment of kink quantum fluctuations in scalar field theory, further analyzing the existence and implications of a zero energy fluctuation mode. Improved understanding of the interplay between zero modes and the kink heat kernel expansion delivers asymptotic estimations of one-loop kink mass shifts with remarkably higher precision than previously obtained by means of the standard Gilkey-DeWitt heat kernel expansion.Comment: 21 pages, 8 figures, to be published in The European Physical Journal