Smart Computer Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns new attack sequences and improves its own internal penetration testing logic. As a result, this agent (AgentPen for simplicity) is able to successfully penetrate hosts it has never interacted with before. A computer security administrator using this tool would receive a comprehensive, automated sequence of actions leading to a security breach, highlighting potential vulnerabilities, and reducing the amount of menial tasks a typical penetration tester would need to execute. To achieve autonomy, we apply an unsupervised machine learning algorithm, Q-learning, with an approximator that incorporates a deep neural network architecture. The security audit itself is modelled as a Markov Decision Process in order to test a number of decisionmaking strategies and compare their convergence to optimality. A series of experimental results is presented to show how this approach can be effectively used to automate penetration testing using a scalable, i.e. not exhaustive, and adaptive approach

Coronal heating in multiple magnetic threads

Context. Heating the solar corona to several million degrees requires the conversion of magnetic energy into thermal energy. In this paper, we investigate whether an unstable magnetic thread within a coronal loop can destabilise a neighbouring magnetic thread. Aims. By running a series of simulations, we aim to understand under what conditions the destabilisation of a single magnetic thread can also trigger a release of energy in a nearby thread. Methods. The 3D magnetohydrodynamics code, Lare3d, is used to simulate the temporal evolution of coronal magnetic fields during a kink instability and the subsequent relaxation process. We assume that a coronal magnetic loop consists of non-potential magnetic threads that are initially in an equilibrium state. Results. The non-linear kink instability in one magnetic thread forms a helical current sheet and initiates magnetic reconnection. The current sheet fragments, and magnetic energy is released throughout that thread. We find that, under certain conditions, this event can destabilise a nearby thread, which is a necessary requirement for starting an avalanche of energy release in magnetic threads. Conclusions. It is possible to initiate an energy release in a nearby, non-potential magnetic thread, because the energy released from one unstable magnetic thread can trigger energy release in nearby threads, provided that the nearby structures are close to marginal stability

The Bose-Hubbard model on a triangular lattice with diamond ring-exchange

Ring-exchange interactions have been proposed as a possible mechanism for a Bose-liquid phase at zero temperature, a phase that is compressible with no superfluidity. Using the Stochastic Green Function algorithm (SGF), we study the effect of these interactions for bosons on a two-dimensional triangular lattice. We show that the supersolid phase, that is known to exist in the ground state for a wide range of densities, is rapidly destroyed as the ring-exchange interactions are turned on. We establish the ground-state phase diagram of the system, which is characterized by the absence of the expected Bose-liquid phase.Comment: 6 pages, 10 figure

The role of the intervertebral disc niche on embryonic stem cell differentiation and intervertebral disc regeneration

Poster Session - Orthopedic Regeneration: no. 91DMM 2011 entitled: Re-engineering Regenerative MedicineThe intervertebral disc (IVD) is a highly specialised environment, where cell, water and proteoglycan decrease is strongly associated with disc degeneration, which can lead to back pain. We postulate that the IVD niche is a major factor for IVD cell development and maintenance, and can play a role in progenitor differentiation to become disc cells. The understanding of how stem/progenitor cells develop into IVD cells could potentially facilitate the development of regenerative therapies for degenerated IVD. Here, we aim to introduce human embryonic stem cells (ESCs) into the IVD environment using a model of decellularised bovine disc, and examine how the IVD niche impacts on phenotypic changes in vitro using …postprin

Local Density of the Bose Glass Phase

We study the Bose-Hubbard model in the presence of on-site disorder in the canonical ensemble and conclude that the local density of the Bose glass phase behaves differently at incommensurate filling than it does at commensurate one. Scaling of the superfluid density at incommensurate filling of $\rho=1.1$ and on-site interaction $U=80t$ predicts a superfluid-Bose glass transition at disorder strength of $\Delta_c \approx 30t$. At this filling the local density distribution shows skew behavior with increasing disorder strength. Multifractal analysis also suggests a multifractal behavior resembling that of the Anderson localization. Percolation analysis points to a phase transition of percolating non-integer filled sites around the same value of disorder. Our findings support the scenario of percolating superfluid clusters enhancing Anderson localization near the superfluid-Bose glass transition. On the other hand, the behavior of the commensurate filled system is rather different. Close to the tip of the Mott lobe ($\rho=1, U=22t$) we find a Mott insulator-Bose glass transition at disorder strength of $\Delta_c \approx 16t$. An analysis of the local density distribution shows Gaussian like behavior for a wide range of disorders above and below the transition.Comment: 12 pages, 14 figure

Smart Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

No embargo require

On the Validity of the Tomonaga Luttinger Liquid Relations for the One-dimensional Holstein Model

For the one-dimensional Holstein model, we show that the relations among the scaling exponents of various correlation functions of the Tomonaga Luttinger liquid (LL), while valid in the thermodynamic limit, are significantly modified by finite size corrections. We obtain analytical expressions for these corrections and find that they decrease very slowly with increasing system size. The interpretation of numerical data on finite size lattices in terms of LL theory must therefore take these corrections into account. As an important example, we re-examine the proposed metallic phase of the zero-temperature, half-filled one-dimensional Holstein model without employing the LL relations. In particular, using quantum Monte Carlo calculations, we study the competition between the singlet pairing and charge ordering. Our results do not support the existence of a dominant singlet pairing state.Comment: 7 page

The Fundamental Plane of Gamma-ray Globular Clusters

We have investigated the properties of a group of $\gamma$-ray emitting globular clusters (GCs) which have recently been uncovered in our Galaxy. By correlating the observed $\gamma$-ray luminosities $L_{\gamma}$ with various cluster properties, we probe the origin of the high energy photons from these GCs. We report $L_{\gamma}$ is positively correlated with the encounter rate $\Gamma_{c}$ and the metalicity $\left[{\rm Fe/H}\right]$ which place an intimate link between the gamma-ray emission and the millisecond pulsar population. We also find a tendency that $L_{\gamma}$ increase with the energy densities of the soft photon at the cluster location. Furthermore, the two-dimensional regression analysis suggests that $L_{\gamma}$, soft photon densities, and $\Gamma_{c}$/$\left[{\rm Fe/H}\right]$ possibly span fundamental planes which potentially provide better predictions for the $\gamma$-ray properties of GCs.Comment: 17 pages, 4 figures, 3 tables, published in Ap