Smart Computer Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

A significant challenge in modern computer security is the growing skill gap as intruder capabilities increase, making it necessary to begin automating elements of penetration testing so analysts can contend with the growing number of cyber threats. In this paper, we attempt to assist human analysts by automating a single host penetration attack. To do so, a smart agent performs different attack sequences to find vulnerabilities in a target system. As it does so, it accumulates knowledge, learns new attack sequences and improves its own internal penetration testing logic. As a result, this agent (AgentPen for simplicity) is able to successfully penetrate hosts it has never interacted with before. A computer security administrator using this tool would receive a comprehensive, automated sequence of actions leading to a security breach, highlighting potential vulnerabilities, and reducing the amount of menial tasks a typical penetration tester would need to execute. To achieve autonomy, we apply an unsupervised machine learning algorithm, Q-learning, with an approximator that incorporates a deep neural network architecture. The security audit itself is modelled as a Markov Decision Process in order to test a number of decisionmaking strategies and compare their convergence to optimality. A series of experimental results is presented to show how this approach can be effectively used to automate penetration testing using a scalable, i.e. not exhaustive, and adaptive approach

A machine learning approach for smart computer security audit

This thesis presents a novel application of machine learning technology to automate network security audit and penetration testing processes in particular. A model-free reinforcement learning approach is presented. It is characterized by the absence of the environmental model. The model is derived autonomously by the audit system while acting in the tested computer network. The penetration testing process is specified as a Markov decision process (MDP) without definition of reward and transition functions for every state/action pair. The presented approach includes application of traditional and modified Q-learning algorithms. A traditional Q-learning algorithm learns the action-value function stored in the table, which gives the expected utility of executing a particular action in a particular state of the penetration testing process. The modified Q-learning algorithm differs by incorporation of the state space approximator and representation of the action-value function as a linear combination of features. Two deep architectures of the approximator are presented: autoencoder joint with artificial neural network (ANN) and autoencoder joint with recurrent neural network (RNN). The autoencoder is used to derive the feature set defining audited hosts. ANN is intended to approximate the state space of the audit process based on derived features. RNN is a more advanced version of the approximator and differs by the existence of the additional loop connections from hidden to input layers of the neural network. Such architecture incorporates previously executed actions into new inputs. It gives the opportunity to audit system learn sequences of actions leading to the goal of the audit, which is defined as receiving administrator rights on the host. The model-free reinforcement learning approach based on traditional Q-learning algorithms was also applied to reveal new vulnerabilities, buffer overflow in particular. The penetration testing system showed the ability to discover a string, exploiting potential vulnerability, by learning its formation process on the go. In order to prove the concept and to test the efficiency of an approach, audit tool was developed. Presented results are intended to demonstrate the adaptivity of the approach, performance of the algorithms and deep machine learning architectures. Different sets of hyperparameters are compared graphically to test the ability of convergence to the optimal action policy. An action policy is a sequence of actions, leading to the audit goal (getting admin rights on the remote host). The testing environment is also presented. It consists of 80+ virtual machines based on a vSphere virtualization platform. This combination of hosts represents a typical corporate network with Users segment, Demilitarized zone (DMZ) and external segment (Internet). The network has typical corporate services available: web server, mail server, file server, SSH, SQL server. During the testing process, the audit system acts as an attacker from the Internet

Smart Security Audit: Reinforcement Learning with a Deep Neural Network Approximator

No embargo require

The Additional Line Component within the Iron K\alpha Profile in MCG-6-30-15: Evidence for Blob Ejection?

The EPIC data of MCG -6-30-15 observed by XMM-Newton were analyzed for the complexities of the iron K-alpha line. Here we report that the additional line component (ALC) at 6.9 keV undoubtedly appears within the broad iron Kalpha; line profile at the high state, whereas it disappears at the low state. These state-dependent behaviors exclude several possible origins and suggest an origin of the ALC in matter being ejected from the vicinity of the black hole. At the low state, the newborn blob ejected from the accretion disk is so Thomson-thick that hard X-rays are blocked from ionizing the old blobs, leading to the disappearance of the ALC. When the blob becomes Thomson-thin as a result of expansion, the hard X-ray will penetrate it and ionize the old ones, emitting the ALC at the high state. The blob ejection is the key to switching the ALC on or off.Comment: 6 pages, 4 Figure

Testing Comptonizing coronae on a long BeppoSAX observation of the Seyfert 1 galaxy NGC 5548

We test accurate models of Comptonization spectra over the high quality data of the BeppoSAX long look at NGC 5548, allowing for different geometries of the scattering region, different temperatures of the input soft photon field and different viewing angles. We find that the BeppoSAX data are well represented by a plane parallel or hemispherical corona viewed at an inclination angle of 30$^{\circ}$. For both geometries the best fit temperature of the soft photons is close to 15$^{+3}_{-9}$ eV. The corresponding best fit values of the hot plasma temperature and optical depth are $kT_{\rm e}\simeq$ 250--260 keV and $\tau\simeq$ 0.16--0.37 for the slab and hemisphere respectively. These values are substantially different from those derived fitting the data with a power-law + cut off approximation to the Comptonization component (kT_{\rm e}\lta 60 keV, $\tau\simeq$ 2.4). This is due to the fact that accurate Comptonization spectra in anisotropic geometries show "intrinsic" curvature which reduces the necessity of a high energy cut-off. The Comptonization parameter derived for the slab model {is} larger than predicted for a two phase plane parallel corona in energy balance, suggesting that a more ``photon-starved'' geometry is necessary. The spectral softening detected during a flare which occurred in the central part of the observation corresponds to a decrease of the Comptonization parameter, probably associated with an increase of the soft photon luminosity, the {hard} photon luminosity remaining constant.Comment: 36 pages, 9 figures, accepted by Ap

Near-threshold Photoproduction of Phi Mesons from Deuterium

We report the first measurement of the differential cross section on $\phi$-meson photoproduction from deuterium near the production threshold for a proton using the CLAS detector and a tagged-photon beam in Hall B at Jefferson Lab. The measurement was carried out by a triple coincidence detection of a proton, $K^+$ and $K^-$ near the theoretical production threshold of 1.57 GeV. The extracted differential cross sections $\frac{d\sigma}{dt}$ for the initial photon energy from 1.65-1.75 GeV are consistent with predictions based on a quasifree mechanism. This experiment establishes a baseline for a future experimental search for an exotic $\phi$-N bound state from heavier nuclear targets utilizing subthreshold/near-threshold production of $\phi$ mesons

A comparison of forward and backward pp pair knockout in 3He(e,e'pp)n

Measuring nucleon-nucleon Short Range Correlations (SRC) has been a goal of the nuclear physics community for many years. They are an important part of the nuclear wavefunction, accounting for almost all of the high-momentum strength. They are closely related to the EMC effect. While their overall probability has been measured, measuring their momentum distributions is more difficult. In order to determine the best configuration for studying SRC momentum distributions, we measured the $^3$He$(e,e'pp)n$ reaction, looking at events with high momentum protons ($p_p > 0.35$ GeV/c) and a low momentum neutron ($p_n< 0.2$ GeV/c). We examined two angular configurations: either both protons emitted forward or one proton emitted forward and one backward (with respect to the momentum transfer, $\vec q$). The measured relative momentum distribution of the events with one forward and one backward proton was much closer to the calculated initial-state $pp$ relative momentum distribution, indicating that this is the preferred configuration for measuring SRC.Comment: 8 pages, 9 figures, submitted to Phys Rev C. Version 2 incorporates minor corrections in response to referee comment

Comment on the narrow structure reported by Amaryan et al

The CLAS Collaboration provides a comment on the physics interpretation of the results presented in a paper published by M. Amaryan et al. regarding the possible observation of a narrow structure in the mass spectrum of a photoproduction experiment.Comment: to be published in Physical Review

Differential cross sections and recoil polarizations for the reaction gamma p -> K+ Sigma0

High-statistics measurements of differential cross sections and recoil polarizations for the reaction $\gamma p \rightarrow K^+ \Sigma^0$ have been obtained using the CLAS detector at Jefferson Lab. We cover center-of-mass energies ($\sqrt{s}$) from 1.69 to 2.84 GeV, with an extensive coverage in the $K^+$ production angle. Independent measurements were made using the $K^{+}p\pi^{-}$($\gamma$) and $K^{+}p$($\pi^-, \gamma$) final-state topologies, and were found to exhibit good agreement. Our differential cross sections show good agreement with earlier CLAS, SAPHIR and LEPS results, while offering better statistical precision and a 300-MeV increase in $\sqrt{s}$ coverage. Above $\sqrt{s} \approx 2.5$ GeV, $t$- and $u$-channel Regge scaling behavior can be seen at forward- and backward-angles, respectively. Our recoil polarization ($P_\Sigma$) measurements represent a substantial increase in kinematic coverage and enhanced precision over previous world data. At forward angles we find that $P_\Sigma$ is of the same magnitude but opposite sign as $P_\Lambda$, in agreement with the static SU(6) quark model prediction of $P_\Sigma \approx -P_\Lambda$. This expectation is violated in some mid- and backward-angle kinematic regimes, where $P_\Sigma$ and $P_\Lambda$ are of similar magnitudes but also have the same signs. In conjunction with several other meson photoproduction results recently published by CLAS, the present data will help constrain the partial wave analyses being performed to search for missing baryon resonances.Comment: 23 pages, 17 figure

Reference architectures, platforms, and pilots for european smart and healthy living—analysis and comparison

Motivated by the aging trend, much effort is being invested into implementing ICT (Information and Communications Technology)-enabled systems to provide a better quality of life and support the independent living of older people. As a result, many systems, often labeled as eHealth or AAL (Ambient/Active Assisted Living), were developed over the years. In creating such systems, which very often serve various needs, different architectures have emerged. This work focuses on analyzing and comparing the work and architectures from seven (six of which are in progress) EU-funded healthcare projects, with a total budget of 126MEUR in which we participate. After establishing the theoretical foundation by defining core concepts, we give a brief background on architectures in eHealth and AAL. We elaborate on the chosen analysis method based on three established healthcare and AAL taxonomies we identified by performing a literature survey and the selected Reference Architecture Model (RAM). Since there is no standard way of describing architectures in the eHealth and AAL domain, we conducted the online survey during August and September 2020 and identified CREATE-IoT 3D RAM as the most appropriate option. We present a classification of selected projects based on established taxonomies and map projects’ architectures to CREATE-IoT 3D RAM, which we also propose as standard RAM for future digital healthcare and AAL projects. During our analysis, we identify the most common types of assistance: communication support, reminders, monitoring, and guidance to address health and communication issues. We conclude that proper ecosystems are critical for lowering entry barriers and facilitating sustainable solutions for smart and healthy living