Reinforcement learning for efficient network penetration testing

Penetration testing (also known as pentesting or PT) is a common practice for actively assessing the defenses of a computer network by planning and executing all possible attacks to discover and exploit existing vulnerabilities. Current penetration testing methods are increasingly becoming non-standard, composite and resource-consuming despite the use of evolving tools. In this paper, we propose and evaluate an AI-based pentesting system which makes use of machine learning techniques, namely reinforcement learning (RL) to learn and reproduce average and complex pentesting activities. The proposed system is named Intelligent Automated Penetration Testing System (IAPTS) consisting of a module that integrates with industrial PT frameworks to enable them to capture information, learn from experience, and reproduce tests in future similar testing cases. IAPTS aims to save human resources while producing much-enhanced results in terms of time consumption, reliability and frequency of testing. IAPTS takes the approach of modeling PT environments and tasks as a partially observed Markov decision process (POMDP) problem which is solved by POMDP-solver. Although the scope of this paper is limited to network infrastructures PT planning and not the entire practice, the obtained results support the hypothesis that RL can enhance PT beyond the capabilities of any human PT expert in terms of time consumed, covered attacking vectors, accuracy and reliability of the outputs. In addition, this work tackles the complex problem of expertise capturing and re-use by allowing the IAPTS learning module to store and re-use PT policies in the same way that a human PT expert would learn but in a more efficient way

Standardization in cyber-physical systems: the ARUM case

Cyber-physical systems concept supports the realization of the Industrie 4.0 vision towards the computerization of traditional industries, aiming to achieve intelligent and reconfigurable factories. Standardization assumes a critical role in the industrial adoption of cyber-physical systems, namely in the integration of legacy systems as well as the smooth migration from existing running systems to the new ones. This paper analyses some existing standards in related fields and presents identified limitations and efforts for a wider acceptance of such systems by industry. A special attention is devoted to the efforts to develop a standard-compliant service-oriented multi-agent system solution within the ARUM project.info:eu-repo/semantics/publishedVersio

Helping humans and agents avoid undesirable consequences with models of intervention

2021 Fall.Includes bibliographical references.When working in an unfamiliar online environment, it can be helpful to have an observer that can intervene and guide a user toward a desirable outcome while avoiding undesirable outcomes or frustration. The Intervention Problem is deciding when to intervene in order to help a user. The Intervention Problem is similar to, but distinct from, Plan Recognition because the observer must not only recognize the intended goals of a user but also when to intervene to help the user when necessary. In this dissertation, we formalize a family of intervention problems to address two sub-problems: (1) The Intervention Recognition Problem, and (2) The Intervention Recovery Problem. The Intervention Recognition Problem views the environment as a state transition system where an agent (or a human user), in order to achieve a desirable outcome, executes actions that change the environment from one state to the next. Some states in the environment are undesirable and the user does not have the ability to recognize them and the intervening agent wants to help the user in the environment avoid the undesirable state. In this dissertation, we model the environment as a classical planning problem and discuss three intervention models to address the Intervention Recognition Problem. The three models address different dimensions of the Intervention Recognition Problem, specifically the actors in the environment, information hidden from the intervening agent, type of observations and noise in the observations. The first model: Intervention by Recognizing Actions Enabling Multiple Undesirable Consequences, is motivated by a study where we observed how home computer users practice cyber-security and take action to unwittingly put their online safety at risk. The model is defined for an environment where three agents: the user, the attacker and the intervening agent are present. The intervening agent helps the user reach a desirable goal that is hidden from the intervening agent by recognizing critical actions that enable multiple undesirable consequences. We view the problem of recognizing critical actions as a multi-factor decision problem of three domain-independent metrics: certainty, timeliness and desirability. The three metrics simulate the trade-off between the safety and freedom of the observed agent when selecting critical actions to intervene. The second model: Intervention as Classical Planning, we model scenarios where the intervening agent observes a user and a competitor attempting to achieve different goals in the same environment. A key difference in this model compared to the first model is that the intervening agent is aware of the user's desirable goal and the undesirable state. The intervening agent exploits the classical planning representation of the environment and uses automated planning to project the possible outcomes in the environment exactly and approximately. To recognize when intervention is required, the observer analyzes the plan suffixes leading to the user's desirable goal and the undesirable state and learns the differences between the plans that achieve the desirable goal and plans that achieve the undesirable state using machine learning. Similar to the first model, learning the differences between the safe and unsafe plans allows the intervening agent to balance specific actions with those that are necessary for the user to allow some freedom. The third model: Human-aware Intervention, we assume that the user is a human solving a cognitively engaging planning task. When human users plan, unlike an automated planner, they do not have the ability to use heuristics to search for the best solution. They often make mistakes and spend time exploring the search space of the planning problem. The complication this adds to the Intervention Recognition Problem is that deciding to intervene by analyzing plan suffixes generated by an automated planner is no longer feasible. Using a cognitively engaging puzzle solving task (Rush Hour) we study how human users solve the puzzle as a planning task and develop the Human-aware Intervention model combining automated planning and machine learning. The intervening agent uses a domain specific feature set more appropriate for human behavior to decide in real time whether to intervene the human user. Our experiments using the benchmark planning domains and human subject studies show that the three intervention recognition models out performs existing plan recognition algorithms in predicting when intervention is required. Our solution to address the Intervention Recovery Problem goes beyond the typical preventative measures to help the human user recover from intervention. We propose the Interactive Human-aware Intervention where a human user solves a cognitively engaging planning task with the assistance of an agent that implements the Human-aware Intervention. The Interactive Human-aware Intervention is different from typical preventive measures where the agent executes actions to modify the domain such that the undesirable plan can not progress (e.g., block an action). Our approach interactively guides the human user toward the solution to the planning task by revealing information about the remaining planning task. We evaluate the Interactive Human-aware Intervention using both subjective and objective measures in a human subject study

Кибербезопасность в образовательных сетях

The paper discusses the possible impact of digital space on a human, as well as human-related directions in cyber-security analysis in the education: levels of cyber-security, social engineering role in cyber-security of education, “cognitive vaccination”. “A Human” is considered in general meaning, mainly as a learner. The analysis is provided on the basis of experience of hybrid war in Ukraine that have demonstrated the change of the target of military operations from military personnel and critical infrastructure to a human in general. Young people are the vulnerable group that can be the main goal of cognitive operations in long-term perspective, and they are the weakest link of the System.У статті обговорюється можливий вплив цифрового простору на людину, а також пов'язані з людиною напрямки кібербезпеки в освіті: рівні кібербезпеки, роль соціального інжинірингу в кібербезпеці освіти, «когнітивна вакцинація». «Людина» розглядається в загальному значенні, головним чином як та, що навчається. Аналіз надається на основі досвіду гібридної війни в Україні, яка продемонструвала зміну цілей військових операцій з військовослужбовців та критичної інфраструктури на людину загалом. Молодь - це вразлива група, яка може бути основною метою таких операцій в довгостроковій перспективі, і вони є найслабшою ланкою системи.В документе обсуждается возможное влияние цифрового пространства на человека, а также связанные с ним направления в анализе кибербезопасности в образовании: уровни кибербезопасности, роль социальной инженерии в кибербезопасности образования, «когнитивная вакцинация». «Человек» рассматривается в общем смысле, в основном как ученик. Анализ представлен на основе опыта гибридной войны в Украине, которая продемонстрировала изменение цели военных действий с военного персонала и критической инфраструктуры на человека в целом. Молодые люди являются уязвимой группой, которая может быть главной целью когнитивных операций в долгосрочной перспективе, и они являются самым слабым звеном Систем

MODELLING VIRTUAL ENVIRONMENT FOR ADVANCED NAVAL SIMULATION

This thesis proposes a new virtual simulation environment designed as element of an interoperable federation of simulator to support the investigation of complex scenarios over the Extended Maritime Framework (EMF). Extended Maritime Framework is six spaces environment (Underwater, Water surface, Ground, Air, Space, and Cyberspace) where parties involved in Joint Naval Operations act. The amount of unmanned vehicles involved in the simulation arise the importance of the Communication modelling, thus the relevance of Cyberspace. The research is applied to complex cases (one applied to deep waters and one to coast and littoral protection) as examples to validate this approach; these cases involve different kind of traditional assets (e.g. satellites, helicopters, ships, submarines, underwater sensor infrastructure, etc.) interact dynamically and collaborate with new autonomous systems (i.e. AUV, Gliders, USV and UAV). The use of virtual simulation is devoted to support validation of new concepts and investigation of collaborative engineering solutions by providing a virtual representation of the current situation; this approach support the creation of dynamic interoperable immersive framework that could support training for Man in the Loop, education and tactical decision introducing the Man on the Loop concepts. The research and development of the Autonomous Underwater Vehicles requires continuous testing so a time effective approach can result a very useful tool. In this context the simulation can be useful to better understand the behaviour of Unmanned Vehicles and to avoid useless experimentations and their costs finding problems before doing them. This research project proposes the creation of a virtual environment with the aim to see and understand a Joint Naval Scenario. The study will be focusing especially on the integration of Autonomous Systems with traditional assets; the proposed simulation deals especially with collaborative operation involving different types of Autonomous Underwater Vehicles (AUV), Unmanned Surface Vehicles (USV) and UAV (Unmanned Aerial Vehicle). The author develops an interoperable virtual simulation devoted to present the overall situation for supervision considering also the sensor capabilities, communications and mission effectiveness that results dependent of the different asset interaction over a complex heterogeneous network. The aim of this research is to develop a flexible virtual simulation solution as crucial element of an HLA federation able to address the complexity of Extended Maritime Framework (EMF). Indeed this new generation of marine interoperable simulation is a strategic advantage for investigating the problems related to the operational use of autonomous systems and to finding new ways to use them respect to different scenarios. The research deal with the creation of two scenarios, one related to military operations and another one on coastal and littoral protection where the virtual simulation propose the overall situation and allows to navigate into the virtual world considering the complex physics affecting movement, perception, interaction and communication. By this approach, it becomes evident the capability to identify, by experimental analysis within the virtual world, the new solutions in terms of engineering and technological configuration of the different systems and vehicles as well as new operational models and tactics to address the specific mission environment. The case of study is a maritime scenario with a representation of heterogeneous network frameworks that involves multiple vehicles both naval and aerial including AUVs, USVs, gliders, helicopter, ships, submarines, satellite, buoys and sensors. For the sake of clarity aerial communications will be represented divided from underwater ones. A connection point for the latter will be set on the keel line of surface vessels representing communication happening via acoustic modem. To represent limits in underwater communications, underwater signals have been considerably slowed down in order to have a more realistic comparison with aerial ones. A maximum communication distance is set, beyond which no communication can take place. To ensure interoperability the HLA Standard (IEEE 1516 evolved) is adopted to federate other simulators so to allow its extensibility for other case studies. Two different scenarios are modelled in 3D visualization: Open Water and Port Protection. The first one aims to simulate interactions between traditional assets in Extended Maritime Framework (EMF) such as satellite, navy ships, submarines, NATO Research Vessels (NRVs), helicopters, with new generation unmanned assets as AUV, Gliders, UAV, USV and the mutual advantage the subjects involved in the scenario can have; in other word, the increase in persistence, interoperability and efficacy. The second scenario models the behaviour of unmanned assets, an AUV and an USV, patrolling a harbour to find possible threats. This aims to develop an algorithm to lead patrolling path toward an optimum, guaranteeing a high probability of success in the safest way reducing human involvement in the scenario. End users of the simulation face a graphical 3D representation of the scenario where assets would be represented. He can moves in the scenario through a Free Camera in Graphic User Interface (GUI) configured to entitle users to move around the scene and observe the 3D sea scenario. In this way, players are able to move freely in the synthetic environment in order to choose the best perspective of the scene. The work is intended to provide a valid tool to evaluate the defencelessness of on-shore and offshore critical infrastructures that could includes the use of new technologies to take care of security best and preserve themselves against disasters both on economical and environmental ones