Performance Metrics for Network Intrusion Systems

Intrusion systems have been the subject of considerable research during the past 33 years, since the original work of Anderson. Much has been published attempting to improve their performance using advanced data processing techniques including neural nets, statistical pattern recognition and genetic algorithms. Whilst some significant improvements have been achieved they are often the result of assumptions that are difficult to justify and comparing performance between different research groups is difficult. The thesis develops a new approach to defining performance focussed on comparing intrusion systems and technologies. A new taxonomy is proposed in which the type of output and the data scale over which an intrusion system operates is used for classification. The inconsistencies and inadequacies of existing definitions of detection are examined and five new intrusion levels are proposed from analogy with other detection-based technologies. These levels are known as detection, recognition, identification, confirmation and prosecution, each representing an increase in the information output from, and functionality of, the intrusion system. These levels are contrasted over four physical data scales, from application/host through to enterprise networks, introducing and developing the concept of a footprint as a pictorial representation of the scope of an intrusion system. An intrusion is now defined as “an activity that leads to the violation of the security policy of a computer system”. Five different intrusion technologies are illustrated using the footprint with current challenges also shown to stimulate further research. Integrity in the presence of mixed trust data streams at the highest intrusion level is identified as particularly challenging. Two metrics new to intrusion systems are defined to quantify performance and further aid comparison. Sensitivity is introduced to define basic detectability of an attack in terms of a single parameter, rather than the usual four currently in use. Selectivity is used to describe the ability of an intrusion system to discriminate between attack types. These metrics are quantified experimentally for network intrusion using the DARPA 1999 dataset and SNORT. Only nine of the 58 attack types present were detected with sensitivities in excess of 12dB indicating that detection performance of the attack types present in this dataset remains a challenge. The measured selectivity was also poor indicting that only three of the attack types could be confidently distinguished. The highest value of selectivity was 3.52, significantly lower than the theoretical limit of 5.83 for the evaluated system. Options for improving selectivity and sensitivity through additional measurements are examined.Stochastic Systems Lt

Data Analytics and Knowledge Discovery for Root Cause Analysis in LTE Self-Organizing Networks.

En las últimas décadas, las redes móviles han cobrado cada vez más importancia en el mundo de las telecomunicaciones. Lo que empezó con el objetivo de dar un servicio de voz a nivel global, ha tomado recientemente la direcci\'on de convertirse en un servicio casi exclusivo de datos en banda ancha, dando lugar a la red LTE. Como consecuencia de la continua aparición de nuevos servicios, los usuarios demandan cada vez redes con mayor capacidad, mejor calidad de servicio y a precios menores. Esto provoca una dura competición entre los operadores, que necesitan reducir costes y cortes en el servicio causados por trabajos de mejora o problemas. Para este fin, las redes autoorganizadas SON (Self-Organizing Network) proporcionan herramientas para la automatización de las tareas de operación y mantenimiento, haciéndolas más rápidas y mantenibles por pequeños equipos de expertos. Las funcionalidades SON se dividen en tres grupos principales: autoconfiguración (Self-configuration, los elementos nuevos se configuran de forma automática), autooptimización (Self-optimization, los parámetros de la red se actualizan de forma automática para dar el mejor servicio posible) y autocuración (Self-healing, la red se recupera automáticamente de problemas). En el ambiente competitivo de las redes móviles, los cortes de servicio provocados por problemas en la red causan un gran coste de oportunidad, dado que afectan a la experiencia de usuario. Self-healing es la función SON que se encarga de la automatización de la resolución de problemas. El objetivo principal de Self-healing es reducir el tiempo que dura la resolución de un problema y liberar a los expertos de tareas repetitivas. Self-healing tiene cuatro procesos principales: detección (identificar que los usuarios tienen problemas en una celda), compensación (redirigir los recursos de la red para cubrir a los usuarios afectados), diagnosis (encontrar la causa de dichos problemas) y recuperación (realizar las acciones necesarias para devolver los elementos afectados a su operación normal). De todas las funcionalidades SON, Self-healing (especialmente la función de diagnosis) es la que constituye el mayor desafío, dada su complejidad, y por tanto, es la que menos se ha desarrollado. No hay sistemas comerciales que hagan una diagnosis automática con la suficiente fiabilidad para convencer a los operadores de red. Esta falta de desarrollo se debe a la ausencia de información necesaria para el diseño de sistemas de diagnosis automática. No hay bases de datos que recojan datos de rendimiento de la red en casos problemáticos y los etiqueten con la causa del problema que puedan ser estudiados para encontrar los mejores algoritmos de tratamiento de datos. A pesar de esto, se han propuesto soluciones basadas en la Inteligencia Artificial (IA) para la diagnosis, tomando como punto de partida la limitada información disponible. Estos algoritmos a su vez necesitan ser entrenados con datos realistas. Nuevamente, dado que no hay bases de datos de problemas reales, los datos de entrenamiento suelen ser extraídos de simulaciones, lo cual les quita realismo. La causa de la falta de datos es que los expertos en resolución de problemas no registran los casos conforme los van solucionando. En el ambiente competitivo en el que trabajan, su tiempo es un recurso limitado que debe ser utilizado para resolver problemas y no para registrarlos. En el caso en que tales bases de datos fueran recogidas, un aspecto importante a tener en cuenta es que el volumen, variabilidad y velocidad de generación de los datos hacen que éste sea considerado un problema Big Data. El problema principal de los sistemas de diagnosis automática es la falta de conocimiento experto. Para resolver esto, el conocimiento experto debe convertirse a un formato utilizable. Este proceso se conoce como adquisición del conocimiento. Hay dos aproximaciones a la adquisición del conocimiento: manual(a través de entrevistas o con la implicación de los expertos en el desarrollo) o a través de la analítica de datos (minería de datos en bases de datos que contienen el resultado del trabajo de los expertos). Esta tesis estudia la aproximación de la analítica de datos, utilizando las técnicas KDD (Knowledge Discovery and Datamining). Para que esta aproximación pueda ser utilizada, se requiere la existencia de una base de datos de casos reales de fallo, lo cual es un gran desafío. La visión general de esta tesis es una plataforma en la que cada vez que un experto diagnostica un problema en la red, éste puede reportarlo con un esfuerzo mínimo y almacenarlo en el sistema. La parte central de este sistema es un algoritmo de diagnosis (en esta tesis un controlador de lógica borrosa) que evoluciona y mejora aprendiendo de cada nuevo ejemplo, hasta llegar al punto en el que los expertos pueden confiar en su precisión para los problemas más comunes. Cada vez que surja un nuevo problema, se añadirá a la base de datos del sistema, incrementando así aún más su potencia. El fin es liberar a los expertos de tareas repetitivas, de modo que puedan dedicar su tiempo a desafíos cuya resolución sea más gratificante. Por tanto, el primer objetivo de esta tesis es la colección de una base de datos de casos reales de fallos. Para ello, se diseña una interfaz de usuario para la recolección de datos teniendo en cuenta como requisito prioritario la facilidad de uso. Una vez que se dispone de datos recogidos, se analizarán para comprender mejor sus propiedades y obtener la información necesaria para el diseño de los algoritmos de analítica de datos. Otro objetivo de esta tesis es la creación de un modelo de fallos de LTE, encontrando las relaciones entre el rendimiento de la red y la ocurrencia de los problemas. La adquisición del conocimiento se realiza mediante la aplicación de algoritmos de analítica sobre los datos recogidos. Se diseña un proceso KDD que extrae los parámetros de un controlador de lógica borrosa y se aplica sobre la base de datos recogida. Finalmente, esta tesis también tiene como objetivo realizar un análisis de los aspectos Big Data de las funciones Self-healing, y tenerlos en cuenta a la hora de diseñar los algoritmos

A Bayesian approach to Hybrid Choice models

Tableau d’honneur de la Faculté des études supérieures et postdoctorales, 2010-2011Les modèles microéconométriques de choix discrets ont pour but d’expliquer le processus du choix individuel des consommateurs parmi un ensemble limité et exhaustive d’options mutuellement exclusives. Les modèles dits de choix hybrides sont une généralisation des modèles de choix discrets standard, où des modèles indépendants plus sophistiqués sont considérés simultanément. Dans cette thèse des techniques d’estimation simultanée sont analysées et appliquées pour un modèle de choix hybride qui, sous la forme d’un système complexe d’équations structurelles généralisées, intègre à la fois des choix discrets et des variables latentes en tant que facteurs explicatifs des processus décisionnels. Ce qui motive l’étude de ce genre de modèles est que pour comprendre le processus du choix il faut incorporer des attitudes, des perceptions et des attributs qualitatifs à l’intérieur de modèles décisionnels économiques conventionnels, tout en prenant ce qui dit la recherche en sciences cognitives ainsi qu’en psychologie sociale. Quoique l’estimation du système d’équations d’un modèle de choix hybride requière l’évaluation d’intégrales multidimensionnelles complexes, on résoudre empiriquement ce problème en applicant la méthode du maximum de vraisemblance simulée. Ensuite on dérive une procédure d’échantillonnage de Gibbs pour l’estimation simultanée bayésienne du modèle qui offre des estimateurs convergents et efficaces. Ceci devient une méthode plus avantageuse comparativement aux méthodes classiques dans un cadre analytique avec un grand nombre de variables latentes. En effet, en vertu de l’approche bayésienne il suffit de considérer des régressions ordinaires pour les variables latentes. Par ailleurs, dériver les intervalles de confiance bayésiennes pour les parts de marché ainsi que pour des dispositions à payer devient trivial. De par sa grande géneralité, le modèle de choix hybride est capable de s’adapter à des situations pratiques. En particulier, la réponse des consommateurs suite à l’innovation technologique est analysée. Par exemple, on étudie les préférences pro-environnementales dans un modèle économique des décisions d’achat de véhicules verts selon lequel les consommateurs soucieux de l’environnement sont prêts à payer davantage pour des véhicules à faibles émissions, en dépit des inconvénients potentiels. En outre, en utilisant un noyau probit et des indicateurs dichotomiques on montre que des connaissances préalables ainsi que des attitudes positives envers l’adoption de nouvelles technologies favorisent l’adoption de la téléphonie IP.Microeconometric discrete choice models aim to explain the process of individual choice by consumers among a mutually exclusive, exhaustive and finite group of alternatives. Hybrid choice models are a generalization of standard discrete choice models where independent expanded models are considered simultaneously. In my dissertation I analyze, implement, and apply simultaneous estimation techniques for a hybrid choice model that, in the form of a complex generalized structural equation model, simultaneously integrates discrete choice and latent explanatory variables, such as attitudes and qualitative attributes. The motivation behind hybrid choice models is that the key to understanding choice comes through incorporating attitudinal and perceptual data to conventional economic models of decision making, taking elements from cognitive science and social psychology. The Bayesian Gibbs sampler I derive for simultaneous estimation of hybrid choice models offers a consistent and efficient estimator that outperforms frequentist full information simulated maximum likelihood. Whereas the frequentist estimator becomes fairly complex in situations with a large choice set of interdependent alternatives with a large number of latent variables, the inclusion of latent variables in the Bayesian approach translates into adding independent ordinary regressions. I also find that when using the Bayesian estimates it is easier to consider behavioral uncertainty; in fact, I show that forecasting and deriving confidence intervals for willingness to pay measures is straightforward. Finally, I confirm the capacity of hybrid choice modeling to adapt to practical situations. In particular, I analyze consumer response to innovation. For instance, I incorporate proenvironmental preferences toward low-emission vehicles into an economic model of purchase behavior where environmentally-conscious consumers are willing to pay more for sustainable solutions despite potential drawbacks. In addition, using a probit kernel and dichotomous effect indicators I show that knowledge as well as a positive attitude toward the adoption of new technologies favor the adoption of IP telephony

Cybersecurity: Past, Present and Future

The digital transformation has created a new digital space known as cyberspace. This new cyberspace has improved the workings of businesses, organizations, governments, society as a whole, and day to day life of an individual. With these improvements come new challenges, and one of the main challenges is security. The security of the new cyberspace is called cybersecurity. Cyberspace has created new technologies and environments such as cloud computing, smart devices, IoTs, and several others. To keep pace with these advancements in cyber technologies there is a need to expand research and develop new cybersecurity methods and tools to secure these domains and environments. This book is an effort to introduce the reader to the field of cybersecurity, highlight current issues and challenges, and provide future directions to mitigate or resolve them. The main specializations of cybersecurity covered in this book are software security, hardware security, the evolution of malware, biometrics, cyber intelligence, and cyber forensics. We must learn from the past, evolve our present and improve the future. Based on this objective, the book covers the past, present, and future of these main specializations of cybersecurity. The book also examines the upcoming areas of research in cyber intelligence, such as hybrid augmented and explainable artificial intelligence (AI). Human and AI collaboration can significantly increase the performance of a cybersecurity system. Interpreting and explaining machine learning models, i.e., explainable AI is an emerging field of study and has a lot of potentials to improve the role of AI in cybersecurity.Comment: Author's copy of the book published under ISBN: 978-620-4-74421-

A Machine Learning Enhanced Scheme for Intelligent Network Management

The versatile networking services bring about huge influence on daily living styles while the amount and diversity of services cause high complexity of network systems. The network scale and complexity grow with the increasing infrastructure apparatuses, networking function, networking slices, and underlying architecture evolution. The conventional way is manual administration to maintain the large and complex platform, which makes effective and insightful management troublesome. A feasible and promising scheme is to extract insightful information from largely produced network data. The goal of this thesis is to use learning-based algorithms inspired by machine learning communities to discover valuable knowledge from substantial network data, which directly promotes intelligent management and maintenance. In the thesis, the management and maintenance focus on two schemes: network anomalies detection and root causes localization; critical traffic resource control and optimization. Firstly, the abundant network data wrap up informative messages but its heterogeneity and perplexity make diagnosis challenging. For unstructured logs, abstract and formatted log templates are extracted to regulate log records. An in-depth analysis framework based on heterogeneous data is proposed in order to detect the occurrence of faults and anomalies. It employs representation learning methods to map unstructured data into numerical features, and fuses the extracted feature for network anomaly and fault detection. The representation learning makes use of word2vec-based embedding technologies for semantic expression. Next, the fault and anomaly detection solely unveils the occurrence of events while failing to figure out the root causes for useful administration so that the fault localization opens a gate to narrow down the source of systematic anomalies. The extracted features are formed as the anomaly degree coupled with an importance ranking method to highlight the locations of anomalies in network systems. Two types of ranking modes are instantiated by PageRank and operation errors for jointly highlighting latent issue of locations. Besides the fault and anomaly detection, network traffic engineering deals with network communication and computation resource to optimize data traffic transferring efficiency. Especially when network traffic are constrained with communication conditions, a pro-active path planning scheme is helpful for efficient traffic controlling actions. Then a learning-based traffic planning algorithm is proposed based on sequence-to-sequence model to discover hidden reasonable paths from abundant traffic history data over the Software Defined Network architecture. Finally, traffic engineering merely based on empirical data is likely to result in stale and sub-optimal solutions, even ending up with worse situations. A resilient mechanism is required to adapt network flows based on context into a dynamic environment. Thus, a reinforcement learning-based scheme is put forward for dynamic data forwarding considering network resource status, which explicitly presents a promising performance improvement. In the end, the proposed anomaly processing framework strengthens the analysis and diagnosis for network system administrators through synthesized fault detection and root cause localization. The learning-based traffic engineering stimulates networking flow management via experienced data and further shows a promising direction of flexible traffic adjustment for ever-changing environments

Net Neutrality

This book is available as open access through the Bloomsbury Open Access programme and is available on www.bloomsburycollections.com. Chris Marsden maneuvers through the hype articulated by Netwrok Neutrality advocates and opponents. He offers a clear-headed analysis of the high stakes in this debate about the Internet's future, and fearlessly refutes the misinformation and misconceptions that about' Professor Rob Freiden, Penn State University Net Neutrality is a very heated and contested policy principle regarding access for content providers to the Internet end-user, and potential discrimination in that access where the end-user's ISP (or another ISP) blocks that access in part or whole. The suggestion has been that the problem can be resolved by either introducing greater competition, or closely policing conditions for vertically integrated service, such as VOIP. However, that is not the whole story, and ISPs as a whole have incentives to discriminate between content for matters such as network management of spam, to secure and maintain customer experience at current levels, and for economic benefit from new Quality of Service standards. This includes offering a ‘priority lane' on the network for premium content types such as video and voice service. The author considers market developments and policy responses in Europe and the United States, draws conclusions and proposes regulatory recommendations

Models and analysis of vocal emissions for biomedical applications

This book of Proceedings collects the papers presented at the 3rd International Workshop on Models and Analysis of Vocal Emissions for Biomedical Applications, MAVEBA 2003, held 10-12 December 2003, Firenze, Italy. The workshop is organised every two years, and aims to stimulate contacts between specialists active in research and industrial developments, in the area of voice analysis for biomedical applications. The scope of the Workshop includes all aspects of voice modelling and analysis, ranging from fundamental research to all kinds of biomedical applications and related established and advanced technologies

Security-Driven Software Evolution Using A Model Driven Approach

High security level must be guaranteed in applications in order to mitigate risks during the deployment of information systems in open network environments. However, a significant number of legacy systems remain in use which poses security risks to the enterprise’ assets due to the poor technologies used and lack of security concerns when they were in design. Software reengineering is a way out to improve their security levels in a systematic way. Model driven is an approach in which model as defined by its type directs the execution of the process. The aim of this research is to explore how model driven approach can facilitate the software reengineering driven by security demand. The research in this thesis involves the following three phases. Firstly, legacy system understanding is performed using reverse engineering techniques. Task of this phase is to reverse engineer legacy system into UML models, partition the legacy system into subsystems with the help of model slicing technique and detect existing security mechanisms to determine whether or not the provided security in the legacy system satisfies the user’s security objectives. Secondly, security requirements are elicited using risk analysis method. It is the process of analysing key aspects of the legacy systems in terms of security. A new risk assessment method, taking consideration of asset, threat and vulnerability, is proposed and used to elicit the security requirements which will generate the detailed security requirements in the specific format to direct the subsequent security enhancement. Finally, security enhancement for the system is performed using the proposed ontology based security pattern approach. It is the stage that security patterns derived from security expertise and fulfilling the elicited security requirements are selected and integrated in the legacy system models with the help of the proposed security ontology. The proposed approach is evaluated by the selected case study. Based on the analysis, conclusions are drawn and future research is discussed at the end of this thesis. The results show this thesis contributes an effective, reusable and suitable evolution approach for software security