Network Traffic Analysis Using Local Outlier Factor

The issue that this study addresses is the high rate of false positives, high maintenance, and lack of stability and precision that the existing network intrusion detection algorithm faces. To address this problem, we proposed a Local Outlier Factor (LOF) Algorithm that locates outliers and anomalies by comparing the deviation of one data point with respect to its neighbors. To gather data, we will use DARPA’s KDDCup99 as well as questions towards analysts. This data will help determine whether the LOF algorithm is more effective than existing solutions that are presented in the network intrusion detection space

A machine learning-based investigation of cloud service attacks

In this thesis, the security challenges of cloud computing are investigated in the Infrastructure as a Service (IaaS) layer, as security is one of the major concerns related to Cloud services. As IaaS consists of different security terms, the research has been further narrowed down to focus on Network Layer Security. Review of existing research revealed that several types of attacks and threats can affect cloud security. Therefore, there is a need for intrusion defence implementations to protect cloud services. Intrusion Detection (ID) is one of the most effective solutions for reacting to cloud network attacks. [Continues.

Big data analytics: a predictive analysis applied to cybersecurity in a financial organization

Project Work presented as partial requirement for obtaining the Master’s degree in Information Management, with a specialization in Knowledge Management and Business IntelligenceWith the generalization of the internet access, cyber attacks have registered an alarming growth in frequency and severity of damages, along with the awareness of organizations with heavy investments in cybersecurity, such as in the financial sector. This work is focused on an organization’s financial service that operates on the international markets in the payment systems industry. The objective was to develop a predictive framework solution responsible for threat detection to support the security team to open investigations on intrusive server requests, over the exponentially growing log events collected by the SIEM from the Apache Web Servers for the financial service. A Big Data framework, using Hadoop and Spark, was developed to perform classification tasks over the financial service requests, using Neural Networks, Logistic Regression, SVM, and Random Forests algorithms, while handling the training of the imbalance dataset through BEV. The main conclusions over the analysis conducted, registered the best scoring performances for the Random Forests classifier using all the preprocessed features available. Using the all the available worker nodes with a balanced configuration of the Spark executors, the most performant elapsed times for loading and preprocessing of the data were achieved using the column-oriented ORC with native format, while the row-oriented CSV format performed the best for the training of the classifiers.Com a generalização do acesso à internet, os ciberataques registaram um crescimento alarmante em frequência e severidade de danos causados, a par da consciencialização das organizações, com elevados investimentos em cibersegurança, como no setor financeiro. Este trabalho focou-se no serviço financeiro de uma organização que opera nos mercados internacionais da indústria de sistemas de pagamento. O objetivo consistiu no desenvolvimento uma solução preditiva responsável pela detecção de ameaças, por forma a dar suporte à equipa de segurança na abertura de investigações sobre pedidos intrusivos no servidor, relativamente aos exponencialmente crescentes eventos de log coletados pelo SIEM, referentes aos Apache Web Servers, para o serviço financeiro. Uma solução de Big Data, usando Hadoop e Spark, foi desenvolvida com o objectivo de executar tarefas de classificação sobre os pedidos do serviço financeiros, usando os algoritmos Neural Networks, Logistic Regression, SVM e Random Forests, solucionando os problemas associados ao treino de um dataset desequilibrado através de BEV. As principais conclusões sobre as análises realizadas registaram os melhores resultados de classificação usando o algoritmo Random Forests com todas as variáveis pré-processadas disponíveis. Usando todos os nós do cluster e uma configuração balanceada dos executores do Spark, os melhores tempos para carregar e pré-processar os dados foram obtidos usando o formato colunar ORC nativo, enquanto o formato CSV, orientado a linhas, apresentou os melhores tempos para o treino dos classificadores

TOWARDS A HOLISTIC EFFICIENT STACKING ENSEMBLE INTRUSION DETECTION SYSTEM USING NEWLY GENERATED HETEROGENEOUS DATASETS

With the exponential growth of network-based applications globally, there has been a transformation in organizations\u27 business models. Furthermore, cost reduction of both computational devices and the internet have led people to become more technology dependent. Consequently, due to inordinate use of computer networks, new risks have emerged. Therefore, the process of improving the speed and accuracy of security mechanisms has become crucial.Although abundant new security tools have been developed, the rapid-growth of malicious activities continues to be a pressing issue, as their ever-evolving attacks continue to create severe threats to network security. Classical security techniquesfor instance, firewallsare used as a first line of defense against security problems but remain unable to detect internal intrusions or adequately provide security countermeasures. Thus, network administrators tend to rely predominantly on Intrusion Detection Systems to detect such network intrusive activities. Machine Learning is one of the practical approaches to intrusion detection that learns from data to differentiate between normal and malicious traffic. Although Machine Learning approaches are used frequently, an in-depth analysis of Machine Learning algorithms in the context of intrusion detection has received less attention in the literature.Moreover, adequate datasets are necessary to train and evaluate anomaly-based network intrusion detection systems. There exist a number of such datasetsas DARPA, KDDCUP, and NSL-KDDthat have been widely adopted by researchers to train and evaluate the performance of their proposed intrusion detection approaches. Based on several studies, many such datasets are outworn and unreliable to use. Furthermore, some of these datasets suffer from a lack of traffic diversity and volumes, do not cover the variety of attacks, have anonymized packet information and payload that cannot reflect the current trends, or lack feature set and metadata.This thesis provides a comprehensive analysis of some of the existing Machine Learning approaches for identifying network intrusions. Specifically, it analyzes the algorithms along various dimensionsnamely, feature selection, sensitivity to the hyper-parameter selection, and class imbalance problemsthat are inherent to intrusion detection. It also produces a new reliable dataset labeled Game Theory and Cyber Security (GTCS) that matches real-world criteria, contains normal and different classes of attacks, and reflects the current network traffic trends. The GTCS dataset is used to evaluate the performance of the different approaches, and a detailed experimental evaluation to summarize the effectiveness of each approach is presented. Finally, the thesis proposes an ensemble classifier model composed of multiple classifiers with different learning paradigms to address the issue of detection accuracy and false alarm rate in intrusion detection systems

Cyber Crime Detection and Prevention Techniques on Cyber Cased Objects Using SVM and Smote

Conventional cybersecurity employs crime prevention mechanisms over distributed networks. This demands crime event management at the network level where Detection and Prevention of cybercrimes is a must. A new Framework IDSEM has been introduced in this paper to handle the contemporary heterogeneous objects in cloud environment. This may aid for deployment of analytical tools over the network. A supervised machine learning algorithm like SVM has been implemented to support IDSEM. A machine learning technique Like SMOTE has been implemented to handle imbalanced classification of the sample data. This approach addresses imbalanced datasets by oversampling the minority classes. This will help to solve Social Engineering Attacks (SEA) like Phishing and Vishing. Classification mechanisms like decision trees and probability functions are used in this context. The IDSEM framework could minimize traffic across the cloud network and detect cybercrimes maximally. When results were compared with existing approaches, the results were found to be good, leading to the development of a unique SMOTE algorithm

Network anomaly detection using adversarial Deep Learning

Dissertação de mestrado integrado em Engenharia InformáticaComputer networks security is becoming an important and challenging topic. In particular, one currently witnesses increasingly complex attacks which are also bound to become more and more sophisticated with the advent of artificial intelligence technologies. Intrusion detection systems are a crucial component in network security. However, the limited number of publicly available network datasets and their poor traffic variety and attack diversity are a major stumbling block in the proper development of these systems. In order to overcome such difficulties and therefore maximise the detection of anomalies in the network, it is proposed the use of Adversarial Deep Learning techniques to increase the amount and variety of existing data and, simultaneously, to improve the learning ability of the classification models used for anomaly detection. This master’s dissertation main goal is the development of a system that proves capable of improving the detection of anomalies in the network through the use of Adversarial Deep Learning techniques, in particular, Generative Adversarial Networks. With this in mind, firstly, a state-of-the-art analysis and a review of existing solutions were addressed. Subsequently, efforts were made to build a modular solution to learn from imbalanced datasets with applications not only in the field of anomaly detection in the network, but also in all areas affected by imbalanced data problems. Finally, it was demonstrated the feasibility of the developed system with its application to a network flow dataset.A segurança das redes de computadores tem-se vindo a tornar num tópico importante e desafiador. Em particular, atualmente testemunham-se ataques cada vez mais complexos que, com o advento das tecnologias de inteligência artificial, tendem a tornar-se cada vez mais sofisticados. Sistemas de deteção de intrusão são uma peça chave na segurança de redes de computadores. No entanto, o número limitado de dados públicos de fluxo de rede e a sua pobre diversidade e variedade de ataques revelam-se num grande obstáculo para o correto desenvolvimento destes sistemas. De forma a ultrapassar tais adversidades e consequentemente melhorar a deteção de anomalias na rede, é proposto que sejam utilizadas técnicas de Adversarial Deep Learning para aumentar o número e variedade de dados existentes e, simultaneamente, melhorar a capacidade de aprendizagem dos modelos de classificação utilizados na deteção de anomalias. O objetivo principal desta dissertação de mestrado é o desenvolvimento de um sistema que se prove capaz de melhorar a deteção de anomalias na rede através de técnicas de Adversarial Deep Learning, em particular, através do uso de Generative Adversarial Networks. Neste sentido, primeiramente, procedeu-se à análise do estado de arte assim como à investigação de soluções existentes. Posteriormente, atuou-se de forma a desenvolver uma solução modular com aplicação não só na área de deteção de anomalias na rede, mas também em todas as áreas afetadas pelo problema de dados desbalanceados. Por fim, demonstrou-se a viabilidade do sistema desenvolvido com a sua aplicação a um conjunto de dados de fluxo de rede