ATLANTIDES: Automatic Configuration for Alert Verification in Network Intrusion Detection Systems

We present an architecture designed for alert verification (i.e., to reduce false positives) in network intrusion-detection systems. Our technique is based on a systematic (and automatic) anomaly-based analysis of the system output, which provides useful context information regarding the network services. The false positives raised by the NIDS analyzing the incoming traffic (which can be either signature- or anomaly-based) are reduced by correlating them with the output anomalies. We designed our architecture for TCP-based network services which have a client/server architecture (such as HTTP). Benchmarks show a substantial reduction of false positives between 50% and 100%

Security risk assessment and protection in the chemical and process industry

This article describes a security risk assessment and protection methodology that was developed for use in the chemical- and process industry in Belgium. The approach of the method follows a risk-based approach that follows desing principles for chemical safety. That approach is beneficial for workers in the chemical industry because they recognize the steps in this model from familiar safety models .The model combines the rings-of-protection approach with generic security practices including: management and procedures, security technology (e.g. CCTV, fences, and access control), and human interactions (pro-active as well as re-active). The method is illustrated in a case-study where a practical protection plan was developed for an existing chemical company. This chapter demonstrates that the method is useful for similar chemical- and process industrial activities far beyond the Belgian borders, as well as for cross-industrial security protection. This chapter offers an insight into how the chemical sector protects itself on the one hand, and an insight into how security risk management can be practiced on the other hand

Resilient event collection in SIEM systems

Tese de mestrado em Segurança Informática, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2013A importância da Segurança da Informação tem crescido rapidamente nos últimos anos, com uma maior consciencialização da sociedade civil e das empresas para o problema. As notícias recorrentes de ataques direcionados e roubo de informação em larga escala que resultam em grandes prejuízos financeiros, por vezes tendo como consequência o encerramento das organizações envolvidas, justificam o investimento em mecanismos de proteção da informação. No âmago da capacidade para monitorização da segurança em tempo-real está o Security Operations Center (SOC), o conjunto de pessoas, processos e sistemas onde se concentram as capacidades de análise e resposta a incidentes de Segurança da Informação. A base tecnológica do SOC é construída sobre o sistema de Gestão de Informação e Eventos de Segurança, vulgo SIEM. Este sistema permite recolher eventos de segurança de diversas fontes e encontrar padrões de ataque analisando relações entre eles. No entanto, tal como acontece com todos os sistemas informáticos, um atacante que tenha conhecimento da sua existência irá procurar ultrapassar as proteções implementadas, prevenindo que a equipa do SOC seja alertada para o ataque em curso. A relevância dos sistemas SIEM tem vindo a aumentar no contexto da maior importância atribuída a questões de segurança da informação. Considerando um número cada vez mais elevado de eventos e as múltiplas origens onde estes são gerados, as equipas de monitorização estão cada vez mais dependentes de consolas únicas onde a informação é centralizada e processada. Como consequência existe também uma maior dependência dos sistemas centrais, tornando-os pontos únicos de falha. Os sistemas SIEM são intrinsecamente complexos devido à necessidade de recolha de eventos de segurança a partir de fontes com tecnologias muito diversas, com localizações dispersas. O facto de desempenharem diversas funções aumenta esta complexidade, necessitando de módulos para recolha, consolidação, processamento e armazenamento de eventos. Para além destes módulos, que podem ou não traduzir-se em componentes fisicamente distintos, os sistemas SIEM estão fortemente dependentes dos sensores colocados junto às fontes de eventos, bem como da rede de comunicações que permite o envio desses eventos entre os diversos componentes, até à consola central. A inexistência de investigação diretamente focada no aumento da resiliência dos sistemas SIEM resulta na implementação de soluções pouco adaptadas aos riscos e desafios associados a infraestruturas de segurança. Estando maioritariamente focada na proteção de segurança ao nível da rede, muitos dos desenvolvimentos recentes centram-se na capacidade de identificar padrões de tráfego maliciosos. Esta abordagem reflete-se em publicações direcionadas aos sistemas de detecção e prevenção de intrusões (IDS/IPS), com menos enfoque na implementação resiliente de sistemas SIEM. A nossa percepção, corroborada por uma pesquisa alargada de trabalhos desenvolvidos nesta área, aponta para um elevado número de implementações padrão, assumindo cenários teóricos e sem tomar em linha de conta o efeito de ataques contra o próprio sistema SIEM. Neste trabalho começamos por efetuar uma análise às falhas de segurança que podem afectar o desempenho do processo de recolha de eventos de segurança, incluindo falhas acidentais mas também possíveis ataques deliberados ao sistema SIEM que possibilitem a uma entidade maliciosa ultrapassar os mecanismos de segurança implementados. Com base nessa análise endereçamos os problemas de fiabilidade que afetam qualquer sistema informático, apontando soluções que permitam lidar com falhas acidentais e, dessa forma, aumentar a disponibilidade do sistema. Ao reduzir a probabilidade de falhas que impeçam a recolha de eventos de segurança, estamos a contribuir diretamente para diminuir a janela de oportunidade disponível para que ataques à infraestrutura não sejam detectados. Focando o risco de falhas maliciosas, propomos soluções que impeçam os atacantes de explorar com sucesso vulnerabilidades no processo de recolha de eventos de segurança. Este processo envolve sistemas heterogéneos, desde a fonte dos eventos até à consola central, passando pela rede de comunicação responsável por interligar toda a infraestrutura. Consideramos fundamental atingir um nível de robustez elevado, mesmo na presença de infraestrutura parcialmente comprometida. O principal objectivo deste trabalho passa por definir um método sistemático de recolha e correlação resiliente de eventos de segurança num sistema SIEM, mesmo na presença de componentes maliciosos sob controlo de atacantes. Para atingir este objectivo centramo-nos na robustez das regras de correlação, desde a sua concepção e desenho até à implementação final no sistema SIEM. Os sistemas SIEM contêm um conjunto alargado de regras padrão que, como demonstramos, partem de premissas demasiado optimistas relativamente ao processo de recolha de eventos. Descrevemos, ao longo do trabalho, de que forma estas regras padrão podem ser melhoradas para lidar com as diversas possibilidades de falhas e ataques maliciosos, aumentando desta forma a resiliência total do sistema SIEM e o nível de confiança que a equipa do SOC pode depositar nesta ferramenta essencial. Utilizando casos de uso reais, demonstramos a metodologia proposta para aumentar a resiliência das regras de correlação. Tendo como ponto de partida uma regra base, aplicamos passo a passo a metodologia, detalhando e avaliando cada evolução da regra, até ser atingido um nível de robustez elevado. Com o propósito de sistematizar a metodologia proposta para o aumento de qualidade das regras de correlação, desenvolvemos uma aplicação denominada AutoRule. Esta ferramenta recebe como entrada uma ou mais regras de correlação e efetua uma análise automática, detectando possíveis lacunas e sugerindo correções. Apesar de não suprir a necessidade de análise com base na experiência prática na definição de regras de correlação, a aplicação AutoRule permite à equipa de configuração do sistema SIEM atuar de forma precisa e direcionada, corrigindo as regras de correlação e, dessa forma, tornando-as mais resilientes. Finalmente, para demonstrar e medir a eficácia da nossa proposta, foi posta em prática a metodologia através de uma implementação em cenário real, recorrendo ao sistema SIEM utilizado para monitorizar os eventos de segurança na rede corporativa da EDP – Energias de Portugal, S.A. Tratando-se de um grupo multinacional com mais de 12000 colaboradores ativos, a rede informática monitorizada por este sistema SIEM fornece a possibilidade de analisar em larga escala os efeitos das melhorias propostas. A metodologia proposta para aumentar a resiliência das regras de correlação traduziu-se num acréscimo da eficácia das mesmas, resultando num sistema mais fiável. A consequência mais direta é uma melhoria operacional do SOC, que passa a dispor de informação mais precisa e mais adequada ao seu contexto de operação. Para além da proposta teórica, a implementação permitiu também validar a operação num cenário real da aplicação AutoRule, desenvolvida para automatizar a análise das regras de correlação. As melhorias introduzidas nas regras de correlação desenvolvidas no contexto da operação do SOC EDP, seguindo os passos da metodologia, foram sendo testadas com recurso à aplicação. Os resultados demonstram que a eficácia medida das regras correspondeu também a um melhor resultado obtido através da análise automática, existindo por isso motivos para confiar nesta análise. A aplicação AutoRule possibilitou ainda uma comparação entre as regras predefinidas, instaladas de forma automática com a solução ArcSight, e as regras que seguiram o processo de melhoria preconizado pela metodologia proposta. As avaliações finais que fazemos da implementação num cenário real são francamente positivas, ratificando a nossa proposta teórica e conferindo-lhe um elevado grau de confiança quanto à possibilidade de aplicação em larga escala, de forma independente da tecnologia de sistema SIEM escolhida.Information Security has become a relevant subject in recent years, with greater awareness to the topic from major companies and general public. The frequent news regarding targeted attacks and large-scale information thefts resulting in major financial losses, sometimes even resulting in company bankruptcy, justify investments in protection mechanisms. At the heart of real-time security monitoring is the Security Information and Event Management system, commonly known as SIEM. These systems allow for security event collection and pattern discovery, by analyzing relationships between those events in real-time. However, as with all computer systems, an attacker who is aware of its existence will seek to overcome the protection mechanisms in place, preventing the security experts from being alerted to the ongoing attacks. We present an analysis of possible attacks to a SIEM system and seek solutions to prevent successful exploitation of those attacks, even if the attackers are able to take control over part of the infrastructure. Instead of suggesting massive changes throughout the multiple systems and network components, we propose an approach based on the capabilities of the SIEM system to collect and correlate security events from multiple sources. We advocate that it is possible to detect faults, malicious or accidental, though real time analysis of the collected events using carefully crafted and resilient correlation rules. Our goal is to define a systematic method to resiliently collect and correlate security events in a SIEM system, despite the presence of components already under the control of attackers. The effectiveness of the proposed methodology is evaluated in a real production environment, simulating attacks and accidental failures and observing their effects in the capability of the SIEM system to identify abnormal behavior. We also develop and demonstrate an application capable of automatically analyzing correlation rules, identifying vulnerabilities and proposing improvements to increase heir overall resilience

Development of a Methodology for Customizing Insider Threat Auditing on a Linux Operating System

Insider threats can pose a great risk to organizations and by their very nature are difficult to protect against. Auditing and system logging are capabilities present in most operating systems and can be used for detecting insider activity. However, current auditing methods are typically applied in a haphazard way, if at all, and are not conducive to contributing to an effective insider threat security policy. This research develops a methodology for designing a customized auditing and logging template for a Linux operating system. An intent-based insider threat risk assessment methodology is presented to create use case scenarios tailored to address an organization’s specific security needs and priorities. These organization specific use cases are verified to be detectable via the Linux auditing and logging subsystems and the results are analyzed to create an effective auditing rule set and logging configuration for the detectable use cases. Results indicate that creating a customized auditing rule set and system logging configuration to detect insider threat activity is possible

AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

Threshold verification using statistical approach for fast attack detection

Network has grows to a mammoth size and becoming more complex, thus exposing the services it offers towards multiple types of intrusion vulnerabilities.One method to overcome intrusion is by introducing Intrusion Detection System (IDS) for detecting the threat before it can damage the network resources.IDS have the ability to analyze network traffic and recognize incoming and on-going network attack.In detecting intrusion attack, Information gathering on such activity can be classified into fast attack and slow attack.Yet, majority of the current intrusion detection systems do not have the ability to differentiate between these two types of attacks. Early detection of fast attack is very useful in a real time environment; in which it can help the targeted network from further intrusion that could let the intruder to gain access to the vulnerable machine.To address this challenge, this paper introduces a fast attack detection framework that set a threshold value to differentiate between the normal network traffic and abnormal network traffic on the victim perspective. The threshold value is abstract with the help of suitable set of feature used to detect the anomaly in the network. By introducing the threshold value, anomaly based detection can build a complete profile to detect any intrusion threat as well as at the same time reducing it false alarm alert

A GENERIC ARCHITECTURE FOR INSIDER MISUSE MONITORING IN IT SYSTEMS

Intrusion Detection Systems (IDS) have been widely deployed within many organisations' IT nenvorks to delect network penetration attacks by outsiders and privilege escalation attacks by insiders. However, traditional IDS are ineffective for detecting o f abuse o f legitimate privileges by authorised users within the organisation i.e. the detection of misfeasance. In essence insider IT abuse does not violate system level controls, yet violates acceptable usage policy, business controls, or code of conduct defined by the organisation. However, the acceptable usage policy can vary from one organisation to another, and the acceptability o f user activities can also change depending upon the user(s), application, machine, data, and other contextual conditions associated with the entities involved. The fact that the perpetrators are authorised users and that the insider misuse activities do not violate system level controls makes detection of insider abuse more complicated than detection o f attacks by outsiders. The overall aim o f the research is to determine novel methods by which monitoring and detection may be improved to enable successful detection of insider IT abuse. The discussion begins with a comprehensive investigation o f insider IT misuse, encompassing the breadth and scale of the problem. Consideration is then given to the sufficiency of existing safeguards, with the conclusion that they provide an inadequate basis for detecting many o f the problems. This finding is used as the justification for considering research into alternative approaches. The realisation of the research objective includes the development of a taxonomy for identification o f various levels within the system from which the relevant data associated with each type of misuse can be collected, and formulation of a checklist for identification of applications that requires misfeasor monitoring. Based upon this foundation a novel architecture for monitoring o f insider IT misuse, has been designed. The design offers new analysis procedures to be added, while providing methods to include relevant contextual parameters from dispersed systems for analysis and reference. The proposed system differs from existing IDS in the way that it focuses on detecting contextual misuse of authorised privileges and legitimate operations, rather than detecting exploitation o f network protocols and system level \ailnerabilities. The main concepts of the new architecture were validated through a proof-of-concept prototype system. A number o f case scenarios were used to demonstrate the validity of analysis procedures developed and how the contextual data from dispersed databases can be used for analysis of various types of insider activities. This helped prove that the existing detection technologies can be adopted for detection o f insider IT misuse, and that the research has thus provided valuable contribution to the domain

Multi-paradigm frameworks for scalable intrusion detection

Research in network security and intrusion detection systems (IDSs) has typically focused on small or artificial data sets. Tools are developed that work well on these data sets but have trouble meeting the demands of real-world, large-scale network environments. In addressing this problem, improvements must be made to the foundations of intrusion detection systems, including data management, IDS accuracy and alert volume;We address data management of network security and intrusion detection information by presenting a database mediator system that provides single query access via a domain specific query language. Results are returned in the form of XML using web services, allowing analysts to access information from remote networks in a uniform manner. The system also provides scalable data capture of log data for multi-terabyte datasets;Next, we address IDS alert accuracy by building an agent-based framework that utilizes web services to make the system easy to deploy and capable of spanning network boundaries. Agents in the framework process IDS alerts managed by a central alert broker. The broker can define processing hierarchies by assigning dependencies on agents to achieve scalability. The framework can also be used for the task of event correlation, or gathering information relevant to an IDS alert;Lastly, we address alert volume by presenting an approach to alert correlation that is IDS independent. Using correlated events gathered in our agent framework, we build a feature vector for each IDS alert representing the network traffic profile of the internal host at the time of the alert. This feature vector is used as a statistical fingerprint in a clustering algorithm that groups related alerts. We analyze our results with a combination of domain expert evaluation and feature selection

A graph oriented approach for network forensic analysis

Network forensic analysis is a process that analyzes intrusion evidence captured from networked environment to identify suspicious entities and stepwise actions in an attack scenario. Unfortunately, the overwhelming amount and low quality of output from security sensors make it difficult for analysts to obtain a succinct high-level view of complex multi-stage intrusions. This dissertation presents a novel graph based network forensic analysis system. The evidence graph model provides an intuitive representation of collected evidence as well as the foundation for forensic analysis. Based on the evidence graph, we develop a set of analysis components in a hierarchical reasoning framework. Local reasoning utilizes fuzzy inference to infer the functional states of an host level entity from its local observations. Global reasoning performs graph structure analysis to identify the set of highly correlated hosts that belong to the coordinated attack scenario. In global reasoning, we apply spectral clustering and Pagerank methods for generic and targeted investigation respectively. An interactive hypothesis testing procedure is developed to identify hidden attackers from non-explicit-malicious evidence. Finally, we introduce the notion of target-oriented effective event sequence (TOEES) to semantically reconstruct stealthy attack scenarios with less dependency on ad-hoc expert knowledge. Well established computation methods used in our approach provide the scalability needed to perform post-incident analysis in large networks. We evaluate the techniques with a number of intrusion detection datasets and the experiment results show that our approach is effective in identifying complex multi-stage attacks