Developing Cyberspace Data Understanding: Using CRISP-DM for Host-based IDS Feature Mining

Current intrusion detection systems generate a large number of specific alerts, but do not provide actionable information. Many times, these alerts must be analyzed by a network defender, a time consuming and tedious task which can occur hours or days after an attack occurs. Improved understanding of the cyberspace domain can lead to great advancements in Cyberspace situational awareness research and development. This thesis applies the Cross Industry Standard Process for Data Mining (CRISP-DM) to develop an understanding about a host system under attack. Data is generated by launching scans and exploits at a machine outfitted with a set of host-based data collectors. Through knowledge discovery, features are identified within the data collected which can be used to enhance host-based intrusion detection. By discovering relationships between the data collected and the events, human understanding of the activity is shown. This method of searching for hidden relationships between sensors greatly enhances understanding of new attacks and vulnerabilities, bolstering our ability to defend the cyberspace domain

SUSTAV ZA OTKRIVANJE I OBRANU KORIŠTENJEM RUDARENJA PODATAKA

Network security helps to prevent the network against the intruders from performing malicious activities. The security can be provided to the networks using firewalls, anti-virus software and scanners, cryptographic systems, Secure Socket Layer (SSL) and Intrusion Detection Systems (IDS).Authentication is the commonly used technique to protect the unauthorized users from the network. But, it is easy to compromise the login passwords using brute force attacks. The IDS and firewalls concentrate on the external attacks, while the internal attacks are not taken into account. In order to solve these issues, this paper proposes an Inner Interruption Discovery and Defense System (IIDDS) at the System Call (SC) level using data mining and forensic techniques. The user’s profiles are maintained and compared with the actual dataset using Hellinger distance. A hash function is applied on the incoming messages and they are summarized in the sketch dataset. The experimental results evaluate the proposed system in terms of accuracy and response time.Mrežna sigurnost pomaže zaštititi mrežu od uljeza u obavljanju zlonamjernih aktivnosti. Sigurnost se može osigurati mrežama koristeći vatrozide, antivirusni softver i skenere, kriptografske sustave, Secure Socket Layer (SSL) i sustave za otkrivanje upada (IDS). Autentifikacija je najčešće korištena tehnika za zaštitu neovlaštenih korisnika na mreži. No, lako je kompromitirati lozinke za prijavu pomoću napada na silu. IDS i vatrozidi koncentriraju se na vanjske napade, dok se interni napadi ne uzimaju u obzir. Da bi se riješili ti problemi, u članku se predlaže unutarnje prekidanje i obrambeni sustav (IIDDS) na razini System Call (SC) razine pomoću rudarenja podataka i forenzičke tehnike. Profili korisnika održavaju se i uspoređuju sa stvarnim skupom podataka pomoću Hellingerove udaljenosti. Na dolazne poruke primjenjuje se hash funkcija i oni su sažeti u skupu skica podataka. Eksperimentalni rezultati procjenjuju predloženi sustav u smislu točnosti i vremena odziva

Data mining Techniques for Digital Forensic Analysis

The computer forensic involve the protection, classification, taking out information and documents the evidence stored as data or magnetically encoded information. But the organizations have an increasing amount of data from many sources like computing peripherals, personal digital assistants (PDA), consumer electronic devices, computer systems, networking equipment and various types of media, among other sources. To find similar kinds of evidences, crimes happened previously, the law enforcement officers, police forces and detective agencies is time consuming and headache. The main motive of this work is by combining a data mining techniques with computer forensic tools to get the data ready for analysis, find crime patterns, understand the mind of the criminal, assist investigation agencies have to be one step ahead of the bad guys, to speed up the process of solving crimes and carry out computer forensics analyses for criminal affairs

Sharing Computer Network Logs for Security and Privacy: A Motivation for New Methodologies of Anonymization

Logs are one of the most fundamental resources to any security professional. It is widely recognized by the government and industry that it is both beneficial and desirable to share logs for the purpose of security research. However, the sharing is not happening or not to the degree or magnitude that is desired. Organizations are reluctant to share logs because of the risk of exposing sensitive information to potential attackers. We believe this reluctance remains high because current anonymization techniques are weak and one-size-fits-all--or better put, one size tries to fit all. We must develop standards and make anonymization available at varying levels, striking a balance between privacy and utility. Organizations have different needs and trust other organizations to different degrees. They must be able to map multiple anonymization levels with defined risks to the trust levels they share with (would-be) receivers. It is not until there are industry standards for multiple levels of anonymization that we will be able to move forward and achieve the goal of widespread sharing of logs for security researchers.Comment: 17 pages, 1 figur

From Intrusion Detection to Attacker Attribution: A Comprehensive Survey of Unsupervised Methods

Over the last five years there has been an increase in the frequency and diversity of network attacks. This holds true, as more and more organisations admit compromises on a daily basis. Many misuse and anomaly based Intrusion Detection Systems (IDSs) that rely on either signatures, supervised or statistical methods have been proposed in the literature, but their trustworthiness is debatable. Moreover, as this work uncovers, the current IDSs are based on obsolete attack classes that do not reflect the current attack trends. For these reasons, this paper provides a comprehensive overview of unsupervised and hybrid methods for intrusion detection, discussing their potential in the domain. We also present and highlight the importance of feature engineering techniques that have been proposed for intrusion detection. Furthermore, we discuss that current IDSs should evolve from simple detection to correlation and attribution. We descant how IDS data could be used to reconstruct and correlate attacks to identify attackers, with the use of advanced data analytics techniques. Finally, we argue how the present IDS attack classes can be extended to match the modern attacks and propose three new classes regarding the outgoing network communicatio

Behavior Profiling of Email

This paper describes the forensic and intelligence analysis capabilities of the Email Mining Toolkit (EMT) under development at the Columbia Intrusion Detection (IDS) Lab. EMT provides the means of loading, parsing and analyzing email logs, including content, in a wide range of formats. Many tools and techniques have been available from the fields of Information Retrieval (IR) and Natural Language Processing (NLP) for analyzing documents of various sorts, including emails. EMT, however, extends these kinds of analyses with an entirely new set of analyses that model "user behavior." EMT thus models the behavior of individual user email accounts, or groups of accounts, including the "social cliques" revealed by a user's email behavior

Survey of Intrusion Detection Research

The literature holds a great deal of research in the intrusion detection area. Much of this describes the design and implementation of specific intrusion detection systems. While the main focus has been the study of different detection algorithms and methods, there are a number of other issues that are of equal importance to make these systems function well in practice. I believe that the reason that the commercial market does not use many of the ideas described is that there are still too many unresolved issues. This survey focuses on presenting the different issues that must be addressed to build fully functional and practically usable intrusion detection systems (IDSs). It points out the state of the art in each area and suggests important open research issues