Usage Aware PageRank

Traditional link analysis approaches assume equal weights assigned to different links and pages. In original PageRank formulation, the user model assumes that the user has equal probability to follow each link from a given page, thus the score of a page equally affects all of the pages it points to. It also assumes that the probability for a user to go to a URL directly without following a link is the same for all URLs. In this paper, we investigate different weighting schemes that take into account the probability to go to a page directly (by typing or using bookmarks), as well as the relative probability to follow a link from a given page. Both of these probabilities can be approximated from usage logs if they are available. We introduce a naturalextension to the original PageRank formulation that we will call Usage aware PageRank (UPR). The new formulation combines static link structure graph with the usage graph that will be obtained via web logs or other means. It is also quite general; how much emphasis will be given to the graphs is controlled by a parameter. If the parameter is set to zero, the algorithm becomes equivalent to the original PageRank, if it is set to one, the emphasis shifts to the usage graph, and for values in between, both of the graphs will be used with weights specified by the parameter. UPR is also quite inexpensive. After a onetime precalculation step, an iteration of UPR takes about the same time as a PageRank iteration

MINDS: Architecture & Design

This chapter provides an overview of the Minnesota Intrusion Detection System (MINDS), which uses a suite of data mining based algorithms to address different aspects of cyber security. The various components of MINDS such as the scan detector, anomaly detector and the profiling module detect different types of attacks and intrusions on a computer network. The scan detector aims at detecting scans which are the precursors to any network attack. The anomaly detection algorithm is very effective in detecting behavioral anomalies in the network traffic which typically translate to malicious activities such as denial-of-service (DoS) traffic, worms, policy violations and inside abuse. The profiling module helps a network analyst to understand the characteristics of the network traffic and detect any deviations from the normal profile. Our analysis shows that the intrusions detected by MINDS are complementary to those of traditional signature based systems, such as SNORT, which implies that they both can be combined to increase overall attack coverage. MINDS has shown great operational success in detecting network intrusions in two live deployments at the University of Minnesota and as a part of the Interrogator architecture at the US Army Research Labs Center for Intrusion Monitoring and Protection (ARL-CIMP)

A comparative study of anomaly detection schemes in network intrusion detection

Abstract. Intrusion detection corresponds to a suite of techniques that can be used to identify attacks against computers and network infrastructures. Anomaly detection is a key element of intrusion detection systems in which perturbations of normal behavior suggest the presence of intentionally or unintentionally induced attacks, faults, defects, etc. Several recently developed anomaly and outlier detection schemes have been proposed for detecting novel attacks whose nature is unknown. To benefit the anomaly detection framework, a procedure for extracting additional useful features is also implemented. In addition, evaluation of anomaly detection algorithms is performed using standard metrics as well as specific metrics that are especially suitable in detecting intrusions that involve multiple network connections. The detailed comparison of anomaly detection algorithms applied to DARPA 1998 Intrusion Detection Evaluation Data demonstrate that depending on the attack type some anomaly detection schemes are more successful in detecting novel anomalies than others. However, during the past few months the most prominent techniques have also been applied to real network data, and they have been very successful in automatically identifying several novel intrusions, which were at the same time reported by CERT (Computer Emergency Response Team/Coordination Center) for additional investigation, since state-of-the-art intrusion detection techniques could not detect them. 1

Data mining for network intrusion detection

This paper gives an overview of our research in building rare class prediction models for identifying known intrusions and their variations and anomaly/outlier detection schemes for detecting novel attacks whose nature is unknown. Experimental results on the KDDCup’99 data set have demonstrated that our rare class predictive models are much more efficient in the detection of intrusive behavior than standard classification techniques. Experimental results on the DARPA 1998 data set, as well as on live network traffic at the University of Minnesota, show that the new techniques show great promise in detecting novel intrusions. In particular, during the past few months our techniques have been successful in automatically identifying several novel intrusions that could not be detected using state-of-the-art tools such as SNORT. In fact, many of these have been on the CERT/CC list of recent advisories and incident notes. 1