Activity Pattern Discovery from Network Captures

Investigating insider threat cases is challenging because activities are conducted with legitimate access that makes distinguishing malicious activities from normal activities difficult. To assist with identifying non-normal activities, we propose using two types of pattern discovery to identify a person\u27s behavioral patterns in network data. The behavioral patterns serve to deemphasize normal behavior so that insider threat investigations can focus attention on potentially more relevant. Results from a controlled experiment demonstrate the highlighting of a suspicious event through the reduction of events belonging to discovered patterns. Abstract © 2016 IEEE

Data Stream Clustering for Real-Time Anomaly Detection: An Application to Insider Threats

Insider threat detection is an emergent concern for academia, industries, and governments due to the growing number of insider incidents in recent years. The continuous streaming of unbounded data coming from various sources in an organisation, typically in a high velocity, leads to a typical Big Data computational problem. The malicious insider threat refers to anomalous behaviour(s) (outliers) that deviate from the normal baseline of a data stream. The absence of previously logged activities executed by users shapes the insider threat detection mechanism into an unsupervised anomaly detection approach over a data stream. A common shortcoming in the existing data mining approaches to detect insider threats is the high number of false alarms/positives (FPs). To handle the Big Data issue and to address the shortcoming, we propose a streaming anomaly detection approach, namely Ensemble of Random subspace Anomaly detectors In Data Streams (E-RAIDS), for insider threat detection. E-RAIDS learns an ensemble of p established outlier detection techniques [Micro-cluster-based Continuous Outlier Detection (MCOD) or Anytime Outlier Detection (AnyOut)] which employ clustering over continuous data streams. Each model of the p models learns from a random feature subspace to detect local outliers, which might not be detected over the whole feature space. E-RAIDS introduces an aggregate component that combines the results from the p feature subspaces, in order to confirm whether to generate an alarm at each window iteration. The merit of E-RAIDS is that it defines a survival factor and a vote factor to address the shortcoming of high number of FPs. Experiments on E-RAIDS-MCOD and E-RAIDS-AnyOut are carried out, on synthetic data sets including malicious insider threat scenarios generated at Carnegie Mellon University, to test the effectiveness of voting feature subspaces, and the capability to detect (more than one)-behaviour-all-threat in real-time. The results show that E-RAIDS-MCOD reports the highest F1 measure and less number of false alarm = 0 compared to E-RAIDS-AnyOut, as well as it attains to detect approximately all the insider threats in real-time

Detecting insider threat within institutions using CERT dataset and different ML techniques

The reason of countries development in industrial and commercial enterprises fields in those countries. The security of a particular country depends on its security institutions, the confidentiality of its employees, their information, the target's information, and information about the forensic evidence for those targets. One of the most important and critical problems in such institutions is the problem of discovering an insider threat that causes loss, damage, or theft the information to hostile or competing parties. This threat is represented by a person who represents one of the employees of the institution, the goal of that person is to steal information or destroy it for the benefit of another institution's desires. The difficulty in detecting this type of threat is due to the difficulty of analyzing the behavior of people within the organization according to their physiological characteristics. In this research, CERT dataset that produced by the University of Carnegie Mellon University has been used in this investigation to detect insider threat. The dataset has been preprocessed. Five effective features were selected to apply three ML techniques Random Forest, Naïve Bayes, and 1 Nearest Neighbor. The results obtained and listed sequentially as 89.75917519%, 91.96650826%, and 94.68205476% with an error rate of 10.24082481%, 8.03349174%, and 5.317945236%