Real-time big data processing for anomaly detection : a survey

The advent of connected devices and omnipresence of Internet have paved way for intruders to attack networks, which leads to cyber-attack, financial loss, information theft in healthcare, and cyber war. Hence, network security analytics has become an important area of concern and has gained intensive attention among researchers, off late, specifically in the domain of anomaly detection in network, which is considered crucial for network security. However, preliminary investigations have revealed that the existing approaches to detect anomalies in network are not effective enough, particularly to detect them in real time. The reason for the inefficacy of current approaches is mainly due the amassment of massive volumes of data though the connected devices. Therefore, it is crucial to propose a framework that effectively handles real time big data processing and detect anomalies in networks. In this regard, this paper attempts to address the issue of detecting anomalies in real time. Respectively, this paper has surveyed the state-of-the-art real-time big data processing technologies related to anomaly detection and the vital characteristics of associated machine learning algorithms. This paper begins with the explanation of essential contexts and taxonomy of real-time big data processing, anomalous detection, and machine learning algorithms, followed by the review of big data processing technologies. Finally, the identified research challenges of real-time big data processing in anomaly detection are discussed. © 2018 Elsevier Lt

A PREDICTIVE USER BEHAVIOUR ANALYTIC MODEL FOR INSIDER THREATS IN CYBERSPACE

Insider threat in cyberspace is a recurring problem since the user activities in a cyber network are often unpredictable. Most existing solutions are not flexible and adaptable to detect sudden change in user’s behaviour in streaming data, which led to a high false alarm rates and low detection rates. In this study, a model that is capable of adapting to the changing pattern in structured cyberspace data streams in order to detect malicious insider activities in cyberspace was proposed. The Computer Emergency Response Team (CERT) dataset was used as the data source in this study. Extracted features from the dataset were normalized using Min-Max normalization. Standard scaler techniques and mutual information gain technique were used to determine the best features for classification. A hybrid detection model was formulated using the synergism of Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU) models. Model simulation was performed using python programming language. Performance evaluation was carried out by assessing and comparing the performance of the proposed model with a selected existing model using accuracy, precision and sensitivity as performance metrics. The result of the simulation showed that the developed model has an increase of 1.48% of detection accuracy, 4.21% of precision and 1.25% sensitivity over the existing model. This indicated that the developed hybrid approach was able to learn from sequences of user actions in a time and frequency domain and improves the detection rate of insider threats in cyberspace

A System for Detecting Malicious Insider Data Theft in IaaS Cloud Environments

The Cloud Security Alliance lists data theft and insider attacks as critical threats to cloud security. Our work puts forth an approach using a train, monitor, detect pattern which leverages a stateful rule based k-nearest neighbors anomaly detection technique and system state data to detect inside attacker data theft on Infrastructure as a Service (IaaS) nodes. We posit, instantiate, and demonstrate our approach using the Eucalyptus cloud computing infrastructure where we observe a 100 percent detection rate for abnormal login events and data copies to outside systems

The Big Picture: Using Desktop Imagery for Detection of Insider Threats

The insider threat is one of the most difficult problems in information security. Prior research addresses its detection by using machine learning techniques to profile user behavior. User behavior is represented as low level system events, which do not provide sufficient contextual information about the user\u27s intentions, and lead to high error rates. Our system uses video of a user\u27s sessions as the representation of their behavior, and detects moments during which they perform sensitive tasks. Analysis of the video is accomplished using OCR, scene detection algorithms, and basic text classification. The system outputs the results to a web interface, and our results show that using desktop imagery is a viable alternative to using system calls for insider threat detection