Anomaly detection systems are a promising tool to identify compromised user credentials and malicious insiders in enterprise networks. Most existing approaches for modelling user behaviour rely on either independent observations for each user or on pre-defined user peer groups. A method is proposed based on recommender system algorithms to learn overlapping user peer groups and to use this learned structure to detect anomalous activity. Results analysing the authentication and process-running activities of thousands of users show that the proposed method can detect compromised user accounts during a red team exercise

Heard, NA

McPhall, A

Moore, J

Turcotte, M

English

Spiral - Imperial College Digital Repository

Poisson Factorization for Peer-Based AnomalyDetectionMelissa Turcotte ∗, Juston Moore ∗, Nick Heard † and Aaron McPhall ∗∗ Advanced Research in Cyber Systems, Los Alamos National Laboratory† Department of Mathematics, Imperial College Londonand Heilbronn Institute for Mathematical Research, University of BristolAbstract—Anomaly detection techniques for identifying com-promised user credentials in an enterprise network are an impor-tant research problem, garnering much attention within industryover recent years. One important aspect of the research problemis peer-based user analysis. A method based on recommendersystem algorithms is proposed here, quantifying when a useractivity is unlikely based on the behavior of similar users.Building several recommender system algorithms for separateuser activities provides an additional advantage of allowing fordifferent peer group structures depending on the user activitybeing considered.I. INTRODUCTIONDetecting compromised or rogue users in an enterprisenetwork continues to be a challenging and important problem.The latest 2016 Verizon Data Breach report [1], claimed that63% of data breaches involved stolen user credentials. Further-more, insider threat continues to be a growing problem, andis often the hardest form of credential misuse to detect. Tra-ditional intrusion detection techniques are largely rule-basedrequiring specific threat signatures, which can be ineffectivewhen dealing with increasingly sophisticated cyber attacks,both from external and internal actors. Anomaly detectionand machine learning techniques for identifying compromisedusers within a network have gained popularity both within theacademic community [2]–[4] and more recently in industrywhere they are commonly referred to as user behavior analytics(UBA).With UBA the goal is to model the behavior patterns ofusers within the network and then predict which activities areunlikely, both with respect to the activity of the individualuser and that of their peers. Peer-based analysis is extremelyimportant with any UBA system to both reduce false alarmsand detect when users are behaving anomalously with respectto their peers. In industry, peer groups are often formed usingdirectory groupings or human resources information and theseglobal peer groups are assigned across all user activities. Amore robust approach would be to learn peer groups basedon users displaying similar behavior as observed in the dataand allow for different groupings of users depending on theactivity or feature of the data being analyzed and then detectanomalies with respect to the inferred peer groups.This article proposes the use of recommender systems forpeer-based anomaly detection, with separate instantiations ofthe algorithm for different features of the data. Recommendersystems are widely used for predicting the ‘rating’ or ‘pref-erence’ a user will give an item, based on historical dataabout which items the user has consumed, where consumedcould refer to a number of actions such as rating, clickingor viewing. Collaborative filtering approaches specifically aimto predict a user preference by exploiting similarity to otherusers through the items that they consume. So far, the use ofrecommender systems for anomaly detection has been underutilized, although notably in [5] a nearest neighbor basedalgorithm is utilized for anomaly detection on bipartite graphs.In this paper a specific collaborative filtering algorithmdeveloped in [6] is employed, where a Poisson factorizationmodel is used for recommendations. In particular, two useractivities are considered: the processes run by the user, andmachines on which users authenticate.Section II reviews the model presented in [6], and SectionIII explains how the fitted model can be used for anomalydetection. Section IV introduces the motivating data set andpresents the results of the analysis.II. POISSON RECOMMENDATIONFor n users and m items, let Y ∈ Nn×m0 be a matrix ofcounts, where element Yui is the random variable for the num-ber of times the user u invoked process i, or authenticated tomachine i. These data can be modeled using a k-dimensionalPoisson factorization model, where each item i and useru are represented by non-negative k-vector latent factorsθu = (θu1, . . . , θuk) and βi = (βi1, . . . , βik) respectively. Thecounts Yui are assumed to follow a Poisson distribution withmean given by the dot product of the latent variables,Yui ∼ Poisson(θu · βi).To capture diversity in the activity levels across the userand item populations, [6] use hierarchical gamma priors forthe latent factorsθujiid∼ Gamma(a, ξu), j = 1, . . . , k, ξu ∼ Gamma(a′, b′),βujiid∼ Gamma(b, ηi), j = 1, . . . , k, ηi ∼ Gamma(c′, d′),(1)so that the hyperparameters ξu and ηi correspond to overallactivity levels for user u and item i.Given an observed user-item matrix of counts Y , inferen-tial interest is focused on the marginal posterior distribution[θ,β|Y ], since this underpins the predictive distribution onwhich user-item pairs are likely to be observed in the future.Since the posterior does not have a closed-form solution,[6] uses variational inference which is an optimization-basedtechnique providing analytic approximations to intractableposterior distributions for complex models. The mean-fieldvariational algorithm from [6] will be utilized here using thecode provided at https://github.com/premgopalan/hgaprec, andthe reader is referred to [6] for details.III. ANOMALY DETECTIONThe problem statement with respect to anomaly detectionis to determine if the observed user-item pairs over sometime period are considered normal with respect to the modellearned over some training period or if they can be consideredanomalous. For user u and item i, an observed count yuiduring a testing period is given an anomaly score equal tothe upper tail probability of yui given the posterior expectedvalues of the latent factors, θ̂u and β̂i,pui = Pr(Yui ≥ yui|θ̂u, β̂i). (2)Note that this serves as a computationally tractable approxima-tion to the true posterior predictive upper tail p-value, whichagain does not have a closed-form solution.Given a sequence of observed counts across items for auser, yu1, . . . , yum, the p-values (2) can be combined to givean overall anomaly score for each user. Fisher’s method iscommonly used to combine p-values obtained from indepen-dent tests into the single test statisticXu = −2m∑i=1log(pui).Under the null hypothesis that the model is correct, Xu ∼χ22m. Outlying behaviors correspond to large values of Xu,and so a single combined p-valuepu = Pr(Xu > xu). (3)from the upper tail of χ22m represents a measure of surprisefor each user u.IV. ANALYSISA. Los Alamos National Laboratory network host logsThe data set used for analysis is taken from an internalcollection of host logs over a two month period from comput-ers running the Microsoft Windows operating system on LosAlamos National Laboratory’s (LANL) enterprise network.The data are available from http://csr.lanl.gov/data/cyber1/ and[7] provides a detailed description of the data.In the test month of data considered here, there are 91known user credentials which were compromised during amonth-long red team exercise within the LANL network. Theaim of the analysis is to detect these compromised credentialsusing the Poisson factorization approach.In particular, two features of user behavior will be analyzed:the processes invoked by the user, and the computers in thenetwork on which they authenticated. For this analysis, interestfocuses on what [6] refers to as the “implicit” data, wherebyobservation of the count matrix Y is treated as censored [8],recording only whether Yui = 0 or Yui > 0. Two data sets areconsidered:• User-Process, a binary matrix with n = 8,786 users andm = 11,571 processes. There are 360,065 observationsover a 1 month training period and 385,631 observationsover a 1 month test period when the red team exerciseoccurred. For the process names some standardizationwas performed whereby version numbers were removedfrom the process name so that processes running withdifferent versions were mapped to the same process.• User-Authentication, a binary matrix with n = 9,232users and m = 12,750 computers with a total of 69,697observations over the 1 month training period and 69,526observations over the 1 month test period when the redteam exercise occurred.Any users or items that were present in the test period andnot in the training period were removed from analysis. Infuture work, the inference procedure should be extended todeal with the arrival of new items, such as utilizing content-based recommendation algorithms as in [9].The variational inference algorithm requires a validationset to determine convergence of the algorithm, so following[6] 1% of all training observations were set aside. Note thatunlike traditional test sets used for recommender systems,observations in the test set will overlap those in the trainingset.B. Anomaly detection resultsPrior parameter settings were chosen so as to maximize theposterior predictive likelihood on the held out validation setresulting in the number of latent variables k = 10 for the User-Authentication data set, k = 50 for the User-Process data setand the prior parameters specified in (1) as a = b = a′ = c′ =.5, b′ = d′ = .01.For evaluation of the Poisson model fit, an N -precisionstatistic is calculated for each user in the test set, with the91 known compromised credentials removed. For each user alist of the top N recommendations are generated, ordered interms of the dot products θu · βi, i = 1, . . . ,m. The precisionfor user u is then the proportion of those recommendationswhich are subsequently observed during the test period.Figure 1 shows the average N -precision, for different N ,across the users as a function of user activity for both the User-Authentication and User-Process data. As might be expected,the precision for users who are least active is much worse thanusers with a higher activity level. The precision performanceis better for recommending processes than authentications; areason for this could be that the most common processes willbe run by almost all users; whereas the machines which usersauthenticate to will be much more sparse and diverse. Thiscan be seen in Figure 2, which compares the popularity ofdifferent processes and the machines that users authenticateon.10 20 30 40 50 60 70 80 90 100User Authentication Activity Percentile0.00.10.20.30.40.50.60.70.8Mean Precisiontop­10top­25top­50top­1000 20 40 60 80 100User Process Activity Percentile0.00.20.40.60.81.0Mean PrecisionFig. 1. Mean precision for users with varying levels of activity100 101 102 103 104Number of Authentications100101102103104Number of Machines100 101 102 103 104Number of Processes100101102103104Number of MachinesFig. 2. Log-log plot of the empirical distribution of the popularity of processesand machines which users authenticate on.0.0 0.2 0.4 0.6 0.8 1.0False Positive Rate0.00.20.40.60.81.0True Positive RateAuthenticationsProcessesCombinedFig. 3. Red team ROC curve for the processes, authentications and combinedscored.Finally, Figure 3 shows the ROC curve for the sequenceof p-values (3) for the User-Process data, User-Authenticationdata, and the combined score, where the combined score istaken as the average of the p-values of from the process dataand the authentication data. The number of true positives inthe top N most anomalous users, as a function of N , is givenin Table I. The performance is best for the User-Authenticationdata, indicating much better signal for this particular red-teamexercise in looking at which machines users authenticated on.The goal of a red-team exercise is often to steal privilegeduser credentials and traverse the network, which would resultin more signal in looking at machines that users authenticatedon, rather than the processes they ran.C. ConclusionA collaborative filtering approach based on Poisson factor-ization is proposed for peer-based anomaly detection of usersTABLE ITop N detectionsTrue positives 5 10 25 50 100Processes 0 1 1 3 4Authentications 3 4 12 17 25Combined 1 2 5 9 22in an enterprise network. The methodology is shown to per-form well in detecting compromised user credentials. Futurework is to extend the model using content-based filtering fornew items observed in the network as mentioned in SectionIV-A; utilizing content would also enhance performance forpredicting known processes as the characteristics of a process,such as its parent process or its source directory will clusteritems with similar properties. One problem associated withrecommender systems is that they struggle to predict itemsused by only one individual in the network; as can be seenin Figure 1, this is a common attribute of these data wherethere are many processes and machines used by only a singleuser. Future research will seek to use the posterior parameterslearned using the recommender system as an informative priorfor a multinomial-Dirichlet model on users and items, as in[10]. The multinomial-Dirichlet distribution can provide anextra layer to model an individual user’s profile, and theinformative prior from the recommender system should givebetter predictive probabilities for new items for the user basedon the behavior of similar users.REFERENCES[1] Verizon, “2016 data breach investigations report,” Tech. Rep., 2016.[2] S. H. Oh and W. S. Lee, “An anomaly intrusion detection method byclustering normal user behavior,” Computers and Security, vol. 22, no. 7,pp. 596 – 612, 2003.[3] B. Viswanath, M. A. Bashir, M. Crovella, S. Guha, K. P. Gummadi,B. Krishnamurthy, and A. Mislove, “Towards detecting anomaloususer behavior in online social networks,” in 23rd USENIX SecuritySymposium (USENIX Security 14). USENIX Association, 2014, pp.223–238.[4] S. Mukkamala, G. Janoski, and A. Sung, “Intrusion detection using neu-ral networks and support vector machines,” in Neural Networks, 2002.IJCNN ’02. Proceedings of the 2002 International Joint Conference on,vol. 2, 2002, pp. 1702–1707.[5] J. Sun, H. Qu, D. Chakrabarti, and C. Faloutsos, “Neighborhoodformation and anomaly detection in bipartite graphs,” in Fifth IEEEInternational Conference on Data Mining (ICDM’05), 2005, p. 8.[6] P. Gopalan, J. M. Hofman, and D. M. Blei, “Scalable recommendationwith poisson factorization,” CoRR, vol. abs/1311.1704, 2013.[7] A. D. Kent, “Cyber-security data sources for dynamic network research,”in Dynamic Networks in Cybersecurity. Imperial College Press, 2015.[8] W. H. Greene, “Censored data and truncated distributions,” Available atSSRN 825845, 2005.[9] P. K. Gopalan, L. Charlin, and D. Blei, “Content-based recommenda-tions with poisson factorization,” in Advances in Neural InformationProcessing Systems 27. Curran Associates, Inc., 2014, pp. 3176–3184.[10] M. J. M. Turcotte, N. A. Heard, and A. D. Kent, “Modelling user be-haviour in a network using computer event logs,” in Dynamic Networksin Cybersecurity. Imperial College Press, 2016.

Poisson factorization for peer-based anomaly detection

Turcotte, M.

Moore, J.

Heard, Na

McPhall, A.

ZENODO

http://spiral.imperial.ac.uk/bitstream/10044/1/42761/2/poisson_factorisation.pdf

Poisson factorization for peer-based anomaly detection

Abstract

Similar works

Full text

Available Versions

Spiral - Imperial College Digital Repository

ZENODO