Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data

Recent years have seen the rise of more sophisticated attacks including advanced persistent threats (APTs) which pose severe risks to organizations and governments by targeting confidential proprietary information. Additionally, new malware strains are appearing at a higher rate than ever before. Since many of these malware are designed to evade existing security products, traditional defenses deployed by most enterprises today, e.g., anti-virus, firewalls, intrusion detection systems, often fail at detecting infections at an early stage. We address the problem of detecting early-stage infection in an enterprise setting by proposing a new framework based on belief propagation inspired from graph theory. Belief propagation can be used either with "seeds" of compromised hosts or malicious domains (provided by the enterprise security operation center -- SOC) or without any seeds. In the latter case we develop a detector of C&C communication particularly tailored to enterprises which can detect a stealthy compromise of only a single host communicating with the C&C server. We demonstrate that our techniques perform well on detecting enterprise infections. We achieve high accuracy with low false detection and false negative rates on two months of anonymized DNS logs released by Los Alamos National Lab (LANL), which include APT infection attacks simulated by LANL domain experts. We also apply our algorithms to 38TB of real-world web proxy logs collected at the border of a large enterprise. Through careful manual investigation in collaboration with the enterprise SOC, we show that our techniques identified hundreds of malicious domains overlooked by state-of-the-art security products

Database Intrusion Detection Using Role Profiling

Insider threats cause the majority of computer system security problems and are also among the most challenging research topics in database security. An anomaly-based intrusion detection system (IDS), which can profile inside users’ normal behaviors and detect anomalies when a user’s behaviors deviate from his/her profiles, is effective to protect computer systems against insider threats since the IDS can profile each insider and then monitor them continuously. Although many IDSes have been developed at the network or host level since 1980s, there are still very few IDSes specifically tailored to database systems. We initially build our anomaly-based database IDS using two different profiling methods: one is to build profiles for each individual user (user profiling) and the other is to mine profiles for roles (role profiling). Detailed comparative evaluations between role profiling and user profiling are conducted, and we also analyze the reasons why role profiling is more effective and efficient than user profiling. Another contribution of this thesis is that we introduce role hierarchy into database IDS and remarkably reduce the false positive rate without increasing the false negative rate