Anomaly detection via high-dimensional data analysis on web access data.

Abstract

Suen, Ho Yan.Thesis (M.Phil.)--Chinese University of Hong Kong, 2009.Includes bibliographical references (leaves 99-104).Abstract also in Chinese.Abstract --- p.iAcknowledgement --- p.ivChapter 1 --- Introduction --- p.1Chapter 1.1 --- Motivation --- p.1Chapter 1.2 --- Organization --- p.4Chapter 2 --- Literature Review --- p.6Chapter 2.1 --- Related Works --- p.6Chapter 2.2 --- Background Study --- p.7Chapter 2.2.1 --- World Wide Web --- p.7Chapter 2.2.2 --- Distributed Denial of Service Attack --- p.11Chapter 2.2.3 --- Tools for Dimension Reduction --- p.13Chapter 2.2.4 --- Tools for Anomaly Detection --- p.20Chapter 2.2.5 --- Receiver operating characteristics (ROC) Analysis --- p.22Chapter 3 --- System Design --- p.25Chapter 3.1 --- Methodology --- p.25Chapter 3.2 --- System Overview --- p.27Chapter 3.3 --- Reference Profile Construction --- p.31Chapter 3.4 --- Real-time Anomaly Detection and Response --- p.32Chapter 3.5 --- Chapter Summary --- p.34Chapter 4 --- Reference Profile Construction --- p.35Chapter 4.1 --- Web Access Logs Collection --- p.35Chapter 4.2 --- Data Preparation --- p.37Chapter 4.3 --- Feature Extraction and Embedding Engine (FEE Engine) --- p.40Chapter 4.3.1 --- Sub-Sequence Extraction --- p.42Chapter 4.3.2 --- Hash Function on Sub-sequences (optional) --- p.45Chapter 4.3.3 --- Feature Vector Construction --- p.46Chapter 4.3.4 --- Diffusion Wavelets Embedding --- p.47Chapter 4.3.5 --- Numerical Example of Feature Set Reduction --- p.49Chapter 4.3.6 --- Reference Profile and Further Use of FEE Engine --- p.50Chapter 4.4 --- Chapter Summary --- p.50Chapter 5 --- Real-time Anomaly Detection and Response --- p.52Chapter 5.1 --- Session Filtering and Data Preparation --- p.54Chapter 5.2 --- Feature Extraction and Embedding --- p.54Chapter 5.3 --- Distance-based Outlier Scores Calculation --- p.55Chapter 5.4 --- Anomaly Detection and Response --- p.56Chapter 5.4.1 --- Length-Based Anomaly Detection Modules --- p.56Chapter 5.4.2 --- Characteristics of Anomaly Detection Modules --- p.59Chapter 5.4.3 --- Dynamic Threshold Adaptation --- p.60Chapter 5.5 --- Chapter Summary --- p.63Chapter 6 --- Experimental Results --- p.65Chapter 6.1 --- Experiment Datasets --- p.65Chapter 6.1.1 --- Normal Web Access Logs --- p.66Chapter 6.1.2 --- Attack Data Generation --- p.68Chapter 6.2 --- ROC Curve Construction --- p.70Chapter 6.3 --- System Parameters Selection --- p.71Chapter 6.4 --- Performance of Anomaly Detection --- p.82Chapter 6.4.1 --- Performance Analysis --- p.85Chapter 6.4.2 --- Performance in defending DDoS attacks --- p.87Chapter 6.5 --- Computation Requirement --- p.91Chapter 6.6 --- Chapter Summary --- p.95Chapter 7 --- Conclusion and Future Work --- p.96Bibliography --- p.9