A Novel Data Engineering Process Which Integrates Alert Information, Security Logs, And SOC Analysts

Abstract

We build up a user centric ML system for the cyber security operation center in endeavor environment. We examine the regular data sources in SOC, their work process, and how to leverage and procedure these data sets to construct an effective ML system. The work is besieged towards two groups of readers. The primary group is data scientists or ML researchers who do not have cyber security domain awareness but want to build ML systems for safety operations center. The second group of people is those cyber security practitioners who have deep information and expertise in cyber security, but do not have ML knowledge and wish to construct one by them. All through the work, we use the system we built in the Symantec SOC construction setting as an example to display the full steps from data collection, label creation, feature engineering, ML algorithm selection, and model show evaluations, to risk score making