Machine learning and neural networks have become increasingly popular
solutions for encrypted malware traffic detection. They mine and learn complex
traffic patterns, enabling detection by fitting boundaries between malware
traffic and benign traffic. Compared with signature-based methods, they have
higher scalability and flexibility. However, affected by the frequent variants
and updates of malware, current methods suffer from a high false positive rate
and do not work well for unknown malware traffic detection. It remains a
critical task to achieve effective malware traffic detection. In this paper, we
introduce CBSeq to address the above problems. CBSeq is a method that
constructs a stable traffic representation, behavior sequence, to characterize
attacking intent and achieve malware traffic detection. We novelly propose the
channels with similar behavior as the detection object and extract side-channel
content to construct behavior sequence. Unlike benign activities, the behavior
sequences of malware and its variant's traffic exhibit solid internal
correlations. Moreover, we design the MSFormer, a powerful Transformer-based
multi-sequence fusion classifier. It captures the internal similarity of
behavior sequence, thereby distinguishing malware traffic from benign traffic.
Our evaluations demonstrate that CBSeq performs effectively in various known
malware traffic detection and exhibits superior performance in unknown malware
traffic detection, outperforming state-of-the-art methods.Comment: Submitted to IEEE TIF