Advanced monitoring in P2P botnets

Botnets are increasingly being held responsible for most of the cybercrimes that occur nowadays. They are used to carry out malicious activities like banking credential theft and Distributed Denial of Service (DDoS) attacks to generate profit for their owner, the botmaster. Traditional botnets utilized centralized and decentralized Command-and-Control Servers (C2s). However, recent botnets have been observed to prefer P2P-based architectures to overcome some of the drawbacks of the earlier architectures. A P2P architecture allows botnets to become more resilient and robust against random node failures and targeted attacks. However, the distributed nature of such botnets requires the defenders, i.e., researchers and law enforcement agencies, to use specialized tools such as crawlers and sensor nodes to monitor them. In return to such monitoring, botmasters have introduced various countermeasures to impede botnet monitoring, e.g., automated blacklisting mechanisms. The presence of anti-monitoring mechanisms not only render any gathered monitoring data to be inaccurate or incomplete, it may also adversely affect the success rate of botnet takedown attempts that rely upon such data. Most of the existing monitoring mechanisms identified from the related works only attempt to tolerate anti-monitoring mechanisms as much as possible, e.g., crawling bots with lower frequency. However, this might also introduce noise into the gathered data, e.g., due to the longer delay for crawling the botnet. This in turn may also reduce the quality of the data. This dissertation addresses most of the major issues associated with monitoring in P2P botnets as described above. Specifically, it analyzes the anti-monitoring mechanisms of three existing P2P botnets: 1) GameOver Zeus, 2)Sality, and 3) ZeroAccess, and proposes countermeasures to circumvent some of them. In addition, this dissertation also proposes several advanced anti-monitoring mechanisms from the perspective of a botmaster to anticipate future advancement of the botnets. This includes a set of lightweight crawler detection mechanisms as well as several novel mechanisms to detect sensor nodes deployed in P2P botnets. To ensure that the defenders do not loose this arms race, this dissertation also includes countermeasures to circumvent the proposed anti-monitoring mechanisms. Finally, this dissertation also investigates if the presence of third party monitoring mechanisms, e.g., sensors, in botnets influences the overall churn measurements. In addition, churn models for Sality and ZeroAccess are also derived using fine-granularity churn measurements. The works proposed in this dissertation have been evaluated using either real-world botnet datasets, i.e., that were gathered using crawlers and sensor nodes, or simulated datasets. Evaluation results indicate that most of the anti-monitoring mechanisms implemented by existing botnets can either be circumvented or tolerated to obtain monitoring data with a better quality. However, many crawlers and sensor nodes in existing botnets are found vulnerable to the antimonitoring mechanisms that are proposed from the perspective of a botmaster in this dissertation. Analysis of the fine-grained churn measurements for Sality and ZeroAccess indicate that churn in these botnets are similar to that of regular P2P file-sharing networks like Gnutella and Bittorent. In addition, the presence of highly responsive sensor nodes in the botnets are found not influencing the overall churn measurements. This is mainly due to low number of sensor nodes currently deployed in the botnets. Existing and future botnet monitoring mechanisms should apply the findings of this dissertation to ensure high quality monitoring data, and to remain undetected from the bots or the botmasters

AUTOMATIC FEATURE ENGINEERING FOR DISCOVERING AND EXPLAINING MALICIOUS BEHAVIORS

A key task of cybersecurity is to discover and explain malicious behaviors of malware. The understanding of malicious behaviors helps us further develop good features and apply machine learning techniques to detect various attacks. The effectiveness of machine learning techniques primarily depends on the manual feature engineering process, based on human knowledge and intuition. However, given the adversaries’ efforts to evade detection and the growing volume of publications on malicious behaviors, the feature engineering process likely draws from a fraction of the relevant knowledge. Therefore, it is necessary and important to design an automated system to engineer features for discovering malicious behaviors and detecting attacks. First, we describe a knowledge-based feature engineering technique for malware detection. It mines documents written in natural language (e.g. scientific literature), and represents and queries the knowledge about malware in a way that mirrors the human feature engineering process. We implement the idea in a system called FeatureSmith, which generates a feature set for detecting Android malware. We train a classifier using these features on a large data set of benign and malicious apps. This classifier achieves comparable performance to a state-of-the-art Android malware detector that relies on manually engineered features. In addition, FeatureSmith is able to suggest informative features that are absent from the manually engineered set and to link the features generated to abstract concepts that describe malware behaviors. Second, we propose a data-driven feature engineering technique called ReasonSmith, which explains machine learning models by ranking features based on their global importance. Instead of interpreting how neural networks make decisions for one specific sample, ReasonSmith captures general importance in terms of the whole data set. In addition, ReasonSmith allows us to efficiently identify data biases and artifacts, by comparing feature rankings over time. We further summarize the common data biases and artifacts for malware detection problems at the level of API calls. Third, we study malware detection from a global view, and explore automatic feature engineering problem in analyzing campaigns that include a series of actions. We implement a system ChainSmith to bridge large-scale field measurement and manual campaign report by extracting and categorizing IOCs (indicators of compromise) from security blogs. The semantic roles of IOCs allow us to link qualitative data (e.g. security blogs) to quantitative measurements, which brings new insights to malware campaigns. In particular, we study the effectiveness of different persuasion techniques used on enticing user to download the payloads. We find that the campaign usually starts from social engineering and “missing codec” ruse is a common persuasion technique that generates the most suspicious downloads each day

On the malware detection problem : challenges and novel approaches

Orientador: André Ricardo Abed GrégioCoorientador: Paulo Lício de GeusTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba,Inclui referênciasÁrea de concentração: Ciência da ComputaçãoResumo: Software Malicioso (malware) é uma das maiores ameaças aos sistemas computacionais atuais, causando danos à imagem de indivíduos e corporações, portanto requerendo o desenvolvimento de soluções de detecção para prevenir que exemplares de malware causem danos e para permitir o uso seguro dos sistemas. Diversas iniciativas e soluções foram propostas ao longo do tempo para detectar exemplares de malware, de Anti-Vírus (AVs) a sandboxes, mas a detecção de malware de forma efetiva e eficiente ainda se mantém como um problema em aberto. Portanto, neste trabalho, me proponho a investigar alguns desafios, falácias e consequências das pesquisas em detecção de malware de modo a contribuir para o aumento da capacidade de detecção das soluções de segurança. Mais especificamente, proponho uma nova abordagem para o desenvolvimento de experimentos com malware de modo prático mas ainda científico e utilizo-me desta abordagem para investigar quatro questões relacionadas a pesquisa em detecção de malware: (i) a necessidade de se entender o contexto das infecções para permitir a detecção de ameaças em diferentes cenários; (ii) a necessidade de se desenvolver melhores métricas para a avaliação de soluções antivírus; (iii) a viabilidade de soluções com colaboração entre hardware e software para a detecção de malware de forma mais eficiente; (iv) a necessidade de predizer a ocorrência de novas ameaças de modo a permitir a resposta à incidentes de segurança de forma mais rápida.Abstract: Malware is a major threat to most current computer systems, causing image damages and financial losses to individuals and corporations, thus requiring the development of detection solutions to prevent malware to cause harm and allow safe computers usage. Many initiatives and solutions to detect malware have been proposed over time, from AntiViruses (AVs) to sandboxes, but effective and efficient malware detection remains as a still open problem. Therefore, in this work, I propose taking a look on some malware detection challenges, pitfalls and consequences to contribute towards increasing malware detection system's capabilities. More specifically, I propose a new approach to tackle malware research experiments in a practical but still scientific manner and leverage this approach to investigate four issues: (i) the need for understanding context to allow proper detection of localized threats; (ii) the need for developing better metrics for AV solutions evaluation; (iii) the feasibility of leveraging hardware-software collaboration for efficient AV implementation; and (iv) the need for predicting future threats to allow faster incident responses