Compression of Virtual-Machine Memory in Dynamic Malware Analysis

Lossless compression of memory dumps from virtual machines that run malware samples is considered with the goal of significantly reducing archival costs in dynamic-malware-analysis applications. Given that, in such dynamic-analysis scenarios, malware samples are typically run in virtual machines just long enough to activate any self-decryption or other detection- avoidance maneuvers, the virtual-machine memory typically changes little from that of the baseline state, with the difference being attributable in large degree to the loading of additional executables and libraries. Consequently, delta coding is proposed to compress the current virtual-machine memory dump by coding its differences with respect to a predicted memory image formed by loading the same executables and libraries into the baseline memory. Experimental results reveal a significant improvement in compression efficiency as compared to straightforward delta encoding without such predictive executable / library loading

A Framework for Analyzing Advanced Malware and Software

Vulnerabilities in software, whether they be malicious or benign are a major concern in every sector. My research broadly focused on security testing of software, including malware. For the last few years, ransomware attacks have become increasingly prevalent with the growth of cryptocurrencies.The first part of my research presents a strategy to recover from ransomware attacks by backing up critical information in slack space. In this work, I designed RDS3, a novel ransomware defense strategy, in which we stealthily back up data in the spare space of a computing device, such that the data encrypted by ransomware can be restored. The key concept is that unused space can backup critical data, which is fully isolated from the system. In this way, no ransomware will be able to \u27\u27touch\u27\u27 the backup data regardless of what privilege it is able to obtain.Next, my research focused on understanding ransomware from both structural and behavioral perspectives to design CRDETECTOR, crypto-ransomware detector. Reverse engineering is performed on executables at different levels such as raw binaries, assembly codes, libraries, and function calls to better analysis and interpret the purpose of code segments. In this work, I applied data-mining techniques to correlate multi-level code components (derived from reverse engineering process) to find unique signatures to identify ransomware families.As part of security testing of software, I conducted research on InfiniBand (IB) which supports remote direct memory access without making two copies of data (one in user space and the other in kernel space) and thus provides very low latency and very high throughput. To this end, for many industries, IB has become a promising new inter-connect protocol over Ethernet technologies and ensuring the security of is critical. To do this, the first step is to have a thorough understanding of the vulnerabilities of its current implementations, which is unfortunately still missing in the literature. While my extensive penetration testing could not find any significant security loopholes, there are certain aspects in both the design and the implementations that need to be addressed

Hardware Mechanisms for Efficient Memory System Security

The security of a computer system hinges on the trustworthiness of the operating system and the hardware, as applications rely on them to protect code and data. As a result, multiple protections for safeguarding the hardware and OS from attacks are being continuously proposed and deployed. These defenses, however, are far from ideal as they only provide partial protection, require complex hardware and software stacks, or incur high overheads. This dissertation presents hardware mechanisms for efficiently providing strong protections against an array of attacks on the memory hardware and the operating system’s code and data. In the first part of this dissertation, we analyze and optimize protections targeted at defending memory hardware from physical attacks. We begin by showing that, contrary to popular belief, current DDR3 and DDR4 memory systems that employ memory scrambling are still susceptible to cold boot attacks (where the DRAM is frozen to give it sufficient retention time and is then re-read by an attacker after reboot to extract sensitive data). We then describe how memory scramblers in modern memory controllers can be transparently replaced by strong stream ciphers without impacting performance. We also demonstrate how the large storage overheads associated with authenticated memory encryption schemes (which enable tamper-proof storage in off-chip memories) can be reduced by leveraging compact integer encodings and error-correcting code (ECC) DRAMs – without forgoing the error detection and correction capabilities of ECC DRAMs. The second part of this dissertation presents Neverland: a low-overhead, hardware-assisted, memory protection scheme that safeguards the operating system from rootkits and kernel-mode malware. Once the system is done booting, Neverland’s hardware takes away the operating system’s ability to overwrite certain configuration registers, as well as portions of its own physical address space that contain kernel code and security-critical data. Furthermore, it prohibits the CPU from fetching privileged code from any memory region lying outside the physical addresses assigned to the OS kernel and drivers. This combination of protections makes it extremely hard for an attacker to tamper with the kernel or introduce new privileged code into the system – even in the presence of software vulnerabilities. Neverland enables operating systems to reduce their attack surface without having to rely on complex integrity monitoring software or hardware. The hardware mechanisms we present in this dissertation provide building blocks for constructing a secure computing base while incurring lower overheads than existing protections.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147604/1/salessaf_1.pd

Techniques for the reverse engineering of banking malware

Malware attacks are a significant and frequently reported problem, adversely affecting the productivity of organisations and governments worldwide. The well-documented consequences of malware attacks include financial loss, data loss, reputation damage, infrastructure damage, theft of intellectual property, compromise of commercial negotiations, and national security risks. Mitiga-tion activities involve a significant amount of manual analysis. Therefore, there is a need for automated techniques for malware analysis to identify malicious behaviours. Research into automated techniques for malware analysis covers a wide range of activities. This thesis consists of a series of studies: an anal-ysis of banking malware families and their common behaviours, an emulated command and control environment for dynamic malware analysis, a technique to identify similar malware functions, and a technique for the detection of ransomware. An analysis of the nature of banking malware, its major malware families, behaviours, variants, and inter-relationships are provided in this thesis. In doing this, this research takes a broad view of malware analysis, starting with the implementation of the malicious behaviours through to detailed analysis using machine learning. The broad approach taken in this thesis differs from some other studies that approach malware research in a more abstract sense. A disadvantage of approaching malware research without domain knowledge, is that important methodology questions may not be considered. Large datasets of historical malware samples are available for countermea-sures research. However, due to the age of these samples, the original malware infrastructure is no longer available, often restricting malware operations to initialisation functions only. To address this absence, an emulated command and control environment is provided. This emulated environment provides full control of the malware, enabling the capabilities of the original in-the-wild operation, while enabling feature extraction for research purposes. A major focus of this thesis has been the development of a machine learn-ing function similarity method with a novel feature encoding that increases feature strength. This research develops techniques to demonstrate that the machine learning model trained on similarity features from one program can find similar functions in another, unrelated program. This finding can lead to the development of generic similar function classifiers that can be packaged and distributed in reverse engineering tools such as IDA Pro and Ghidra. Further, this research examines the use of API call features for the identi-fication of ransomware and shows that a failure to consider malware analysis domain knowledge can lead to weaknesses in experimental design. In this case, we show that existing research has difficulty in discriminating between ransomware and benign cryptographic software. This thesis by publication, has developed techniques to advance the disci-pline of malware reverse engineering, in order to minimize harm due to cyber-attacks on critical infrastructure, government institutions, and industry.Doctor of Philosoph

APTs way: evading Your EBNIDS

APTs and government-supported attackers use a broad arsenal of techniques to avoid having their exploits detected by IDSes. Signature Based IDSes are not efficient against nation-state-sponsored attackers which use custom shellcode encoders in an exploit. Emulation Based NIDSes (EBNIDS) have been proposed as a solution to mitigate such attacks. EBNISes detect a suspicious network stream (pre-processing) and after converting them to emulate-able byte sequences run it in an instrumented environment (Emulation), finally matching the behavior with certain heuristics (Heuristics Detection). In this talk, we will present novel ways that an APT might use to circumvente the Pre-Processing, Emulation and Heuristic Detection steps of EBNIDSes by employing a wide range of evasion techniques

Effectiveness of Similarity Digest Algorithms for Binary Code Similarity in Memory Forensic Analysis

Hoy en dı́a, cualquier organización que esté conectada a Internet es susceptible de sufrir incidentes de ciberseguridad y por tanto, debe contar con un plan de respuesta a incidentes. Este plan ayuda a prevenir, detectar, priorizar y gestionar los incidentes de ciberseguridad. Uno de los pasos para gestionar estos incidentes es la fase de eliminación, que se encarga de neutralizar la persistencia de los ataques, evaluar el alcance de los mismos e identificar el grado de compromiso. Uno de los puntos clave de esta fase es la identicación mediante triaje de la información que es relevante en el incidente. Esto suele hacerse comparando los elementos disponibles con información conocida, centrándose ası́ en aquellos elementos que tienen relevancia para la investigación (llamados evidencias).Este objetivo puede alcanzarse estudiando dos fuentes de información. Por un lado, mediante el análisis de los datos persistentes, como los datos de los discos duros o los dispositivos USB. Por otro lado, mediante el análisis de los datos volátiles, como los datos de la memoria RAM. A diferencia del análisis de datos persistentes, el análisis de datos volátiles permite determinar el alcance de algunos tipos de ataque que no guardan su código en dispositivos de persistencia o cuando los archivos ejecutables almacenados en el disco están cifrados; cuyo código sólo se muestra cuando está en la memoria y se está ejecutado.Existe una limitación en el uso de hashes criptográficos, comúnmente utilizados en el caso de identificación de evidencias en datos persistentes, para identificar evidencias de memoria. Esta limitación se debe a que las evidencias nunca serán idénticas porque la ejecución modifica el contenido de la memoria constantemente. Además, es imposible adquirir la memoria más de una vez con todos los programas en el mismo punto de ejecución. Por lo tanto, los hashes son un método de identificación inválido para el triaje de memoria. Como solución a este problema, en esta tesis se propone el uso de algoritmos de similitud de digest, que miden la similitud entre dos entradas de manera aproximada.Las principales aportaciones de esta tesis son tres. En primer lugar, se realiza un estudio del dominio del problema en el que se evalúa la gestión de la memoria y la modificación de la misma en ejecución. A continuación, se estudian los algoritmos de similitud de digest, desarrollando una clasificación de sus fases y de los ataques contra estos algoritmos, correlacionando las caracterı́sticas de la primera clasificación con los ataques identificados. Por último, se proponen dos métodos de preprocesamiento del contenido de volcados de memoria para mejorar la identificación de los elementos de interés para el análisis.Como conclusión, en esta tesis se muestra que la modificación de bytes dispersos afecta negativamente a los cálculos de similitud entre evidencias de memoria. Esta modificación se produce principalmente por el gestor de memoria del sistema operativo. Además, se muestra que las técnicas propuestas para preprocesar el contenido de volcados de memoria permiten mejorar el proceso de identificación de evidencias en memoria.<br /

REMOTE MOBILE SCREEN (RMS): AN APPROACH FOR SECURE BYOD ENVIRONMENTS

Bring Your Own Device (BYOD) is a policy where employees use their own personal mobile devices to perform work-related tasks. Enterprises reduce their costs since they do not have to purchase and provide support for the mobile devices. BYOD increases job satisfaction and productivity in the employees, as they can choose which device to use and do not need to carry two or more devices. However, BYOD policies create an insecure environment, as the corporate network is extended and it becomes harder to protect it from attacks. In this scenario, the corporate information can be leaked, personal and corporate spaces are not separated, it becomes difficult to enforce security policies on the devices, and employees are worried about their privacy. Consequently, a secure BYOD environment must achieve the following goals: space isolation, corporate data protection, security policy enforcement, true space isolation, non-intrusiveness, and low resource consumption. We found that none of the currently available solutions achieve all of these goals. We developed Remote Mobile Screen (RMS), a framework that meets all the goals for a secure BYOD environment. To achieve this, the enterprise provides the employee with a Virtual Machine (VM) running a mobile operating system, which is located in the enterprise network and to which the employee connects using the mobile device. We provide an implementation of RMS using commonly available software for an x86 architecture. We address RMS challenges related to compatibility, scalability and latency. For the first challenge, we show that at least 90.2% of the productivity applications from Google Play can be installed on an x86 architecture, while at least 80.4% run normally. For the second challenge, we deployed our implementation on a high-performance server and run up to 596 VMs using 256 GB of RAM. Further, we show that the number of VMs is proportional to the available RAM. For the third challenge, we used our implementation on GENI and conclude that an application latency of 150 milliseconds can be achieved. Adviser: Byrav Ramamurth

Analysis avoidance techniques of malicious software

Anti Virus (AV) software generally employs signature matching and heuristics to detect the presence of malicious software (malware). The generation of signatures and determination of heuristics is dependent upon an AV analyst having successfully determined the nature of the malware, not only for recognition purposes, but also for the determination of infected files and startup mechanisms that need to be removed as part of the disinfection process. If a specimen of malware has not been previously extensively analyzed, it is unlikely to be detected by AV software. In addition, malware is becoming increasingly profit driven and more likely to incorporate stealth and deception techniques to avoid detection and analysis to remain on infected systems for a myriad of nefarious purposes. Malware extends beyond the commonly thought of virus or worm, to customized malware that has been developed for specific and targeted miscreant purposes. Such customized malware is highly unlikely to be detected by AV software because it will not have been previously analyzed and a signature will not exist. Analysis in such a case will have to be conducted by a digital forensics analyst to determine the functionality of the malware. Malware can employ a plethora of techniques to hinder the analysis process conducted by AV and digital forensics analysts. The purpose of this research has been to answer three research questions directly related to the employment of these techniques as: 1. What techniques can malware use to avoid being analyzed? 2. How can the use of these techniques be detected? 3. How can the use of these techniques be mitigated