Binary Analysis Framework

The binary analysis of software has become an integral activity for security researchers and attackers alike. As the value of being able to exploit a vulnerability has increased, the need to discover, fix and prevent such vulnerabilities has never been greater. This paper proposes the Binary Analysis Framework, which is intended to be used by security researchers to query and analyze information about system and third party libraries. Researchers can use the tool to evaluate and discover unknown vulnerabilities in these libraries. Furthermore, the framework can be utilized to analyze mitigation techniques implemented by operating system and thirdparty vendors. The Binary Analysis Framework takes a novel approach to system-level security by introducing a framework that provides for binary analysis of libraries utilizing a relational data model for permanent storage of the binary instructions, as well as providing novel ways of searching and interacting with the parsed instructions

Neutral Networks of Real-World Programs and their Application to Automated Software Evolution

The existing software development ecosystem is the product of evolutionary forces, and consequently real-world software is amenable to improvement through automated evolutionary techniques. This dissertation presents empirical evidence that software is inherently robust to small randomized program transformations, or \u27mutations. Simple and general mutation operations are demonstrated that can be applied to software source code, compiled assembler code, or directly to binary executables. These mutations often generate variants of working programs that differ significantly from the original, yet remain fully functional. Applying successive mutations to the same software program uncovers large \u27neutral networks\u27 of fully functional variants of real-world software projects. These properties of \u27mutational robustness\u27 and the corresponding \u27neutral networks\u27 have been studied extensively in biology and are believed to be related to the capacity for unsupervised evolution and adaptation. As in biological systems, mutational robustness and neutral networks in software systems enable automated evolution. The dissertation presents several applications that leverage software neutral networks to automate common software development and maintenance tasks. Neutral networks are explored to generate diverse implementations of software for improving runtime security and for proactively repairing latent bugs. Next, a technique is introduced for automatically repairing bugs in the assembler and executables compiled from off-the-shelf software. As demonstration, a proprietary executable is manipulated to patch security vulnerabilities without access to source code or any aid from the software vendor. Finally, software neutral networks are leveraged to optimize complex nonfunctional runtime properties. This optimization technique is used to reduce the energy consumption of the popular PARSEC benchmark applications by 20% as compared to the best available public domain compiler optimizations. The applications presented herein apply evolutionary computation techniques to existing software using common software engineering tools. By enabling evolutionary techniques within the existing software development toolchain, this work is more likely to be of practical benefit to the developers and maintainers of real-world software systems

A fast and scalable binary similarity method for open source libraries

Abstract. Usage of third party open source software has become more and more popular in the past years, due to the need for faster development cycles and the availability of good quality libraries. Those libraries are integrated as dependencies and often in the form of binary artifacts. This is especially common in embedded software applications. Dependencies, however, can proliferate and also add new attack surfaces to an application due to vulnerabilities in the library code. Hence, the need for binary similarity analysis methods to detect libraries compiled into applications. Binary similarity detection methods are related to text similarity methods and build upon the research in that area. In this research we focus on fuzzy matching methods, that have been used widely and successfully in text similarity analysis. In particular, we propose using locality sensitive hashing schemes in combination with normalised binary code features. The normalization allows us to apply the similarity comparison across binaries produced by different compilers using different optimization flags and being build for various machine architectures. To improve the matching precision, we use weighted code features. Machine learning is used to optimize the feature weights to create clusters of semantically similar code blocks extracted from different binaries. The machine learning is performed in an offline process to increase scalability and performance of the matching system. Using above methods we build a database of binary similarity code signatures for open source libraries. The database is utilized to match by similarity any code blocks from an application to known libraries in the database. One of the goals of our system is to facilitate a fast and scalable similarity matching process. This allows integrating the system into continuous software development, testing and integration pipelines. The evaluation shows that our results are comparable to other systems proposed in related research in terms of precision while maintaining the performance required in continuous integration systems.Nopea ja skaalautuva käännettyjen ohjelmistojen samankaltaisuuden tunnistusmenetelmä avoimen lähdekoodin kirjastoille. Tiivistelmä. Kolmansien osapuolten kehittämien ohjelmistojen käyttö on yleistynyt valtavasti viime vuosien aikana nopeutuvan ohjelmistokehityksen ja laadukkaiden ohjelmistokirjastojen tarjonnan kasvun myötä. Nämä kirjastot ovat yleensä lisätty kehitettävään ohjelmistoon riippuvuuksina ja usein jopa käännettyinä binääreinä. Tämä on yleistä varsinkin sulatetuissa ohjelmistoissa. Riippuvuudet saattavat kuitenkin luoda uusia hyökkäysvektoreita kirjastoista löytyvien haavoittuvuuksien johdosta. Nämä kolmansien osapuolten kirjastoista löytyvät haavoittuvuudet synnyttävät tarpeen tunnistaa käännetyistä binääriohjelmistoista löytyvät avoimen lähdekoodin ohjelmistokirjastot. Binäärien samankaltaisuuden tunnistusmenetelmät usein pohjautuvat tekstin samankaltaisuuden tunnistusmenetelmiin ja hyödyntävät tämän tieteellisiä saavutuksia. Tässä tutkimuksessa keskitytään sumeisiin tunnistusmenetelmiin, joita on käytetty laajasti tekstin samankaltaisuuden tunnistamisessa. Tutkimuksessa hyödynnetään sijainnille sensitiivisiä tiivistemenetelmiä ja normalisoituja binäärien ominaisuuksia. Ominaisuuksien normalisoinnin avulla binäärien samankaltaisuutta voidaan vertailla ohjelmiston kääntämisessä käytetystä kääntäjästä, optimisaatiotasoista ja prosessoriarkkitehtuurista huolimatta. Menetelmän tarkkuutta parannetaan painotettujen binääriominaisuuksien avulla. Koneoppimista hyödyntämällä binääriomisaisuuksien painotus optimoidaan siten, että samankaltaisista binääreistä puretut ohjelmistoblokit luovat samankaltaisien ohjelmistojen joukkoja. Koneoppiminen suoritetaan erillisessä prosessissa, mikä parantaa järjestelmän suorituskykyä. Näiden menetelmien avulla luodaan tietokanta avoimen lähdekoodin kirjastojen tunnisteista. Tietokannan avulla minkä tahansa ohjelmiston samankaltaiset binääriblokit voidaan yhdistää tunnettuihin avoimen lähdekoodin kirjastoihin. Menetelmän tavoitteena on tarjota nopea ja skaalautuva samankaltaisuuden tunnistus. Näiden ominaisuuksien johdosta järjestelmä voidaan liittää osaksi ohjelmistokehitys-, integraatioprosesseja ja ohjelmistotestausta. Vertailu muihin kirjallisuudessa esiteltyihin menetelmiin osoittaa, että esitellyn menetlmän tulokset on vertailtavissa muihin kirjallisuudessa esiteltyihin menetelmiin tarkkuuden osalta. Menetelmä myös ylläpitää suorituskyvyn, jota vaaditaan jatkuvan integraation järjestelmissä

Bagheera: an advanced polimorphic and infection engine for Linux

Computer viruses have been evolving since the '80s, adopting new techniques with the intention of avoiding being detected by anti-virus programs. One of these techniques is polymorphism, which is used to change the virus' structure each time an infection is carried out. This technique was broadly adopted by the virus-writing community and led to the birth of Polymorphic Engines, which can grant polymorphism to any virus. This project focuses on the study of those engines and, in particular, on exploring the techniques used to avoid detection from anti-viruses. In addition, this project also focuses on the analysis and development of techniques to infect ELF binaries on Linux platforms. The final goal is to design and build a modern polymorphic and infection engine, namely Bagheera, and to evaluate its effectiveness against a state of the art anti-virus in a Linux platform

Static Binary Rewriting for ROP Gadget Removal

Return-Oriented Programming (ROP) is an exploitation technique that is commonly used by malicious users. It works by leveraging return statements in binaries to gain control over the execution of programs. Some mitigations for ROP include changing the binary during compilation time, rewriting the binary after compilation, and adding runtime checks to the binary. The focus of this study was rewriting the binary after compilation. Rewriting during compilation time requires end users to have access to source code, which, in most cases, they will not. Adding runtime checks adds additional overhead to the target binary. The areas this study aimed to improve in the binary rewriting space were twofold. The first was improving static binary rewriting. This was done by attempting to see if the amount of information needed to correctly rewrite a binary could be reduced compared to other tools. The second area was attempting to use static binary rewriting to reduce the number of potential ROP gadgets in a binary. The ROP gadgets that were targeted were those created by splitting an instruction that contains a return in them to create new ROP gadgets. This was chosen because most current tools focused on the safety of standard returns from function ends. To determine if static binary rewriting could be used to reduce the amount of ROP gadgets created from mid-instruction ROP gadgets, a design science approach was taken. There were two artifacts that were created through two design cycles. The first artifact aimed to create a static binary rewriter that collected minimal amount of information from binaries. The second artifact built upon the first artifact and attempted to use it to remove instructions that contained a mid-instruction return. After the removal of the mid-instruction return, the second artifact inserted instructions that allowed for the same functionality of the binary, but without the return byte

Methods to improve debug flow for intellectual property protection

Abstract. Every company wants to protect their intellectual property and limit customer visibility of confidential information. A company may protect its proprietary information by different ways. This thesis will compare different methods that try to protect intellectual property while maintaining the software debugging capability. Working with binary libraries without debug information makes customer support very difficult. When a company is developing a new product, time to market is important. Usually, the last months are very busy resolving urgent customer issues. Especially during this period, the slow process of debugging customer issues without debug information can cause delays and increase time to market. The goal of this thesis is to compare methods that protects intellectual property by making reverse engineering more difficult. Study of the upcoming GNU Compiler Collection (GCC) features related to debug data formats, such as DWARF5, is also carried out while working with the thesis. The approaches tried were split DWARF, injecting ELF files, stripping debug data, and code obfuscation. Also optimisation and their effect on disassembly was studied. The best solution was to compile the software with debug symbols and strip them to a separate file. This way the symbol data can be loaded separately into GDB. The symbol data layout and addresses are also always correct with the solution.Virheiden etsinnän työnkulun parantaminen immateriaaliomaisuudet huomioiden. Tiivistelmä. Yritykset haluavat suojella immateriaaliomaisuuksiaan ja rajoittaa asiakkaiden näkyvyyttä tietylle tasolle asti. Tämä lopputyö vertailee eri metodeja jotka koittavat suojata immateriaaliomaisuuksia, ilman että ohjelmiston virheidenkorjattavuus kärsii. Binäärikirjastot ilman virheenkorjaustietoja vaikeuttavat asiakkaan tukemista. Uutta tuotetta kehitettäessä, markkinoille tuloaika on yritykselle tärkeää. Yleensä viimeiset kuukaudet ovat kiireisiä asiakkaan ongelmien tutkimuksien kanssa ja kyseiset ongelmat tulisi olla ratkaistuna mahdollisimman nopeasti. Tämän lopputyön tavoitteena on vertailla mahdollisia metodeja, jotka suojaavat immateriaaliomaisuutta takaisinmallinnusta vastaan. Tarkoituksena on myös tutkia tulevia GNU kääntäjä-kokoelman (GCC:n) ominaisuuksia liittyen virheenkorjaustietoformaatteihin, kuten DWARF5. Ongelman ratkaisuun koitettiin pilkottuja virheenkorjaustietoja, ELFtiedoston injektointia, virheenkorjaustiedon riisumista ohjelmistosta ja koodin obfuskointia. Myös optimoinnin vaikutusta konekielestä takaisinmallinnettuun Assembly-muotoon tutkittiin. Paras ratkaisu oli kääntää ohjelmisto virheenkorjaustiedolla ja riisua ne omaan erilliseen tiedostoon. Näin ohjelmiston symbolitieto pystytään latamaan erikseen virheenjäljittemänä käytettyyn GNU Debuggeriin (GDB:hen). Näin symbolitietojen rakenne ja osoitteet ovat myös aina paikkansapitävät

Automated Analysis of ARM Binaries using the Low-Level Virtual Machine Compiler Framework

Binary program analysis is a critical capability for offensive and defensive operations in Cyberspace. However, many current techniques are ineffective or time-consuming and few tools can analyze code compiled for embedded processors such as those used in network interface cards, control systems and mobile phones. This research designs and implements a binary analysis system, called the Architecture-independent Binary Abstracting Code Analysis System (ABACAS), which reverses the normal program compilation process, lifting binary machine code to the Low-Level Virtual Machine (LLVM) compiler\u27s intermediate representation, thereby enabling existing security-related analyses to be applied to binary programs. The prototype targets ARM binaries but can be extended to support other architectures. Several programs are translated from ARM binaries and analyzed with existing analysis tools. Programs lifted from ARM binaries are an average of 3.73 times larger than the same programs compiled from a high-level language (HLL). Analysis results are equivalent regardless of whether the HLL source or ARM binary version of the program is submitted to the system, confirming the hypothesis that LLVM is effective for binary analysis

Autoscopy: Detecting Pattern-Searching Rootkits via Control Flow Tracing

Traditional approaches to rootkit detection assume the execution of code at a privilege level below that of the operating system kernel, with the use of virtual machine technologies to enable the detection system itself to be immune from the virus or rootkit code. In this thesis, we approach the problem of rootkit detection from the standpoint of tracing and instrumentation techniques, which work from within the kernel and also modify the kernel\u27s run-time state to detect aberrant control flows. We wish to investigate the role of emerging tracing frameworks (Kprobes, DTrace etc.) in enforcing operating system security without the reliance on a full-blown virtual machine just for the purposes of such policing. We first build a novel rootkit prototype that uses pattern-searching techniques to hijack hooks embedded in dynamically allocated memory, which we present as a showcase of emerging attack techniques. We then build an intrusion detection system-- autoscopy, atop kprobes, that detects anomalous control flow patterns typically exhibited by rootkits within a running kernel. Furthermore, to validate our approach, we show that we were able to successfully detect 15 existing Linux rootkits. We also conduct performance analyses, which show the overhead of our system to range from 2% to 5% on a wide range of standard benchmarks. Thus by leveraging tracing frameworks within operating systems, we show that it is possible to introduce real-world security in devices where performance and resource constraints are tantamount to security considerations