CryptoEval: Evaluating the Risk of Cryptographic Misuses in Android Apps with Data-Flow Analysis

The misunderstanding and incorrect configurations of cryptographic primitives have exposed severe security vulnerabilities to attackers. Due to the pervasiveness and diversity of cryptographic misuses, a comprehensive and accurate understanding of how cryptographic misuses can undermine the security of an Android app is critical to the subsequent mitigation strategies but also challenging. Although various approaches have been proposed to detect cryptographic misuses in Android apps, seldom studies have focused on estimating the security risks introduced by cryptographic misuses. To address this problem, we present an extensible framework for deciding the threat level of cryptographic misuses in Android apps. Firstly, we propose a unified specification for representing cryptographic misuses to make our framework extensible and develop adapters to unify the detection results of the state-of-the-art cryptographic misuse detectors, resulting in an adapter-based detection toolchain for a more comprehensive list of cryptographic misuses. Secondly, we employ a misuse-originating data-flow analysis to connect each cryptographic misuse to a set of data-flow sinks in an app, based on which we propose a quantitative data-flow-driven metric for assessing the overall risk of the app introduced by cryptographic misuses. To make the per-app assessment more useful in the app vetting at the app-store level, we apply unsupervised learning to predict and classify the top risky threats, to guide more efficient subsequent mitigations. In the experiments on an instantiated implementation of the framework, we evaluate the accuracy of our detection and the effect of data-flow-driven risk assessment of our framework. Our empirical study on over 40,000 apps as well as the analysis of popular apps reveals important security observations on the real threats of cryptographic misuses in Android apps

Light-weight verification of cryptographic API usage

A pluggable type system is a light-weight approach for compile-time program verification, which provides more powerful types to both developers and compilers. Developers use pluggable types to boost program understanding, while compilers leverage the new types to enforce interesting properties, e.g., NullPointerException freedom. This thesis presents a new type system, the Crypto Checker, to help developers prevent cryptographic APIs misuse. In addition, this thesis presents the Property File Handler and the Network Request Checker as two type system extensions. The Property File Handler performs type refinement to Java Properties by reading property files at compile time, while the Network Request Checker reports all the possible network requests to prevent potential information leakage in Java and Android applications. Using cryptographic APIs to encrypt and decrypt data, calculate digital signatures, or compute hashes is error prone. Weak or unsupported cryptographic algorithms can cause information leakage and runtime exceptions, such as a NoSuchAlgorithmException in Java. Using the wrong cryptographic service provider can also lead to unsupported cryptographic algorithms. Moreover, for Android developers who want to store their key material in the Android Keystore, misused cryptographic algorithms and providers make the key material unsafe. This thesis presents the Crypto Checker, a pluggable type system that detects the use of forbidden algorithms and providers at compile time. For typechecked code, the Crypto Checker guarantees that only trusted algorithms and providers are used, and thereby ensures that the cryptographic APIs never cause runtime exceptions or use weak algorithms or providers. The type system consists of an easy-to-understand type qualifier hierarchy. The Crypto Checker is flexible and easy-to-use---it allows developers to determine which algorithms and providers are permitted by writing specifications using type qualifiers. We implemented the Crypto Checker for Java and evaluated it with 32 open-source Java applications (over 2 million LOC). We found 2 issues that cause runtime exceptions and 62 violations of security recommendations and best practices. We also used the Crypto Checker to analyze 65 examples from a public benchmark of hard security issues and discuss the differences between our approach and a different static analysis in detail. Malicious or unsafe applications collect and send users' data to untrusted external servers via network requests, e.g., HTTP and socket requests, which will cause information leakage. Detecting the possible network requests on the source code level without running the applications is a light-weight approach to solve the problem. Application stores that have the source code of the uploaded apps can take advantage of this to ensure application security. Security teams in companies also use similar technologies to guarantee compliance. This thesis presents the Network Request Checker, a type system extension for Java to detect and report all the possible network requests to developers at compile time. The Network Request Checker can be integrated into any other pluggable type system or be seen as a stand-alone type system depending on developers' needs. We evaluated this type system with 6 real-world Java and Android applications and discuss the experimental results. To improve the Crypto Checker, the Network Request Checker, and other type systems' precision, i.e., to obtain more valuable information from the program at compile time, this thesis presents the Property File Handler, a type system extension that reads property files to perform type refinement on Java Properties. When an application reads property files, the Property File Handler will also try to load, store, and propagate the information from property files. A simple type hierarchy is proposed to achieve this functionality. By using the Property File Handler, we found and fixed a potential false negative with Java Properties, while other static analysis tools, e.g., SonarSource, did not handle that code correctly

A Reevaluation Of Why Crypto-Detectors Fail: A Systematic Revaluation Of Cryptographic Misuse Detection Techniques

The correct use of cryptography is central to ensuring data security in modern software systems. Hence, several academic and commercial static analysis tools have been developed for detecting and mitigating crypto-API misuse. While developers are optimistically adopting these crypto-API misuse detectors (or crypto-detectors) in their software development cycles, this momentum must be accompanied by a rigorous understanding of their effectiveness at finding crypto-API misuse in practice. The original paper presents the MASC framework, which enables a systematic and data-driven evaluation of crypto-detectors using mutation testing. MASC was grounded in a comprehensive view of the problem space by developing a data-driven taxonomy of existing crypto-API misuse, containing 105 misuse cases organized among nine semantic clusters. 12 generalizable usage based mutation operators were developed and three mutation scopes that can expressively instantiate thousands of compilable variants of the misuse cases for thoroughly evaluating crypto-detectors. Using MASC, nine major crypto-detectors were evaluated and 19 unique, undocumented flaws that severely impact the ability of crypto-detectors to discover misuses in practice were found. For my thesis, I built upon this previous research and greatly expanded the MASC framework. MASC was expanded in all areas by adding new functionality, new operators, new misuses, and expanding the taxonomy. In addition, I reevaluated the most up-to-date versions of the original 9 crypto-detectors and evaluated 5 additional crypto-detectors. On top of this I also doubled the amount of applications I used to evaluate the tools. To analyze crypto-detectors I looked at both the 9 crypto-detectors evaluated in the original work and 5 new crypto-detectors. For the original crypto-detectors, I evaluated them with the updated MASC against their most up-to-date versions to determine if old flaws that have previously been fixed have a tendency to reappear. Both old and new crypto-detectors were evaluated with mutated Android and Java applications that were used in the original MASC paper, 15 newly mutated Android and Java applications, and minimal examples of cryptographic misuse

Static and Dynamic Analysis in Cryptographic-API Misuse Detection of Mobile Application

With Android devices becoming more advanced and gaining more popularity, the number of cryptographic-API misuses in mobile applications is escalating. Numerous snippets of code in Android are from Stack Overflow and over 90% of them contain several crypto-issues. Various crypto-misuse detectors come out aiming to report vulnerabilities of apps and better secure users’ privacy. These detectors can be broadly classified into two categories based on the analysis strategies employed to catch misuses – static analysis (i.e., by scanning the code base) and dynamic analysis (i.e., by executing the code). However, there are not enough research on comparing their underlying differences, making it difficult to explain the pervasiveness of static crypto-detectors in both academia and industry. The lack of studies potentially limits the improvement of crypto-detection efficiency. In this study, a holistic evaluation and comparison on static and dynamic analysis’ underlying mechanisms, robustness, and efficiency are carried out. A systematic empirical experiment is implemented on testing 1003 popular Android applications across 21 categories from Google Play. We find that 93.3% of the apps make at least one mistake using cryptographic APIs and closely analyze top four cryptographic rules reported to be violated most frequently by static crypto detector. Instead of merely comparing statistics such as false positives (i.e., false alarms), we focus on examining the crypto rules whose number of violations reported by static and dynamic crypto detectors diverge greatly. In addition, we firstly posit a new taxonomy schema that classifies cryptographic rules based on how they are inspected rather than their attack type or severity level. This schema will be useful to both researchers and practitioners to decide how to efficiently combine static and dynamic techniques to improve the reliability and accuracy of crypto-detection

A Systematic Approach to Benchmark and Improve Automated Static Detection of Java-API Misuses

Today's software industry relies heavily on the reuse of existing software libraries. Such libraries provide the building blocks for modern software products. Reusing them allow developers to focus on innovation, while standing on the shoulders of giants. To use libraries effectively, developers need to know the Application Programming Interfaces (APIs) through which they communicate with the libraries. This includes both the APIs' semantics and the (implicit) usage constraints that come with them. In the face of the rapidly growing and evolving supply of software libraries, this has become a challenging task. As a result, incorrect usages of APIs, or API misuses, are a prevalent cause of software bugs, crashes, and vulnerabilities. In reaction to this problem, researchers have proposed a multitude of developer-assistance tools. One particular class of such tools automates the detection of API misuses in software code. We call these tools API-misuse detectors. Existing misuse detectors address different aspects of API misuse. However, no attempt has been made to systematically define the problem space of API misuse and to assess the prevalence of API misuses compared to other types of bugs. This makes it impossible to judge the relevance of research on API-misuse detection. Moreover, previous empirical evaluations of misuse detectors commonly measure the detectors' precision. However, since the studies use different datasets, it is unclear to which extend the results are comparable. It is also unclear where the detectors make trade-offs between their precision and their recall. In this thesis, we first present a systematic analysis of the problem of API misuse. We find that API misuse causes 9.1% of all software bugs in real-world projects, including many critical issues, such as program crashes, data loss, and security vulnerabilities. Then, we survey the literature to consolidate over a decade of research on API-misuse detection and build MUBench, a public automated benchmark for API-misuse detectors. This enables us to conduct the first-ever qualitative and quantitative comparison of existing misuse detectors. We find that these detectors have the potential to discover many API misuses, but suffer from extremely low precision and recall in practice. Finally, we systematically design MUDetect, a new API-misuse detector that addresses many of the problems of existing detectors. Using MUBench, we demonstrate that MUDetect clearly outperforms existing detectors with respect to both precision and recall. Our results provide strong evidence that, following our systematic approach, we can develop API-misuse detectors that are fit for practical application

A Lightweight Type System with Uniqueness and Typestates for the Java Cryptography API

Java cryptographic APIs facilitate building secure applications, but not all developers have strong cryptographic knowledge to use these APIs correctly. Several studies have shown that misuses of those cryptographic APIs may cause significant security vulnerabilities, compromising the integrity of applications and exposing sensitive data. Hence, it is an important problem to design methodologies and techniques, which can guide developers in building secure applications with minimum effort, and that are accessible to non-experts in cryptography. In this thesis, we present a methodology that reasons about the correct usage of Java cryptographic APIs with types, specifically targeting to cryptographic applications. Our type system combines aliasing control and the abstraction of object states into typestates, allowing users to express a set of user-defined disciplines on the use of cryptographic APIs and invariants on variable usage. More specifically, we employ the typestate automaton to depict typestates within our type system, and we control aliases by applying the principle of uniqueness to sensitive data. We mainly focus on the usage of initialization vectors. An initialization vector is a binary vector used as the input to initialize the state for the encryption of a plaintext block sequence. Randomization and uniqueness are crucial to an initialization vector. Failing to maintain a unique initialization vector for encryption can compromise confidentiality. Encrypting the same plaintext with the same initialization vector always yields the same ciphertext, thereby simplifying the attacker's task of guessing the cipher pattern. To address this problem practically, we implement our approach as a pluggable type system on top of the EISOP Checker Framework. To minimize the cryptographic expertise required by application developers looking to incorporate secure computing concepts into their software, our approach allows cryptographic experts to plug in the protocols into the system. In this setting, developers merely need to provide minimal annotations on sensitive data—requiring little cryptographic knowledge. We also evaluated our work by performing experiments over one benchmark and 7 real-world Java projects from Github. We found that 6 out 7 projects have security issues. In summary, we found 12 misuses in initialization vectors