EMULATOR vs REAL PHONE: Android Malware Detection Using Machine Learning

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.The Android operating system has become the most popular operating system for smartphones and tablets leading to a rapid rise in malware. Sophisticated Android malware employ detection avoidance techniques in order to hide their malicious activities from analysis tools. These include a wide range of anti-emulator techniques, where the malware programs attempt to hide their malicious activities by detecting the emulator. For this reason, countermeasures against anti-emulation are becoming increasingly important in Android malware detection. Analysis and detection based on real devices can alleviate the problems of anti-emulation as well as improve the effectiveness of dynamic analysis. Hence, in this paper we present an investigation of machine learning based malware detection using dynamic analysis on real devices. A tool is implemented to automatically extract dynamic features from Android phones and through several experiments, a comparative analysis of emulator based vs. device based detection by means of several machine learning algorithms is undertaken. Our study shows that several features could be extracted more effectively from the on-device dynamic analysis compared to emulators. It was also found that approximately 24% more apps were successfully analysed on the phone. Furthermore, all of the studied machine learning based detection performed better when applied to features extracted from the on-device dynamic analysis

Longitudinal performance analysis of machine learning based Android malware detectors

This paper presents a longitudinal study of the performance of machine learning classifiers for Android malware detection. The study is undertaken using features extracted from Android applications first seen between 2012 and 2016. The aim is to investigate the extent of performance decay over time for various machine learning classifiers trained with static features extracted from date-labelled benign and malware application sets. Using date-labelled apps allows for true mimicking of zero-day testing, thus providing a more realistic view of performance than the conventional methods of evaluation that do not take date of appearance into account. In this study, all the investigated machine learning classifiers showed progressive diminishing performance when tested on sets of samples from a later time period. Overall, it was found that false positive rate (misclassifying benign samples as malicious) increased more substantially compared to the fall in True Positive rate (correct classification of malicious apps) when older models were tested on newer app samples

Deep learning guided Android malware and anomaly detection

In the past decade, the cyber-crime related to mobile devices has increased. Mobile devices, especially the ones running on Android operating system are particularly interesting to malware creators, as the users often keep the biggest amount of personal information on their mobile devices, such as their contacts, social media profiles, emails, and bank accounts. Both dynamic and static malware analysis is necessary to prevent and detect malware, as both techniques have their benefits and shortcomings. In this paper, we propose a deep learning technique that relies on LSTM and encoder-decoder neural network architectures for dynamic malware analysis based on CPU, memory and battery usage. The proposed system is able to detect and notify users about anomalies in system that is likely consequence of malware behaviour. The method was implemented as a part of OWASP Seraphimdroids anti-malware mechanism and notifies users about anomalies on their devices. The method proved to perform with an F1-score of 79.2%.Comment: First (draft) version of the pape

DL-Droid: Deep learning based android malware detection using real devices

open access articleThe Android operating system has been the most popular for smartphones and tablets since 2012. This popularity has led to a rapid raise of Android malware in recent years. The sophistication of Android malware obfuscation and detection avoidance methods have significantly improved, making many traditional malware detection methods obsolete. In this paper, we propose DL-Droid, a deep learning system to detect malicious Android applications through dynamic analysis using stateful input generation. Experiments performed with over 30,000 applications (benign and malware) on real devices are presented. Furthermore, experiments were also conducted to compare the detection performance and code coverage of the stateful input generation method with the commonly used stateless approach using the deep learning system. Our study reveals that DL-Droid can achieve up to 97.8% detection rate (with dynamic features only) and 99.6% detection rate (with dynamic + static features) respectively which outperforms traditional machine learning techniques. Furthermore, the results highlight the significance of enhanced input generation for dynamic analysis as DL-Droid with the state-based input generation is shown to outperform the existing state-of-the-art approaches

Emulation vs Instrumentation for Android Malware Detection

In resource constrained devices, malware detection is typically based on offline analysis using emulation. In previous work it has been claimed that such emulation fails for a significant percentage of Android malware because well-designed malware detects that the code is being emulated. An alternative to emulation is malware analysis based on code that is executing on an actual Android device. In this research, we collect features from a corpus of Android malware using both emulation and on-phone instrumentation. We train machine learning models based on emulated features and also train models based on features collected via instrumentation, and we compare the results obtained in these two cases

DroidFusion: A Novel Multilevel Classifier Fusion Approach for Android Malware Detection

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI linkAndroid malware has continued to grow in volume and complexity posing significant threats to the security of mobile devices and the services they enable. This has prompted increasing interest in employing machine learning to improve Android malware detection. In this paper, we present a novel classifier fusion approach based on a multilevel architecture that enables effective combination of machine learning algorithms for improved accuracy. The framework (called DroidFusion), generates a model by training base classifiers at a lower level and then applies a set of ranking-based algorithms on their predictive accuracies at the higher level in order to derive a final classifier. The induced multilevel DroidFusion model can then be utilized as an improved accuracy predictor for Android malware detection. We present experimental results on four separate datasets to demonstrate the effectiveness of our proposed approach. Furthermore, we demonstrate that the DroidFusion method can also effectively enable the fusion of ensemble learning algorithms for improved accuracy. Finally, we show that the prediction accuracy of DroidFusion, despite only utilizing a computational approach in the higher level, can outperform stacked generalization, a well-known classifier fusion method that employs a meta-classifier approach in its higher level

A Survey and Evaluation of Android-Based Malware Evasion Techniques and Detection Frameworks

Android platform security is an active area of research where malware detection techniques continuously evolve to identify novel malware and improve the timely and accurate detection of existing malware. Adversaries are constantly in charge of employing innovative techniques to avoid or prolong malware detection effectively. Past studies have shown that malware detection systems are susceptible to evasion attacks where adversaries can successfully bypass the existing security defenses and deliver the malware to the target system without being detected. The evolution of escape-resistant systems is an open research problem. This paper presents a detailed taxonomy and evaluation of Android-based malware evasion techniques deployed to circumvent malware detection. The study characterizes such evasion techniques into two broad categories, polymorphism and metamorphism, and analyses techniques used for stealth malware detection based on the malware’s unique characteristics. Furthermore, the article also presents a qualitative and systematic comparison of evasion detection frameworks and their detection methodologies for Android-based malware. Finally, the survey discusses open-ended questions and potential future directions for continued research in mobile malware detection

Détection de programmes malveillants dédiée aux appareils mobiles

La conception d’une méthode efficace de détection de programmes malveillants dédiée aux appareils mobiles se place dans le contexte d’une architecture multiservices centrée sur le paiement mobile appelée ATISCOM. Cette architecture est développée par le Laboratoire de recherche en réseautique et informatique mobile de Polytechnique Montréal en collaboration avec Flexgroups et subventionnée par le CRSNG. Plusieurs enjeux dans ce projet sont dédiés à la sécurité de la plateforme qui est très sensible puisque elle doit manipuler des informations privées et financières et fonctionner en réseau et sur des appareils mobiles. La menace la plus importante pour les téléphones intelligents est celle du malware, ou logiciel malveillant, et ce mémoire propose d’y répondre. Nous avons établi une revue de littérature du domaine de la détection de malware sur Android, la plateforme choisie pour ce projet. Elle montre la présence importante des logiciels malveillants dans les environnements mobiles, la menace qu’ils représentent et leur évolution. Celle-ci décrit ensuite les domaines principaux de l’analyse statique et dynamique, sur serveur et sur appareil mobile. Elle montre de plus la présence grandissante de l’apprentissage automatique, et le meilleur équilibre entre précision et performance des systèmes hybrides. Après analyse des méthodes basées sur l’analyse dynamique (et statique) sur appareil mobile les plus prometteuses, nous distinguons leurs lacunes et décidons de bâtir une architecture client-serveur hybride utilisant l’apprentissage automatique pour pallier à ces dernières. La tâche se révèlera trop importante pour une simple maîtrise et nous concentrerons nos efforts sur une méthode d’analyse statique légère pouvant offrir une précision suffisante et rouler sur mobile. Ceci constituera la première pierre pour construire la méthode hybride de l’architecture idéale. ----------ABSTRACT: The design of an efficient malware detection method for mobile device is part of the ATISCOM architecture, which aims to be multiservices, centered on mobile payment. This architecture is developed by LARIM at Polytechnique Montreal, with its industrial partner Flexgroups and with the financial help of CRSNG. There are multiple goals in this project dedicated to improve the security of the platform, which is supposed to handle private and financial information on mobile devices and networks, and thus is very sensitive. The main threat for mobile security is mobile malware, and this work tries to answer it. We start this paper with a literature review on malware detection for Android, which will be the chosen platform for this project. It first shows the high and increasing number of malware for smartphones in the news. We then describe the sub-domains, such as static and dynamic analysis, server-side and on-device detection. This also shows that machine learning takes a big chunk of the recent papers in the domain, and that the best compromise between precision and performance is often attained by hybrid systems. We review the latest and most interesting papers in the dynamic analysis sub-domain and a few static analysis papers, all for on-device detection. We list their weaknesses (and also the numbers on performance and precision for future comparison) and decide to make our own machine learning clientserver hybrid method. But it would be too huge a work for a simple master so we’ll focus on a lightweight static analysis on-device detection method for starters

Un modèle de détection de logiciels malveillants pour terminaux mobiles

Un modèle de détection de logiciels malveillants pour terminaux mobiles contribue au domaine de la sécurité informatique. La cybersécurité est une problématique actuelle majeure principalement motivée par le nombre croissant de cyberattaques. En effet, les pertes de données à cause des brèches informatiques ont coûté 45 milliards de dollars canadiens en 2018. En plus, du point de vue financier, des problèmes éthiques apparaissent également si des informations personnelles de clients et utilisateurs sont divulguées. En raison de la popularité des téléphones intelligents et des tablettes, les terminaux mobiles deviennent la cible de cyberattaques. Il est donc essentiel d’étudier de nouveaux moyens de prévenir, de détecter et de contrer les cyberattaques. Dans ces mécanismes de détection, l’apprentissage machine est utilisé pour créer des classificateurs qui permettent de déterminer si une application est dangereuse ou non. L’avantage d’un réseau de neurones est qu’il permet de s’adapter à des situations inédites. Contrairement à un système de règles de sécurité fixes, nous allons utiliser cette nouvelle technologie afin de pouvoir identifier des types de comportements malveillants et de pouvoir le généraliser à des programmes malveillants futurs.L’objectif de cette recherche consiste à proposer un modèle de détection hybride de programmes malveillants sur Android basé sur deux réseaux de neurones de classification entraînés par des ensembles de caractéristiques statiques et dynamiques. Nous avons tout d’abord procédé à une revue de littérature afin de connaître les techniques de détection existantes. Cette étude n’est pas exhaustive, mais permet de cerner les principaux enjeux rencontrés ainsi que les solutions proposées par la communauté scientifique. Ces dernières peuvent se répartir en deux groupes; les méthodes statiques consistent à examiner le code de l’application mobile, tandis que les méthodes dynamiques analysent le comportement d’une application lorsque cette dernière est exécutée roule sur un terminal mobile. Notre but est d’utiliser ces deux méthodes afin de profiter de leurs avantages respectifs. Pour ce faire, nous avons choisi d’utiliser la base de données hybride Omnidroid composée de 25,999 caractéristiques statiques et de 5,932 caractéristiques dynamiques. Nous montrons lors de nos travaux que 22,636 caractéristiques statiques ainsi que 2,210 caractéristiques dynamiques de la base de données d’Omnidroid sont vides. Nous menons également un plan d’expérience composé de centaines d’entraînements afin de régler les valeurs des hyperparamètres améliorant l’apprentissage sur ce jeu de données ainsi que pour sélectionner les caractéristiques restantes les plus pertinentes.----------ABSTRACT:A malware detection model for mobile devices contributes to the field of computer security. Cybersecurity is a major current problem mainly motivated by the growing number of cyber attacks. Indeed, data loss due to computer breaches cost Canada $45 billion in 2018. In addition, ethical problems also arise if personal information of customers and users is disclosed. Due to the popularity of smartphones and tablets, mobile devices are becoming the target of cyberattacks. It is therefore essential to explore new ways to prevent, detect and counter cyberattacks. In these detection mechanisms, machine learning is used to create classifiers that determine whether an application is dangerous or not. The advantage of a neural network is that it allows you to adapt to new situations. Unlike a system of fixed security rules, we will use this new technology in order to be able to identify types of malicious behavior and to be able to generalize it to future malicious programs. The goal of this research is to propose a hybrid malware detection model on Android itself based on two classification neural networks driven by sets of static and dynamic features. We first conducted a literature review to find out about existing detection techniques. These can be divided into two groups; static methods consist of examining the code of the mobile application while dynamic methods analyze the behavior of an application when it is running on a mobile terminal. Our goal is to use these two methods to take advantage of their respective advantages. To do this, we chose to use the hybrid database “Omnidroid” composed of 25,999 static features and 5,932 dynamic features. We show that 22,636 static features as well as 2,210 dynamic features of the Omnidroid database are empty. We are also carrying out an experiment plan composed of hundreds of trainings in order to adjust the values of the hyperparameters improving the learning on this dataset as well as to select the most relevant remaining features