Adaptive Honeypot Engagement through Reinforcement Learning of Semi-Markov Decision Processes

A honeynet is a promising active cyber defense mechanism. It reveals the fundamental Indicators of Compromise (IoCs) by luring attackers to conduct adversarial behaviors in a controlled and monitored environment. The active interaction at the honeynet brings a high reward but also introduces high implementation costs and risks of adversarial honeynet exploitation. In this work, we apply infinite-horizon Semi-Markov Decision Process (SMDP) to characterize a stochastic transition and sojourn time of attackers in the honeynet and quantify the reward-risk trade-off. In particular, we design adaptive long-term engagement policies shown to be risk-averse, cost-effective, and time-efficient. Numerical results have demonstrated that our adaptive engagement policies can quickly attract attackers to the target honeypot and engage them for a sufficiently long period to obtain worthy threat information. Meanwhile, the penetration probability is kept at a low level. The results show that the expected utility is robust against attackers of a large range of persistence and intelligence. Finally, we apply reinforcement learning to the SMDP to solve the curse of modeling. Under a prudent choice of the learning rate and exploration policy, we achieve a quick and robust convergence of the optimal policy and value.Comment: The presentation can be found at https://youtu.be/GPKT3uJtXqk. arXiv admin note: text overlap with arXiv:1907.0139

Integrating mobile and cloud resources management using the cloud personal assistant

The mobile cloud computing model promises to address the resource limitations of mobile devices, but effectively implementing this model is difficult. Previous work on mobile cloud computing has required the user to have a continuous, high-quality connection to the cloud infrastructure. This is undesirable and possibly infeasible, as the energy required on the mobile device to maintain a connection, and transfer sizeable amounts of data is large; the bandwidth tends to be quite variable, and low on cellular networks. The cloud deployment itself needs to efficiently allocate scalable resources to the user as well. In this paper, we formulate the best practices for efficiently managing the resources required for the mobile cloud model, namely energy, bandwidth and cloud computing resources. These practices can be realised with our mobile cloud middleware project, featuring the Cloud Personal Assistant (CPA). We compare this with the other approaches in the area, to highlight the importance of minimising the usage of these resources, and therefore ensure successful adoption of the model by end users. Based on results from experiments performed with mobile devices, we develop a no-overhead decision model for task and data offloading to the CPA of a user, which provides efficient management of mobile cloud resources

Gestion conjointe de ressources de communication et de calcul pour les réseaux sans fils à base de cloud

Mobile Edge Cloud brings the cloud closer to mobile users by moving the cloud computational efforts from the internet to the mobile edge. We adopt a local mobile edge cloud computing architecture, where small cells are empowered with computational and storage capacities. Mobile users’ offloaded computational tasks are executed at the cloud-enabled small cells. We propose the concept of small cells clustering for mobile edge computing, where small cells cooperate in order to execute offloaded computational tasks. A first contribution of this thesis is the design of a multi-parameter computation offloading decision algorithm, SM-POD. The proposed algorithm consists of a series of low complexity successive and nested classifications of computational tasks at the mobile side, leading to local computation, or offloading to the cloud. To reach the offloading decision, SM-POD jointly considers computational tasks, handsets, and communication channel parameters. In the second part of this thesis, we tackle the problem of small cell clusters set up for mobile edge cloud computing for both single-user and multi-user cases. The clustering problem is formulated as an optimization that jointly optimizes the computational and communication resource allocation, and the computational load distribution on the small cells participating in the computation cluster. We propose a cluster sparsification strategy, where we trade cluster latency for higher system energy efficiency. In the multi-user case, the optimization problem is not convex. In order to compute a clustering solution, we propose a convex reformulation of the problem, and we prove that both problems are equivalent. With the goal of finding a lower complexity clustering solution, we propose two heuristic small cells clustering algorithms. The first algorithm is based on resource allocation on the serving small cells where tasks are received, as a first step. Then, in a second step, unserved tasks are sent to a small cell managing unit (SCM) that sets up computational clusters for the execution of these tasks. The main idea of this algorithm is task scheduling at both serving small cells, and SCM sides for higher resource allocation efficiency. The second proposed heuristic is an iterative approach in which serving small cells compute their desired clusters, without considering the presence of other users, and send their cluster parameters to the SCM. SCM then checks for excess of resource allocation at any of the network small cells. SCM reports any load excess to serving small cells that re-distribute this load on less loaded small cells. In the final part of this thesis, we propose the concept of computation caching for edge cloud computing. With the aim of reducing the edge cloud computing latency and energy consumption, we propose caching popular computational tasks for preventing their re-execution. Our contribution here is two-fold: first, we propose a caching algorithm that is based on requests popularity, computation size, required computational capacity, and small cells connectivity. This algorithm identifies requests that, if cached and downloaded instead of being re-computed, will increase the computation caching energy and latency savings. Second, we propose a method for setting up a search small cells cluster for finding a cached copy of the requests computation. The clustering policy exploits the relationship between tasks popularity and their probability of being cached, in order to identify possible locations of the cached copy. The proposed method reduces the search cluster size while guaranteeing a minimum cache hit probability.Cette thèse porte sur le paradigme « Mobile Edge cloud» qui rapproche le cloud des utilisateurs mobiles et qui déploie une architecture de clouds locaux dans les terminaisons du réseau. Les utilisateurs mobiles peuvent désormais décharger leurs tâches de calcul pour qu’elles soient exécutées par les femto-cellules (FCs) dotées de capacités de calcul et de stockage. Nous proposons ainsi un concept de regroupement de FCs dans des clusters de calculs qui participeront aux calculs des tâches déchargées. A cet effet, nous proposons, dans un premier temps, un algorithme de décision de déportation de tâches vers le cloud, nommé SM-POD. Cet algorithme prend en compte les caractéristiques des tâches de calculs, des ressources de l’équipement mobile, et de la qualité des liens de transmission. SM-POD consiste en une série de classifications successives aboutissant à une décision de calcul local, ou de déportation de l’exécution dans le cloud.Dans un deuxième temps, nous abordons le problème de formation de clusters de calcul à mono-utilisateur et à utilisateurs multiples. Nous formulons le problème d’optimisation relatif qui considère l’allocation conjointe des ressources de calculs et de communication, et la distribution de la charge de calcul sur les FCs participant au cluster. Nous proposons également une stratégie d’éparpillement, dans laquelle l’efficacité énergétique du système est améliorée au prix de la latence de calcul. Dans le cas d’utilisateurs multiples, le problème d’optimisation d’allocation conjointe de ressources n’est pas convexe. Afin de le résoudre, nous proposons une reformulation convexe du problème équivalente à la première puis nous proposons deux algorithmes heuristiques dans le but d’avoir un algorithme de formation de cluster à complexité réduite. L’idée principale du premier est l’ordonnancement des tâches de calculs sur les FCs qui les reçoivent. Les ressources de calculs sont ainsi allouées localement au niveau de la FC. Les tâches ne pouvant pas être exécutées sont, quant à elles, envoyées à une unité de contrôle (SCM) responsable de la formation des clusters de calculs et de leur exécution. Le second algorithme proposé est itératif et consiste en une formation de cluster au niveau des FCs ne tenant pas compte de la présence d’autres demandes de calculs dans le réseau. Les propositions de cluster sont envoyées au SCM qui évalue la distribution des charges sur les différentes FCs. Le SCM signale tout abus de charges pour que les FCs redistribuent leur excès dans des cellules moins chargées.Dans la dernière partie de la thèse, nous proposons un nouveau concept de mise en cache des calculs dans l’Edge cloud. Afin de réduire la latence et la consommation énergétique des clusters de calculs, nous proposons la mise en cache de calculs populaires pour empêcher leur réexécution. Ici, notre contribution est double : d’abord, nous proposons un algorithme de mise en cache basé, non seulement sur la popularité des tâches de calculs, mais aussi sur les tailles et les capacités de calculs demandés, et la connectivité des FCs dans le réseau. L’algorithme proposé identifie les tâches aboutissant à des économies d’énergie et de temps plus importantes lorsqu’elles sont téléchargées d’un cache au lieu d’être recalculées. Nous proposons ensuite d’exploiter la relation entre la popularité des tâches et la probabilité de leur mise en cache, pour localiser les emplacements potentiels de leurs copies. La méthode proposée est basée sur ces emplacements, et permet de former des clusters de recherche de taille réduite tout en garantissant de retrouver une copie en cache

Replication-Aware Data Dissemination for Vehicular Ad Hoc Networks using Location Determination

Location determination is one of the most dif- ficult tasks to be achieved in Vehicular Ad Hoc networks (VANETs), as the nodes change their positions quickly due to high velocity. So, data dissemination to the vehicles in presence of high node mobility is one of the challenging issues to be resolved. To address these issues, in this paper, we propose a new Replication-Aware Data Dissemination (RADD) scheme for VANETs by estimating the location of the nodes. Separate algorithms are designed for position estimation, accessing the message from the remote vehicles, and route the packets to the destination. Also, vehicles on the road are sparsely distributed in some regions, so finding an appropriate vehicle for replica placement is also difficult as it needs a specialized approach. To search the suitable vehicle for replica placement, Bloom filters are used using which searching becomes fast and improves the overall per- formance of the system. Moreover, passive RFID tags are used on the vehicles and the RFID readers are placed on the RSUs to collect the data from these tags. These tags and readers are used to determine the vehicles positions in short range communication where GPS system does not work well. Hence, in the proposed scheme, there is no need of placement of access points on the either side of the road which reduces the complexity of the message dissemina- tion in the proposed scheme. The complexity analysis of the proposed scheme is evaluated in different network con- ditions with respect to data dissemination from source to destination. The performance of the proposed scheme was evaluated using different evaluation metrics in comparison to the other existing state-of-the-art schemes. The results obtained show that the proposed scheme performs better than the other existing schemes of its category with respect to various metrics. Specifically, there are improvements of 18 % in reliability, 27 % in replication cost, and 7 % in PDR with respect to the existing state-of-art protocolsThe work has been funded by grant from Instituto de Telecomunicacoes, Next Generation Networks and Applications Group (NetGNA), Covilha Delegation, by Government of Russian Federation, Grant 074-U01, by National Funding from the FCT - Fundacao para a Ciencia e a Tecnologia through the UID/EEA/50008/2013 Project.Kumar, N.; Rodrigues, JJPC.; Lloret, J.; Dua, A. (2015). Replication-Aware Data Dissemination for Vehicular Ad Hoc Networks using Location Determination. Mobile Networks and Applications. 20(2):251-267. doi:10.1007/s11036-015-0572-9S251267202Ghafoor KZ, Mohammed MA, Lloret J, Bakar KA, Zainuddin ZM (2013) Routing protocols in vehicular ad hoc networks: survey and research challenges. Netw Protoc Algorithm 5(4):39–83Dua A, Kumar N, Bawa S (2014) A systematic review on routing protocols for vehicular ad hoc networks. Veh Commun 1(1):33–52Kumar N, Chilamkurti N, Rodrigues J JPC (2014) Learning automata-based opportunistic data aggregation and forwarding scheme for alert generation in vehicular ad hoc networks. Comput Commun 59(1):22–32Bali RS, Kumar N, Rodriques J JPC (2014) Clustering in vehicular ad hoc networks: Taxonomy challenges and solutions. Veh Commun 1(3):134–152Kumar N, Chilamkurti N (2014) Collaborative trust aware Intelligent Intrusion Detection System in VANETs, 2014. Comput Electr Eng 40(6):1981–1996Kumar N, Lee JH (2013) Peer-to-Peer cooperative caching for data dissemination on urban vehicular communications. IEEE Syst J. doi: 10.1109/JSYST.2013.2285611Kumar N, Chilamkurti N, Park JH (2013) ALCA: agent learning-based clustering algorithm in vehicular ad hoc networks. Pers Ubiquit Comput 17(8):1683–1692Chen YS, Lin YW, Pan CY (2011) DIR: diagonal-intersection-based routing protocol for vehicular ad hoc networks. Telecommun Syst 46(4):299–316Kumar N (2014) Misra S. IEEE Systems Journal. doi: 10.1109/JSYST.2014.2335451.Kumar N, Lee J-H, Rodrigues JPC (2014) Intelligent Mobile Video Surveillance System as a Bayesian Coalition Game in Vehicular Sensor Networks: Learning Automata Approach. IEEE Trans Intell Transp Syst. doi: 10.1109/TITS.2014.2354372Garmehi M, Analoui M, Pathan M, Buyya R (2014) An economic replica placement mechanism for streaming content distribution in Hybrid CDN-P2P networks. Comput Commun. doi: 10.1016/j.comcom.2014.06.007Ahmadifard N, Nabizadeh H, Abbaspour M (2014) ISEFF: An ID-based scalable and efcient distributed file sharing technique in vehicular ad hoc networks. Wirel Pers Commun 75(2):821– 841Dias J AFF, Rodrigues J JPC, Isento JN, Pereira P RBA, Lloret J (2011) Performance assessment of fragmentation mechanisms for vehicular delay-tolerant networks. EURASIP J Wirel Commun Netw 2011(195):1–14Chen YS, Lin YW (2012) A mobicast routing protocol with carry-and-forward in vehicular ad hoc networks. Int J Commun Syst. doi: 10.1002/dac.2404Xia F, Ahmed AM, Yang LT, Ma J, Rodrigues JPC (2013) Exploiting social relationship to enable efficient replica allocation in adhoc social networks. IEEE Trans Parallel Distrib Syst. doi: 10.1109/TPDS.2013.2295805Ou CH (2014) A roadside unit-based localization scheme for vehicular ad hoc networks. Int J Commun Syst 27(1):135–150Saritha V, Viswanatham VM (2013) An efficient cross layer based channel reservation method for vehicular networks. doi: 10.1002/dac.2609Babu AV, Ajeer VKM (2013) Analytical model for connectivity of vehicular ad hoc networks in the presence of channel randomness. Int J Commun Syst 26(7):927–946Xiaonan W, Huanyan Q (2012) Constructing a VANET based on cluster chains. Int J Commun Syst. doi: 10.1002/dac.2484Slavik M, Mahgoub I, Alwakeel M (2014). Int J Commun Syst. doi: 10.1002/dac.2799Li C , Zhao C, Zhu L, Lin H, Li J (2013) Geographic routing protocol for vehicular ad hoc networks in city scenarios: a proposal and analysis. Int J Commun Syst. doi: 10.1002/dac.2602Chen YS, Hsu CS, Cheng CH (2013) Network mobility protocol for vehicular ad hoc networks. Int J Commun Syst. doi: 10.1002/dac.2525Zhang J, Xu Y (2013) Privacy-preserving authentication protocols with efficient verification in VANETs. Int J Commun Syst. doi: 10.1002/dac.2566Alawi MA, Saeed RA, Hassan AA, Alsaqour RA (2013) Simplified gateway selection scheme for multihop relay in vehicular ad hoc network. Int J Commun Syst. doi: 10.1002/dac.2581Oh S, Gruteser M, Pompili D (2012) Coordination-free Safety Messages Dissemination Protocol for Vehicular Networks. IEEE Trans Veh Technol. doi: 10.1109/TVT.2012.2197871Mershad K, Artail H (2013) Finding a STAR in a vehicular cloud. IEEE Intell Transp Syst Mag 5(2):55–68Liang H, Cai LX, Huang D, Shen X, Peng D (2012) An SMDP-based service model for interdomain resource allocation in mobile cloud networks. IEEE Trans Veh Technol 61(5):2222–2232Ibrahim WM, Taha AEM, Hassanein HS (2014) Using smart vehicles for localizing isolated Things. Comput CommunTrindade J, Vazao T (2014) Routing on large scale mobile ad hoc networks using bloom filters. Ad Hoc Networks. doi: 10.1016/j.adhoc.2014.05.016Sultan SA, Doori MMA, Bayatti AHA, Zedan H (2014) A comprehensive survey on vehicular Ad Hoc network. J Netw Comput Appl 37:380–392Aslam B, Wang P, Zou C (2008) An economical deployable and secure vehicular ad hoc network. In: Proceedings of Military Communication Conf., San Diego, CA, 2008, pp 1–7Kafsi M, Papadimitratos P, Doussey O, Alpcanz T, Hubaux JP (2008) VANET connectivity analysis, EPFL/T-Labs. Tech. Rep, SwitzerlandMohandas BK, Nayak A, Niak K, Goel N (2008) ABSRP-A service discovery approach for vehicular ad-hoc networks. In: Proceedings of 3rd IEEE Asia Pacific Services Computing Conference, Pisa, Italy, December 2008, pp 1590–1594Yang P, Wu W, Moniri M, Chibelushi CC (2013) Efcient Object Localization Using Sparsely Distributed Passive RFID Tags. IEEE Trans Ind Electron 60(12):5914–5924Jerbi M, Senouci SM, Meraihi R, Doudane YG (2007) An Improved Vehicular Ad Hoc Routing Protocol for City Environments. In: Proceedings of IEEE International Conference on Communications, ICC ’07, Glasgow, Scotland, June 2007Liu G, Lee BS, Seet BC, Foh CH, Wong KJ, Lee KK (2004) A routing strategy for metropolis vehicular communications. In: Proceedings of International Conference on Information Networking (ICOIN), Busan, Korea, February 2004Zhao T, Liu Z, Yan W, Li X (2011) BFBD: A Bloom filter based Buffering Data Dissemination Algorithm for Vehicular Ad hoc Networks. In: Proceedings of IEEE Consumer Communications and Networking Conference (CCNC), Las Vegas, USA, pp 477–481SUMO Simulation of Urban Mobility. [Online]. Available: http://sumo.sourceforge.net/NS2.35. [Online]. Available: http://www.isi.edu/nsnam/ns/Dua A, Kumar N, Bawa S, Chilamkurti N (2013) Efficient TDMA based Virtual Back off Algorithm for Adaptive Data Dissemination in VANETs. In: Procedings of International Symposium of Wireless and Pervasive Computing Taiwan 2013, pp 1–6Doss R, Zhou W, Sundaresan S, Yu S, Goa L (2012) A minimum disclosure approach to authentication and privacy in RFID systems. Comput Netw 56(15):3401–3416Doss R, Sundaresan S, Zhou W (2013) A practical quadratic residues based scheme for authentication and privacy in mobile RFID systems. Ad Hoc Networks 11(1):383–39