Adaptive learning-based resource management strategy in fog-to-cloud

Technology in the twenty-first century is rapidly developing and driving us into a new smart computing world, and emerging lots of new computing architectures. Fog-to-Cloud (F2C) is among one of them, which emerges to ensure the commitment for bringing the higher computing facilities near to the edge of the network and also help the large-scale computing system to be more intelligent. As the F2C is in its infantile state, therefore one of the biggest challenges for this computing paradigm is to efficiently manage the computing resources. Mainly, to address this challenge, in this work, we have given our sole interest for designing the initial architectural framework to build a proper, adaptive and efficient resource management mechanism in F2C. F2C has been proposed as a combined, coordinated and hierarchical computing platform, where a vast number of heterogeneous computing devices are participating. Notably, their versatility creates a massive challenge for effectively handling them. Even following any large-scale smart computing system, it can easily recognize that various kind of services is served for different purposes. Significantly, every service corresponds with the various tasks, which have different resource requirements. So, knowing the characteristics of participating devices and system offered services is giving advantages to build effective and resource management mechanism in F2C-enabled system. Considering these facts, initially, we have given our intense focus for identifying and defining the taxonomic model for all the participating devices and system involved services-tasks. In any F2C-enabled system consists of a large number of small Internet-of-Things (IoTs) and generating a continuous and colossal amount of sensing-data by capturing various environmental events. Notably, this sensing-data is one of the key ingredients for various smart services which have been offered by the F2C-enabled system. Besides that, resource statistical information is also playing a crucial role, for efficiently providing the services among the system consumers. Continuous monitoring of participating devices generates a massive amount of resource statistical information in the F2C-enabled system. Notably, having this information, it becomes much easier to know the device's availability and suitability for executing some tasks to offer some services. Therefore, ensuring better service facilities for any latency-sensitive services, it is essential to securely distribute the sensing-data and resource statistical information over the network. Considering these matters, we also proposed and designed a secure and distributed database framework for effectively and securely distribute the data over the network. To build an advanced and smarter system is necessarily required an effective mechanism for the utilization of system resources. Typically, the utilization and resource handling process mainly depend on the resource selection and allocation mechanism. The prediction of resources (e.g., RAM, CPU, Disk, etc.) usage and performance (i.e., in terms of task execution time) helps the selection and allocation process. Thus, adopting the machine learning (ML) techniques is much more useful for designing an advanced and sophisticated resource allocation mechanism in the F2C-enabled system. Adopting and performing the ML techniques in F2C-enabled system is a challenging task. Especially, the overall diversification and many other issues pose a massive challenge for successfully performing the ML techniques in any F2C-enabled system. Therefore, we have proposed and designed two different possible architectural schemas for performing the ML techniques in the F2C-enabled system to achieve an adaptive, advance and sophisticated resource management mechanism in the F2C-enabled system. Our proposals are the initial footmarks for designing the overall architectural framework for resource management mechanism in F2C-enabled system.La tecnologia del segle XXI avança ràpidament i ens condueix cap a un nou món intel·ligent, creant nous models d'arquitectures informàtiques. Fog-to-Cloud (F2C) és un d’ells, i sorgeix per garantir el compromís d’acostar les instal·lacions informàtiques a prop de la xarxa i també ajudar el sistema informàtic a gran escala a ser més intel·ligent. Com que el F2C es troba en un estat preliminar, un dels majors reptes d’aquest paradigma tecnològic és gestionar eficientment els recursos informàtics. Per fer front a aquest repte, en aquest treball hem centrat el nostre interès en dissenyar un marc arquitectònic per construir un mecanisme de gestió de recursos adequat, adaptatiu i eficient a F2C.F2C ha estat concebut com una plataforma informàtica combinada, coordinada i jeràrquica, on participen un gran nombre de dispositius heterogenis. La seva versatilitat planteja un gran repte per gestionar-los de manera eficaç. Els serveis que s'hi executen consten de diverses tasques, que tenen requisits de recursos diferents. Per tant, conèixer les característiques dels dispositius participants i dels serveis que ofereix el sistema és un requisit per dissenyar mecanismes eficaços i de gestió de recursos en un sistema habilitat per F2C. Tenint en compte aquests fets, inicialment ens hem centrat en identificar i definir el model taxonòmic per a tots els dispositius i sistemes implicats en l'execució de tasques de serveis. Qualsevol sistema habilitat per F2C inclou en un gran nombre de dispositius petits i connectats (conegut com a Internet of Things, o IoT) que generen una quantitat contínua i colossal de dades de detecció capturant diversos events ambientals. Aquestes dades són un dels ingredients clau per a diversos serveis intel·ligents que ofereix F2C. A més, el seguiment continu dels dispositius participants genera igualment una gran quantitat d'informació estadística. En particular, en tenir aquesta informació, es fa molt més fàcil conèixer la disponibilitat i la idoneïtat dels dispositius per executar algunes tasques i oferir alguns serveis. Per tant, per garantir millors serveis sensibles a la latència, és essencial distribuir de manera equilibrada i segura la informació estadística per la xarxa. Tenint en compte aquests assumptes, també hem proposat i dissenyat un entorn de base de dades segura i distribuïda per gestionar de manera eficaç i segura les dades a la xarxa. Per construir un sistema avançat i intel·ligent es necessita un mecanisme eficaç per a la gestió de l'ús dels recursos del sistema. Normalment, el procés d’utilització i manipulació de recursos depèn principalment del mecanisme de selecció i assignació de recursos. La predicció de l’ús i el rendiment de recursos (per exemple, RAM, CPU, disc, etc.) en termes de temps d’execució de tasques ajuda al procés de selecció i assignació. Adoptar les tècniques d’aprenentatge automàtic (conegut com a Machine Learning, o ML) és molt útil per dissenyar un mecanisme d’assignació de recursos avançat i sofisticat en el sistema habilitat per F2C. L’adopció i la realització de tècniques de ML en un sistema F2C és una tasca complexa. Especialment, la diversificació general i molts altres problemes plantegen un gran repte per realitzar amb èxit les tècniques de ML. Per tant, en aquesta recerca hem proposat i dissenyat dos possibles esquemes arquitectònics diferents per realitzar tècniques de ML en el sistema habilitat per F2C per aconseguir un mecanisme de gestió de recursos adaptatiu, avançat i sofisticat en un sistema F2C. Les nostres propostes són els primers passos per dissenyar un marc arquitectònic general per al mecanisme de gestió de recursos en un sistema habilitat per F2C.Postprint (published version

Enhanced non-parametric sequence learning scheme for internet of things sensory data in cloud infrastructure

The Internet of Things (IoT) Cloud is an emerging technology that enables machine-to-machine, human-to-machine and human-to-human interaction through the Internet. IoT sensor devices tend to generate sensory data known for their dynamic and heterogeneous nature. Hence, it makes it elusive to be managed by the sensor devices due to their limited computation power and storage space. However, the Cloud Infrastructure as a Service (IaaS) leverages the limitations of the IoT devices by making its computation power and storage resources available to execute IoT sensory data. In IoT-Cloud IaaS, resource allocation is the process of distributing optimal resources to execute data request tasks that comprise data filtering operations. Recently, machine learning, non-heuristics, multi-objective and hybrid algorithms have been applied for efficient resource allocation to execute IoT sensory data filtering request tasks in IoT-enabled Cloud IaaS. However, the filtering task is still prone to some challenges. These challenges include global search entrapment of event and error outlier detection as the dimension of the dataset increases in size, the inability of missing data recovery for effective redundant data elimination and local search entrapment that leads to unbalanced workloads on available resources required for task execution. In this thesis, the enhancement of Non-Parametric Sequence Learning (NPSL), Perceptually Important Point (PIP) and Efficient Energy Resource Ranking- Virtual Machine Selection (ERVS) algorithms were proposed. The Non-Parametric Sequence-based Agglomerative Gaussian Mixture Model (NPSAGMM) technique was initially utilized to improve the detection of event and error outliers in the global space as the dimension of the dataset increases in size. Then, Perceptually Important Points K-means-enabled Cosine and Manhattan (PIP-KCM) technique was employed to recover missing data to improve the elimination of duplicate sensed data records. Finally, an Efficient Resource Balance Ranking- based Glow-warm Swarm Optimization (ERBV-GSO) technique was used to resolve the local search entrapment for near-optimal solutions and to reduce workload imbalance on available resources for task execution in the IoT-Cloud IaaS platform. Experiments were carried out using the NetworkX simulator and the results of N-PSAGMM, PIP-KCM and ERBV-GSO techniques with N-PSL, PIP, ERVS and Resource Fragmentation Aware (RF-Aware) algorithms were compared. The experimental results showed that the proposed NPSAGMM, PIP-KCM, and ERBV-GSO techniques produced a tremendous performance improvement rate based on 3.602%/6.74% Precision, 9.724%/8.77% Recall, 5.350%/4.42% Area under Curve for the detection of event and error outliers. Furthermore, the results indicated an improvement rate of 94.273% F1-score, 0.143 Reduction Ratio, and with minimum 0.149% Root Mean Squared Error for redundant data elimination as well as the minimum number of 608 Virtual Machine migrations, 47.62% Resource Utilization and 41.13% load balancing degree for the allocation of desired resources deployed to execute sensory data filtering tasks respectively. Therefore, the proposed techniques have proven to be effective for improving the load balancing of allocating the desired resources to execute efficient outlier (Event and Error) detection and eliminate redundant data records in the IoT-based Cloud IaaS Infrastructure

Deep Reinforcement Learning Framework with Q Learning For Optimal Scheduling in Cloud Computing

Cloud computing is an emerging technology that is increasingly being appreciated for its diverse uses, encompassing data processing, The Internet of Things (IoT) and the storing of data. The continuous growth in the number of cloud users and the widespread use of IoT devices have resulted in a significant increase in the volume of data being generated by these users and the integration of IoT devices with cloud platforms. The process of managing data stored in the cloud has become more challenging to complete. There are numerous significant challenges that must be overcome in the process of migrating all data to cloud-hosted data centers. High bandwidth consumption, longer wait times, greater costs, and greater energy consumption are only some of the difficulties that must be overcome. Cloud computing, as a result, is able to allot resources in line with the specific actions made by users, which is a result of the conclusion that was mentioned earlier. This phenomenon can be attributed to the provision of a superior Quality of Service (QoS) to clients or users, with an optimal response time. Additionally, adherence to the established Service Level Agreement further contributes to this outcome. Due to this circumstance, it is of utmost need to effectively use the computational resources at hand, hence requiring the formulation of an optimal approach for task scheduling. The goal of this proposed study is to find ways to allocate and schedule cloud-based virtual machines (VMs) and tasks in such a way as to reduce completion times and associated costs. This study presents a new method of scheduling that makes use of Q-Learning to optimize the utilization of resources.The algorithm's primary goals include optimizing its objective function, building the ideal network, and utilizing experience replay techniques

Enhancing Smart Cities with IoT and Cloud Computing: A Study on Integrating Wireless Ad Hoc Networks for Efficient Communication

شهدت المدن الذكية تطورا جوهريا زاد من امكانياتها بشكل كبير .في الواقع ، لقد أتاحت التطورات الحديثة في انترنت الاشياء (IOT) فرصا جديدة من خلال حل عدد  من المشاكل الحرجة والتي ادت الى ابتكار المدن الذكية بالاضافة الى انشاء و حوسبة الخدمات و التطبيقات المتطورة للعديد من  المجاميع المطورة في المدينة . من اجل تعزيز تنمية المدن الذكية بأتجاه التواصل و المشاركة ،تركز هذه الدراسة على التطور في مجال المعلوماتية في ضوء انترنت الاشياء (IOT) و الحوسبة السحابية (CC) .جمعت بيانات انترنت الاشياء والتي تخص المدن الذكية بشكل متجانس . اصبح انترنت الاشياء الذي يسمح بتواصل الاشخاص مع بعضهم ممكنا باستخدام الذكاء الاصطناعي .بناءا على ذلك ،استخدمنا (ARF) في حسابات الذكاء الاصطناعي .للتبسيط ،ننصح باستخدام تخصيص اصول الالة الافتراضية للحوسبة السحابية التكيفية (ACC-VMRA ) .لتاكيد جدواها ،سنفحص و نضاعف كيفية تطبيق تطورات انترنت الاشياء (IOT ) و الحوسبة السحابية (CC) في المدن الذكية.تظهر نتائج التجربة ان حساب التحسين الموصى به اكثر انتاجية من الطرق الاخرى المستخدمة حاليا.Smart cities have recently undergone a fundamental evolution that has greatly increased their potentials. In reality, recent advances in the Internet of Things (IoT) have created new opportunities by solving a number of critical issues that are allowing innovations for smart cities as well as the creation and computerization of cutting-edge services and applications for the many city partners. In order to further the development of smart cities toward compelling sharing and connection, this study will explore the information innovation in smart cities in light of the Internet of Things (IoT) and cloud computing (CC). IoT data is first collected in the context of smart cities. The data that is gathered is uniform. The Internet of Things, which enables gadgets to connect with one another mostly without human involvement, is made possible by AI. In line with this, The Ad Hoc Routing Function (ARF) AI computation is used for multi-rule simplification, the use of Adaptive Cloud Computing Virtual Machine Asset Allotment Technique (ACC-VMRA) is advised. To confirm its viability, the applied developments of IoT and CC in smart cities is examined and duplicated. The experiment results show that the recommended enhancement calculation is more productive than other currently used methods

Enhancement of Metaheuristic Algorithm for Scheduling Workflows in Multi-fog Environments

Whether in computer science, engineering, or economics, optimization lies at the heart of any challenge involving decision-making. Choosing between several options is part of the decision- making process. Our desire to make the "better" decision drives our decision. An objective function or performance index describes the assessment of the alternative's goodness. The theory and methods of optimization are concerned with picking the best option. There are two types of optimization methods: deterministic and stochastic. The first is a traditional approach, which works well for small and linear problems. However, they struggle to address most of the real-world problems, which have a highly dimensional, nonlinear, and complex nature. As an alternative, stochastic optimization algorithms are specifically designed to tackle these types of challenges and are more common nowadays. This study proposed two stochastic, robust swarm-based metaheuristic optimization methods. They are both hybrid algorithms, which are formulated by combining Particle Swarm Optimization and Salp Swarm Optimization algorithms. Further, these algorithms are then applied to an important and thought-provoking problem. The problem is scientific workflow scheduling in multiple fog environments. Many computer environments, such as fog computing, are plagued by security attacks that must be handled. DDoS attacks are effectively harmful to fog computing environments as they occupy the fog's resources and make them busy. Thus, the fog environments would generally have fewer resources available during these types of attacks, and then the scheduling of submitted Internet of Things (IoT) workflows would be affected. Nevertheless, the current systems disregard the impact of DDoS attacks occurring in their scheduling process, causing the amount of workflows that miss deadlines as well as increasing the amount of tasks that are offloaded to the cloud. Hence, this study proposed a hybrid optimization algorithm as a solution for dealing with the workflow scheduling issue in various fog computing locations. The proposed algorithm comprises Salp Swarm Algorithm (SSA) and Particle Swarm Optimization (PSO). In dealing with the effects of DDoS attacks on fog computing locations, two Markov-chain schemes of discrete time types were used, whereby one calculates the average network bandwidth existing in each fog while the other determines the number of virtual machines existing in every fog on average. DDoS attacks are addressed at various levels. The approach predicts the DDoS attack’s influences on fog environments. Based on the simulation results, the proposed method can significantly lessen the amount of offloaded tasks that are transferred to the cloud data centers. It could also decrease the amount of workflows with missed deadlines. Moreover, the significance of green fog computing is growing in fog computing environments, in which the consumption of energy plays an essential role in determining maintenance expenses and carbon dioxide emissions. The implementation of efficient scheduling methods has the potential to mitigate the usage of energy by allocating tasks to the most appropriate resources, considering the energy efficiency of each individual resource. In order to mitigate these challenges, the proposed algorithm integrates the Dynamic Voltage and Frequency Scaling (DVFS) technique, which is commonly employed to enhance the energy efficiency of processors. The experimental findings demonstrate that the utilization of the proposed method, combined with the Dynamic Voltage and Frequency Scaling (DVFS) technique, yields improved outcomes. These benefits encompass a minimization in energy consumption. Consequently, this approach emerges as a more environmentally friendly and sustainable solution for fog computing environments