An SMDP-based Resource Management Scheme for Distributed Cloud Systems

In this paper, the resource management problem in geographically distributed cloud systems is considered. The Follow Me Cloud concept which enables service migration across federated data centers (DCs) is adopted. Therefore, there are two types of service requests to the DC, i.e., new requests (NRs) initiated in the local service area and migration requests (MRs) generated when mobile users move across service areas. A novel resource management scheme is proposed to help the resource manager decide whether to accept the service requests (NRs or MRs) or not and determine how much resources should be allocated to each service (if accepted). The optimization objective is to maximize the average system reward and keep the rejection probability of service requests under a certain threshold. Numerical results indicate that the proposed scheme can significantly improve the overall system utility as well as the user experience compared with other resource management schemes.Comment: 5 pages, 5 figures, conferenc

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Mobility-aware mechanisms for fog node discovery and selection

The recent development of delay-sensitive applications has led to the emergence of the fog computing paradigm. Within this paradigm, computation nodes present at the edge of the network can act as fog nodes (FNs) capable of processing users' tasks, thus resulting in latency reductions compared to the existing cloud-based execution model. In order to realize the full potential of fog computing, new research questions have arised, mainly due to the dynamic and heterogeneous fog computing context. This thesis focuses on the following questions in particular: How can a user detect the presence of a nearby FN? How should a user on the move adapt its FN discovery strategy, according to its changing context? How should an FN be selected , in the case of user mobility and FN mobility? These questions will be addressed throughout the different contributions of this thesis. The first contribution consists in proposing a discovery solution allowing a user to become aware of the existence of a nearby FN. Using our solution, the FN advertizes its presence using custom WiFi beacons, which will be detected by the user via a scan process. An implementation of this approach has been developed and its evaluation results have shown that it results in a non-negligible energy consumption given its use of WiFi. This has led to our second contribution, which aims at improving the WiFi scan performed in our discovery approach, especially in the case of user mobility. At a first stage, this improvement consisted in embedding information about the topology of the FNs in the beacons the user receives from previous FNs. We have shown that by adapting the scan behavior based on this information, considerable energy savings can be achieved, while guaranteeing a high discovery rate. However, as this approach is associated with a restrictive FN topology structure, we proposed a different alternative, at a second stage. This alternative leverages the history of cellular context information as an indicator allowing the user to infer whether an FN may be present in its current location. If so, the scan will be enabled. Otherwise, it is disabled. The simulation results comparing different classification algorithms have shown that a sequence-based model, such as a hidden-Markov model is able to effectively predict the FN presence in the current user location. While the previous approaches have focused on a sparse FN deployment, our third contribution considers a high density of FNs. Consequently, as there are multiple nearby FNs that can process the user's tasks, it is important to derive a suitable FN selection strategy. This strategy should consider the time-varying set of FNs caused by the user's mobility. Besides, it should minimize the number of switches from one FN to another, in order to maintain a good quality of service. With these considerations in mind, we have shown that an adaptive greedy approach, that selects an FN having a good-enough delay estimate, achieves the best results. Finally, unlike the previous contribution, where the focus has been on FN selection when the user is mobile, our final contribution deals with mobile vehicular FNs (VFNs). Given the mobility of such VFNs, it is important to make the most of their resources, since they are only available for a short time at a given area. So, we propose that, in order to select an appropriate VFN for a given task, a reference roadside unit (RSU) responsible for task assignment can use advice from a neighbor RSU. This advice consists in the VFN that will result in the lowest delay for the current task, based on the experience of the neighbor RSU. The results have shown that, using the provided advice, the reference RSU can observe significant delay reductions. All in all, the proposed contributions have addressed various problems that may arise in a fog computing context and the obtained results can be used to guide the development of the building blocks of future fog computing solutions.El recent desenvolupament d'aplicacions IoT ha comportat l'aparició del paradigma de fog computing. Dins d'aquest paradigma, els nodes de càlcul presents a la vora de la xarxa poden actuar com a “fog nodes'' (FN) capaços de processar les tasques dels usuaris, produint així reduccions de latència en comparació amb el model d'execució basat en núvol. Per assolir tot el potencial del fog computing, han sorgit noves qüestions de recerca, principalment a causa del context dinàmic i heterogeni de fog computing. Aquesta tesi se centra especialment en les qüestions següents: Com pot un usuari detectar la presència d'un FN? Com hauria d’adaptar un usuari en moviment la seva estratègia de descobriment de FN, segons el seu context? Com s’ha de seleccionar un FN, en el cas de la mobilitat dels usuaris i la mobilitat FN? Aquestes preguntes s’abordaran al llarg de les diferents aportacions d’aquesta tesi. La primera contribució consisteix a proposar una solució de descobriment que permeti a l'usuari detectar l’existència d’un FN proper. Mitjançant la nostra solució, un FN anuncia la seva presència mitjançant beacons Wi-Fi personalitzats, que seran detectats per l'usuari mitjançant un procés d’exploració. S'ha desenvolupat una implementació d'aquest enfocament i els seus resultats d’avaluació han demostrat que resulta en un consum d'energia menyspreable donat el seu ús del Wi-Fi. Això ha suposat la nostra segona contribució, que té com a objectiu millorar l’exploració Wi-Fi, especialment en el cas de la mobilitat dels usuaris. En una primera fase, aquesta millora va consistir a incorporar informació sobre la topologia dels FN en les beacons que rep l'usuari dels FN anteriors. Hem demostrat que mitjançant l'adaptació del comportament d'escaneig basat en aquesta informació es pot aconseguir un estalvi considerable d’energia, alhora que es garanteix un índex elevat de descobriment. Tanmateix, com aquest enfocament s'associa a una estructura de topologia FN restrictiva, vam proposar una alternativa diferent, en una segona etapa. Aquesta alternativa aprofita la història de la informació del context cel·lular com a indicador que permet a l'usuari deduir si un FN pot estar present en la seva ubicació. En cas afirmatiu, l'exploració estarà habilitada. Els resultats de la simulació comparant diferents algoritmes de classificació han demostrat que un model basat en seqüències, com un model HMM, és capaç de predir eficaçment la presència de FNs a la ubicació actual de l'usuari. Si bé els enfocaments anteriors s’han centrat en un desplegament escàs de FNs, la nostra tercera contribució considera una alta densitat d'FNs. En conseqüència, com que hi ha múltiples FNs propers que poden processar les tasques de l'usuari, és important derivar una estratègia de selecció de FN adequada. Aquesta estratègia hauria de tenir en compte el conjunt variable de temps causat per la mobilitat de l'usuari. A més, hauria de minimitzar el nombre de canvis d'un FN a un altre, per mantenir una bona qualitat del servei. Tenint en compte aquestes consideracions, hem demostrat que un enfocament codiciós adaptatiu, que selecciona un FN amb una estimació de retard suficient, aconsegueix els millors resultats. Finalment, a diferència de l'aportació anterior, on l'atenció s'ha fixat en la selecció d'FN quan l'usuari és mòbil, la nostra contribució final tracta sobre les FNs per a vehicles mòbils (VFNs). Tenint en compte la mobilitat d’aquests VFNs, és important aprofitar al màxim els seus recursos, ja que només estan disponibles per a un temps curt. Així doncs, proposem que, per seleccionar un VFN adequat per a una tasca, una unitat RSU responsable de l'assignació de tasques pot utilitzar consells d'un RSU veí. Aquest consell consisteix en escollir el VFN que suposarà el menor retard de la tasca actual, en funció de l’experiència del RSU veí. Els resultats han demostrat que ..

Patent Analytics Based on Feature Vector Space Model: A Case of IoT

The number of approved patents worldwide increases rapidly each year, which requires new patent analytics to efficiently mine the valuable information attached to these patents. Vector space model (VSM) represents documents as high-dimensional vectors, where each dimension corresponds to a unique term. While originally proposed for information retrieval systems, VSM has also seen wide applications in patent analytics, and used as a fundamental tool to map patent documents to structured data. However, VSM method suffers from several limitations when applied to patent analysis tasks, such as loss of sentence-level semantics and curse-of-dimensionality problems. In order to address the above limitations, we propose a patent analytics based on feature vector space model (FVSM), where the FVSM is constructed by mapping patent documents to feature vectors extracted by convolutional neural networks (CNN). The applications of FVSM for three typical patent analysis tasks, i.e., patents similarity comparison, patent clustering, and patent map generation are discussed. A case study using patents related to Internet of Things (IoT) technology is illustrated to demonstrate the performance and effectiveness of FVSM. The proposed FVSM can be adopted by other patent analysis studies to replace VSM, based on which various big data learning tasks can be performed

Learning and Reasoning Strategies for User Association in Ultra-dense Small Cell Vehicular Networks

Recent vehicular ad hoc networks research has been focusing on providing intelligent transportation services by employing information and communication technologies on road transport. It has been understood that advanced demands such as reliable connectivity, high user throughput, and ultra-low latency required by these services cannot be met using traditional communication technologies. Consequently, this thesis reports on the application of artificial intelligence to user association as a technology enabler in ultra-dense small cell vehicular networks. In particular, the work focuses on mitigating mobility-related concerns and networking issues at different mobility levels by employing diverse heuristic as well as reinforcement learning (RL) methods. Firstly, driven by rapid fluctuations in the network topology and the radio environment, a conventional, three-step sequence user association policy is designed to highlight and explore the impact of vehicle speed and different performance indicators on network quality of service (QoS) and user experience. Secondly, inspired by control-theoretic models and dynamic programming, a real-time controlled feedback user association approach is proposed. The algorithm adapts to the changing vehicular environment by employing derived network performance information as a heuristic, resulting in improved network performance. Thirdly, a sequence of novel RL based user association algorithms are developed that employ variable learning rate, variable rewards function and adaptation of the control feedback framework to improve the initial and steady-state learning performance. Furthermore, to accelerate the learning process and enhance the adaptability and robustness of the developed RL algorithms, heuristically accelerated RL and case-based transfer learning methods are employed. A comprehensive, two-tier, event-based, system level simulator which is an integration of a dynamic vehicular network, a highway, and an ultra-dense small cell network is developed. The model has enabled the analysis of user mobility effects on the network performance across different mobility levels as well as served as a firm foundation for the evaluation of the empirical properties of the investigated approaches

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier