MapReduce based RDF assisted distributed SVM for high throughput spam filtering

This thesis was submitted for the degree of Doctor of Philosophy and was awarded by Brunel UniversityElectronic mail has become cast and embedded in our everyday lives. Billions of legitimate emails are sent on a daily basis. The widely established underlying infrastructure, its widespread availability as well as its ease of use have all acted as catalysts to such pervasive proliferation. Unfortunately, the same can be alleged about unsolicited bulk email, or rather spam. Various methods, as well as enabling architectures are available to try to mitigate spam permeation. In this respect, this dissertation compliments existing survey work in this area by contributing an extensive literature review of traditional and emerging spam filtering approaches. Techniques, approaches and architectures employed for spam filtering are appraised, critically assessing respective strengths and weaknesses. Velocity, volume and variety are key characteristics of the spam challenge. MapReduce (M/R) has become increasingly popular as an Internet scale, data intensive processing platform. In the context of machine learning based spam filter training, support vector machine (SVM) based techniques have been proven effective. SVM training is however a computationally intensive process. In this dissertation, a M/R based distributed SVM algorithm for scalable spam filter training, designated MRSMO, is presented. By distributing and processing subsets of the training data across multiple participating computing nodes, the distributed SVM reduces spam filter training time significantly. To mitigate the accuracy degradation introduced by the adopted approach, a Resource Description Framework (RDF) based feedback loop is evaluated. Experimental results demonstrate that this improves the accuracy levels of the distributed SVM beyond the original sequential counterpart. Effectively exploiting large scale, ‘Cloud’ based, heterogeneous processing capabilities for M/R in what can be considered a non-deterministic environment requires the consideration of a number of perspectives. In this work, gSched, a Hadoop M/R based, heterogeneous aware task to node matching and allocation scheme is designed. Using MRSMO as a baseline, experimental evaluation indicates that gSched improves on the performance of the out-of-the box Hadoop counterpart in a typical Cloud based infrastructure. The focal contribution to knowledge is a scalable, heterogeneous infrastructure and machine learning based spam filtering scheme, able to capitalize on collaborative accuracy improvements through RDF based, end user feedback. MapReduce based RDF Assisted Distributed SVM for High Throughput Spam Filterin

Economic recommendation based on pareto efficient resource allocation

A fundamentally important role of the Web economy is Online Resource Allocation (ORA) from producers to consumers, such as product allocation in E-commerce, job allocation in freelancing platforms, and driver resource allocation in P2P riding services. Since users have the freedom to choose, such allocations are not provided in a forced manner, but usually in forms of personalized recommendation, where users have the right to refuse. Current recommendation approaches mostly provide allocations to match the preference of each individual user, instead of treating the Web application as a whole economic system where users therein are mutually correlated on the allocations. This lack of global view leads to Pareto inefficiency, i.e., we can actually improve the recommendations by bettering some users while not hurting the others, and it means that the system did not achieve its best possible allocation. This problem is especially severe when the total amount of each resource is limited, so that its allocation to one (set of) user means that other users are left out. In this paper, we propose Pareto Efficient Economic Recommendation (PEER) - that the system provides the best possible (i.e., Pareto optimal) recommendations, where no user can gain further benefits without hurting the others. To this end, we propose a Multi-Objective Optimization (MOO) framework to maximize the surplus of each user simultaneously, and provide recommendations based on the resulting Pareto optima. To benefit the many existing recommendation algorithms, we further propose a Pareto Improvement Process (PIP) to turn their recommendations into Pareto efficient ones. Experiments on real-world datasets verify that PIP improves existing algorithms on recommendation performance and consumer surplus, while the direct PEER approach gains the best performance on both aspects

Intelligent Circuits and Systems

ICICS-2020 is the third conference initiated by the School of Electronics and Electrical Engineering at Lovely Professional University that explored recent innovations of researchers working for the development of smart and green technologies in the fields of Energy, Electronics, Communications, Computers, and Control. ICICS provides innovators to identify new opportunities for the social and economic benefits of society.　 This conference bridges the gap between academics and R&D institutions, social visionaries, and experts from all strata of society to present their ongoing research activities and foster research relations between them. It provides opportunities for the exchange of new ideas, applications, and experiences in the field of smart technologies and finding global partners for future collaboration. The ICICS-2020 was conducted in two broad categories, Intelligent Circuits & Intelligent Systems and Emerging Technologies in Electrical Engineering

Advanced Technologies for Device-to-device Communications Underlaying Cellular Networks

The past few years have seen a major change in cellular networks, as explosive growth in data demands requires more and more network capacity and backhaul capability. New wireless technologies have been proposed to tackle these challenges. One of the emerging technologies is device-to-device (D2D) communications. It enables two cellular user equip- ment (UEs) in proximity to communicate with each other directly reusing cellular radio resources. In this case, D2D is able to of oad data traf c from central base stations (BSs) and signi cantly improve the spectrum ef ciency of a cellular network, and thus is one of the key technologies for the next generation cellular systems. Radio resource management (RRM) for D2D communications and how to effectively exploit the potential bene ts of D2D are two paramount challenges to D2D communications underlaying cellular networks. In this thesis, we focus on four problems related to these two challenges. In Chapter 2, we utilise the mixed integer non-linear programming (MINLP) to model and solve the RRM optimisation problems for D2D communications. Firstly we consider the RRM optimisation problem for D2D communications underlaying the single carrier frequency division multiple access (SC-FDMA) system and devise a heuristic sub- optimal solution to it. Then we propose an optimised RRM mechanism for multi-hop D2D communications with network coding (NC). NC has been proven as an ef cient technique to improve the throughput of ad-hoc networks and thus we apply it to multi-hop D2D communications. We devise an optimal solution to the RRM optimisation problem for multi-hop D2D communications with NC. In Chapter 3, we investigate how the location of the D2D transmitter in a cell may affect the RRM mechanism and the performance of D2D communications. We propose two optimised location-based RRM mechanisms for D2D, which maximise the throughput and the energy ef ciency of D2D, respectively. We show that, by considering the location information of the D2D transmitter, the MINLP problem of RRM for D2D communications can be transformed into a convex optimisation problem, which can be ef ciently solved by the method of Lagrangian multipliers. In Chapter 4, we propose a D2D-based P2P le sharing system, which is called Iunius. The Iunius system features: 1) a wireless P2P protocol based on Bittorrent protocol in the application layer; 2) a simple centralised routing mechanism for multi-hop D2D communications; 3) an interference cancellation technique for conventional cellular (CC) uplink communications; and 4) a radio resource management scheme to mitigate the interference between CC and D2D communications that share the cellular uplink radio resources while maximising the throughput of D2D communications. We show that with the properly designed application layer protocol and the optimised RRM for D2D communications, Iunius can signi cantly improve the quality of experience (QoE) of users and of oad local traf c from the base station. In Chapter 5, we combine LTE-unlicensed with D2D communications. We utilise LTE-unlicensed to enable the operation of D2D in unlicensed bands. We show that not only can this improve the throughput of D2D communications, but also allow D2D to work in the cell central area, which normally regarded as a “forbidden area” for D2D in existing works. We achieve these results mainly through numerical optimisation and simulations. We utilise a wide range of numerical optimisation theories in our works. Instead of utilising the general numerical optimisation algorithms to solve the optimisation problems, we modify them to be suitable for the speci c problems, thereby reducing the computational complexity. Finally, we evaluate our proposed algorithms and systems through sophisticated numer- ical simulations. We have developed a complete system-level simulation framework for D2D communications and we open-source it in Github: https://github.com/mathwuyue/py- wireless-sys-sim

Heterogeneous Wireless Networks: Traffic Offloading, Resource Allocation and Coverage Analysis

Unlike 2G systems where the radius of macro base station (MBS) could reach several kilometers, the cell radius of LTE-Advanced and next generation wireless networks (NGWNs) such as 5G networks would be random and up to a few hundred meters in order to overcome the radio signal propagation impairments. Heterogeneous wireless networks (HetNets) are becoming an integral part of the NGWNs especially 5G networks, where small cell base stations (SBSs), wireless-fidelity (WiFi) access points (APs), cellular BSs and device-to-device (D2D) enabled links coexist together. HetNets represent novel approaches for the mobile data offloading, resource allocation and coverage probability problems that help to optimize the network traffic. However, heterogeneity and interworking among different radio access technologies bring new challenges such as bandwidth resource allocation, user/cell association, traffic offloading based on the user activity and coverage probability in HetNets. This dissertation attempts to address three key research areas: traffic offloading, bandwidth resource allocation and coverage probability problems in HetNets. In the first part of this dissertation, we derive the mathematical framework to calculate the required active user population factor (AUPF) of small cells based on the probabilistic traffic models. The number of total mobile users and number of active mobile users have different probabilistic distributions such as different combinations of Binomial and Poisson distributions. Furthermore, AUPF is utilized to investigate the downlink BS and backhaul power consumption of HetNets. In the second part, we investigate two different traffic offloading (TO) schemes (a) Path loss (PL) and (b) Signal-to-Interference ratio (SIR) based strategies. In this context, a comparative study on two techniques to offload the traffic from macrocell to small cell is studied. Additionally, the AUPF, small cell access scheme and traffic type are included into a PL based TO strategy to minimize the congested macrocell traffic. In the third part, the joint user assignment and bandwidth resource allocation problem is formulated as a mixed integer non-linear programming (MINLP). Due to its intractability and computational complexity, the MINLP problem is transformed into a convex optimization problem via a binary variable relaxation approach. Based on the mathematical analysis of the problem, a heuristic algorithm for joint user assignment and bandwidth allocation is presented. The proposed solution achieves a near optimal user assignment and bandwidth allocation at reduced computational complexity. Lastly, we investigate the transition between traditional hexagonal BS deployment to random BS placement in HetNets. Independent Poisson Point Processes (PPPs) are used to model the random locations of BSs. Lloyds algorithm is investigated for analyzing the coverage probability in a network which functions as a bridge between random and structural BS deployments. The link distance distribution is obtained by using the Expectation-Maximization (EM) algorithm which is further utilized for calculating the coverage probability

Efficiency and Sustainability of the Distributed Renewable Hybrid Power Systems Based on the Energy Internet, Blockchain Technology and Smart Contracts

The climate changes that are visible today are a challenge for the global research community. In this context, renewable energy sources, fuel cell systems, and other energy generating sources must be optimally combined and connected to the grid system using advanced energy transaction methods. As this book presents the latest solutions in the implementation of fuel cell and renewable energy in mobile and stationary applications such as hybrid and microgrid power systems based on energy internet, blockchain technology, and smart contracts, we hope that they are of interest to readers working in the related fields mentioned above

A patient agent controlled customized blockchain based framework for internet of things

Although Blockchain implementations have emerged as revolutionary technologies for various industrial applications including cryptocurrencies, they have not been widely deployed to store data streaming from sensors to remote servers in architectures known as Internet of Things. New Blockchain for the Internet of Things models promise secure solutions for eHealth, smart cities, and other applications. These models pave the way for continuous monitoring of patient’s physiological signs with wearable sensors to augment traditional medical practice without recourse to storing data with a trusted authority. However, existing Blockchain algorithms cannot accommodate the huge volumes, security, and privacy requirements of health data. In this thesis, our first contribution is an End-to-End secure eHealth architecture that introduces an intelligent Patient Centric Agent. The Patient Centric Agent executing on dedicated hardware manages the storage and access of streams of sensors generated health data, into a customized Blockchain and other less secure repositories. As IoT devices cannot host Blockchain technology due to their limited memory, power, and computational resources, the Patient Centric Agent coordinates and communicates with a private customized Blockchain on behalf of the wearable devices. While the adoption of a Patient Centric Agent offers solutions for addressing continuous monitoring of patients’ health, dealing with storage, data privacy and network security issues, the architecture is vulnerable to Denial of Services(DoS) and single point of failure attacks. To address this issue, we advance a second contribution; a decentralised eHealth system in which the Patient Centric Agent is replicated at three levels: Sensing Layer, NEAR Processing Layer and FAR Processing Layer. The functionalities of the Patient Centric Agent are customized to manage the tasks of the three levels. Simulations confirm protection of the architecture against DoS attacks. Few patients require all their health data to be stored in Blockchain repositories but instead need to select an appropriate storage medium for each chunk of data by matching their personal needs and preferences with features of candidate storage mediums. Motivated by this context, we advance third contribution; a recommendation model for health data storage that can accommodate patient preferences and make storage decisions rapidly, in real-time, even with streamed data. The mapping between health data features and characteristics of each repository is learned using machine learning. The Blockchain’s capacity to make transactions and store records without central oversight enables its application for IoT networks outside health such as underwater IoT networks where the unattended nature of the nodes threatens their security and privacy. However, underwater IoT differs from ground IoT as acoustics signals are the communication media leading to high propagation delays, high error rates exacerbated by turbulent water currents. Our fourth contribution is a customized Blockchain leveraged framework with the model of Patient-Centric Agent renamed as Smart Agent for securely monitoring underwater IoT. Finally, the smart Agent has been investigated in developing an IoT smart home or cities monitoring framework. The key algorithms underpinning to each contribution have been implemented and analysed using simulators.Doctor of Philosoph

Explorer l’hétérogénéité dans la réplication de données décentralisées faiblement cohérentes

Decentralized systems are scalable by design but also difficult to coordinate due to their weak coupling. Replicating data in these geo-distributed systems is therefore a challenge inherent to their structure. The two contributions of this thesis exploit the heterogeneity of user requirements and enable personalizable quality of services for data replication in decentralized systems. Our first contribution Gossip Primary-Secondary enables the consistency criterion Update consistency Primary-Secondary to offer differentiated guarantees in terms of consistency and message delivery latency for large-scale data replication. Our second contribution Dietcoin enriches Bitcoin with diet nodes that can (i) verify the correctness of entire subchains of blocks while avoiding the exorbitant cost of bootstrap verification and (ii) personalize their own security and resource consumption guarantees.Les systèmes décentralisés sont par nature extensibles mais sont également difficiles à coordonner en raison de leur faible couplage. La réplication de données dans ces systèmes géo-répartis est donc un défi inhérent à leur structure. Les deux contributions de cette thèse exploitent l'hétérogénéité des besoins des utilisateurs et permettent une qualité de service personnalisable pour la réplication de données dans les systèmes décentralisés. Notre première contribution Gossip Primary-Secondary étend le critère de cohérence Update consistency Primary-Secondary afin d'offrir des garanties différenciées de cohérence et de latence de messages pour la réplication de données à grande échelle. Notre seconde contribution Dietcoin enrichit Bitcoin avec des nœuds diet qui peuvent (i) vérifier la validité de sous-chaînes de blocs en évitant le coût exorbitant de la vérification initiale et (ii) choisir leur propres garanties de sécurité et de consommation de ressources