Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems

This paper was motivated by the problem of how to make robots fuse and transfer their experience so that they can effectively use prior knowledge and quickly adapt to new environments. To address the problem, we present a learning architecture for navigation in cloud robotic systems: Lifelong Federated Reinforcement Learning (LFRL). In the work, We propose a knowledge fusion algorithm for upgrading a shared model deployed on the cloud. Then, effective transfer learning methods in LFRL are introduced. LFRL is consistent with human cognitive science and fits well in cloud robotic systems. Experiments show that LFRL greatly improves the efficiency of reinforcement learning for robot navigation. The cloud robotic system deployment also shows that LFRL is capable of fusing prior knowledge. In addition, we release a cloud robotic navigation-learning website based on LFRL

A dynamic pricing algorithm for a network of virtual resources

A service chain is a combination of network services (e.g. network address translation (NAT), a firewall, etc.) that are interconnected to support an application (e.g. video-on-demand). Building a service chain requires a set of specialized hardware devices each of which need to be configured with their own command syntax. By moving management functions out of forwarding hardware into controller software, software-defined networking (SDN) simplifies provisioning and reconfiguration of service chains. By moving the network functions out of dedicated hardware devices into software running on standard x86 servers, network function virtualization (NFV) turns the deployment of a service chain into a more (cost)-efficient and flexible process. In an SDN/NFV-based architecture, those service chains are composed of virtual network functions (VNFs) that need to be mapped to physical network components. In literature, several algorithmic approaches exist to do so efficiently and cost-effectively. However, once mapped, a simple revenue model is used for pricing the requested substrate resources. This often leads to a loss of revenue for the infrastructure provider. In this paper, we propose a more advanced, dynamic pricing algorithm for pricing the requested substrate resources. The proposed algorithm increases the infrastructure provider's revenue based on historic data, current infrastructure utilization levels and the pricing of competitors. Our experimental evaluation shows that the proposed algorithm increases the revenue of the infrastructure provider significantly, independent of the average network utilization.Peer Reviewe

Effective and Secure Healthcare Machine Learning System with Explanations Based on High Quality Crowdsourcing Data

Affordable cloud computing technologies allow users to efficiently outsource, store, and manage their Personal Health Records (PHRs) and share with their caregivers or physicians. With this exponential growth of the stored large scale clinical data and the growing need for personalized care, researchers are keen on developing data mining methodologies to learn efficient hidden patterns in such data. While studies have shown that those progresses can significantly improve the performance of various healthcare applications for clinical decision making and personalized medicine, the collected medical datasets are highly ambiguous and noisy. Thus, it is essential to develop a better tool for disease progression and survival rate predictions, where dataset needs to be cleaned before it is used for predictions and useful feature selection techniques need to be employed before prediction models can be constructed. In addition, having predictions without explanations prevent medical personnel and patients from adopting such healthcare deep learning models. Thus, any prediction models must come with some explanations. Finally, despite the efficiency of machine learning systems and their outstanding prediction performance, it is still a risk to reuse pre-trained models since most machine learning modules that are contributed and maintained by third parties lack proper checking to ensure that they are robust to various adversarial attacks. We need to design mechanisms for detection such attacks. In this thesis, we focus on addressing all the above issues: (i) Privacy Preserving Disease Treatment & Complication Prediction System (PDTCPS): A privacy-preserving disease treatment, complication prediction scheme (PDTCPS) is proposed, which allows authorized users to conduct searches for disease diagnosis, personalized treatments, and prediction of potential complications. (ii) Incentivizing High Quality Crowdsourcing Data For Disease Prediction: A new incentive model with individual rationality and platform profitability features is developed to encourage different hospitals to share high quality data so that better prediction models can be constructed. We also explore how data cleaning and feature selection techniques affect the performance of the prediction models. (iii) Explainable Deep Learning Based Medical Diagnostic System: A deep learning based medical diagnosis system (DL-MDS) is present which integrates heterogeneous medical data sources to produce better disease diagnosis with explanations for authorized users who submit their personalized health related queries. (iv) Attacks on RNN based Healthcare Learning Systems and Their Detection & Defense Mechanisms: Potential attacks on Recurrent Neural Network (RNN) based ML systems are identified and low-cost detection & defense schemes are designed to prevent such adversarial attacks. Finally, we conduct extensive experiments using both synthetic and real-world datasets to validate the feasibility and practicality of our proposed systems

A multi-domain VNE algorithm based on load balancing in the IoT networks

The coordinated development of big data, Internet of Things, cloud computing and other technologies has led to an exponential growth in Internet business. However, the traditional Internet architecture gradually shows a rigid phenomenon due to the binding of the network structure and the hardware. In a high-traffic environment, it has been insufficient to meet people’s increasing service quality requirements. Network virtualization is considered to be an effective method to solve the rigidity of the Internet. Among them, virtual network embedding is one of the key problems of network virtualization. Since virtual network mapping is an NP-hard problem, a large number of research has focused on the evolutionary algorithm’s masterpiece genetic algorithm. However, the parameter setting in the traditional method is too dependent on experience, and its low flexibility makes it unable to adapt to increasingly complex network environments. In addition, link-mapping strategies that do not consider load balancing can easily cause link blocking in high-traffic environments. In the IoT environment involving medical, disaster relief, life support and other equipment, network performance and stability are particularly important. Therefore, how to provide a more flexible virtual network mapping service in a heterogeneous network environment with large traffic is an urgent problem. Aiming at this problem, a virtual network mapping strategy based on hybrid genetic algorithm is proposed. This strategy uses a dynamically calculated cross-probability and pheromone based mutation gene selection strategy to improve the flexibility of the algorithm. In addition, a weight update mechanism based on load balancing is introduced to reduce the probability of mapping failure while balancing the load. Simulation results show that the proposed method performs well in a number of performance metrics including mapping average quotation, link load balancing, mapping cost-benefit ratio, acceptance rate and running time.Peer ReviewedPostprint (published version