Vehicular Edge Computing via Deep Reinforcement Learning

The smart vehicles construct Vehicle of Internet which can execute various intelligent services. Although the computation capability of the vehicle is limited, multi-type of edge computing nodes provide heterogeneous resources for vehicular services.When offloading the complicated service to the vehicular edge computing node, the decision should consider numerous factors.The offloading decision work mostly formulate the decision to a resource scheduling problem with single or multiple objective function and some constraints, and explore customized heuristics algorithms. However, offloading multiple data dependency tasks in a service is a difficult decision, as an optimal solution must understand the resource requirement, the access network, the user mobility, and importantly the data dependency. Inspired by recent advances in machine learning, we propose a knowledge driven (KD) service offloading decision framework for Vehicle of Internet, which provides the optimal policy directly from the environment. We formulate the offloading decision of multi-task in a service as a long-term planning problem, and explores the recent deep reinforcement learning to obtain the optimal solution. It considers the future data dependency of the following tasks when making decision for a current task from the learned offloading knowledge. Moreover, the framework supports the pre-training at the powerful edge computing node and continually online learning when the vehicular service is executed, so that it can adapt the environment changes and learns policy that are sensible in hindsight. The simulation results show that KD service offloading decision converges quickly, adapts to different conditions, and outperforms the greedy offloading decision algorithm.Comment: Preliminary report of ongoing wor

Unsupervised Online Bayesian Autonomic Happy Internet-of-Things Management

In Happy IoT, the revenue of service providers synchronizes to the unobservable and dynamic usage-contexts (e.g. emotion, environmental information, etc.) of Smart-device users. Hence, the usage-context-estimation from the unreliable Smart-device sensed data is justified as an unsupervised and non-linear optimization problem. Accordingly, Autonomic Happy IoT Management is aimed at attracting initial user-groups based on the common interests (i.e. recruitment ), then uncovering their latent usage-contexts from unreliable sensed data (i.e. revenue-renewal ) and synchronizing to usage-context dynamics (i.e. stochastic monetization). In this context, we have proposed an unsupervised online Bayesian mechanism, namely Whiz (Greek word, meaning Smart), in which, (a) once latent user-groups are initialized (i.e measurement model ), (b) usage-context is iteratively estimated from the unreliable sensed data (i.e. learning model ), (c) followed by online filtering of Bayesian knowledge about usage-context (i.e. filtering model ). Finally, we have proposed an Expectation Maximization (EM)-based iterative algorithm Whiz, which facilitates Happy IoT by solving (a) recruitment, (b) revenue-renewal and (c) stochastic- monetization problems with (a) measurement, (b) learning, and (c) filtering models, respectively

Deep Bayesian Multi-Target Learning for Recommender Systems

With the increasing variety of services that e-commerce platforms provide, criteria for evaluating their success become also increasingly multi-targeting. This work introduces a multi-target optimization framework with Bayesian modeling of the target events, called Deep Bayesian Multi-Target Learning (DBMTL). In this framework, target events are modeled as forming a Bayesian network, in which directed links are parameterized by hidden layers, and learned from training samples. The structure of Bayesian network is determined by model selection. We applied the framework to Taobao live-streaming recommendation, to simultaneously optimize (and strike a balance) on targets including click-through rate, user stay time in live room, purchasing behaviors and interactions. Significant improvement has been observed for the proposed method over other MTL frameworks and the non-MTL model. Our practice shows that with an integrated causality structure, we can effectively make the learning of a target benefit from other targets, creating significant synergy effects that improve all targets. The neural network construction guided by DBMTL fits in with the general probabilistic model connecting features and multiple targets, taking weaker assumption than the other methods discussed in this paper. This theoretical generality brings about practical generalization power over various targets distributions, including sparse targets and continuous-value ones.Comment: 7 pages, Deep Learning, Probabilistic Machine Learning, Recommender System, Multi-task Learnin

Bayesian Optimization for Policy Search via Online-Offline Experimentation

Online field experiments are the gold-standard way of evaluating changes to real-world interactive machine learning systems. Yet our ability to explore complex, multi-dimensional policy spaces - such as those found in recommendation and ranking problems - is often constrained by the limited number of experiments that can be run simultaneously. To alleviate these constraints, we augment online experiments with an offline simulator and apply multi-task Bayesian optimization to tune live machine learning systems. We describe practical issues that arise in these types of applications, including biases that arise from using a simulator and assumptions for the multi-task kernel. We measure empirical learning curves which show substantial gains from including data from biased offline experiments, and show how these learning curves are consistent with theoretical results for multi-task Gaussian process generalization. We find that improved kernel inference is a significant driver of multi-task generalization. Finally, we show several examples of Bayesian optimization efficiently tuning a live machine learning system by combining offline and online experiments

Interactive Sensing in Social Networks

This paper presents models and algorithms for interactive sensing in social networks where individuals act as sensors and the information exchange between individuals is exploited to optimize sensing. Social learning is used to model the interaction between individuals that aim to estimate an underlying state of nature. In this context the following questions are addressed: How can self-interested agents that interact via social learning achieve a tradeoff between individual privacy and reputation of the social group? How can protocols be designed to prevent data incest in online reputation blogs where individuals make recommendations? How can sensing by individuals that interact with each other be used by a global decision maker to detect changes in the underlying state of nature? When individual agents possess limited sensing, computation and communication capabilities, can a network of agents achieve sophisticated global behavior? Social and game theoretic learning are natural settings for addressing these questions. This article presents an overview, insights and discussion of social learning models in the context of data incest propagation, change detection and coordination of decision making

Leveraging Machine Learning and Big Data for Smart Buildings: A Comprehensive Survey

Future buildings will offer new convenience, comfort, and efficiency possibilities to their residents. Changes will occur to the way people live as technology involves into people's lives and information processing is fully integrated into their daily living activities and objects. The future expectation of smart buildings includes making the residents' experience as easy and comfortable as possible. The massive streaming data generated and captured by smart building appliances and devices contains valuable information that needs to be mined to facilitate timely actions and better decision making. Machine learning and big data analytics will undoubtedly play a critical role to enable the delivery of such smart services. In this paper, we survey the area of smart building with a special focus on the role of techniques from machine learning and big data analytics. This survey also reviews the current trends and challenges faced in the development of smart building services

Making life better one large system at a time: Challenges for UAI research

The rapid growth and diversity in service offerings and the ensuing complexity of information technology ecosystems present numerous management challenges (both operational and strategic). Instrumentation and measurement technology is, by and large, keeping pace with this development and growth. However, the algorithms, tools, and technology required to transform the data into relevant information for decision making are not. The claim in this paper (and the invited talk) is that the line of research conducted in Uncertainty in Artificial Intelligence is very well suited to address the challenges and close this gap. I will support this claim and discuss open problems using recent examples in diagnosis, model discovery, and policy optimization on three real life distributed systems.Comment: Appears in Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence (UAI2007

Intelligent Wireless Communications Enabled by Cognitive Radio and Machine Learning

The ability to intelligently utilize resources to meet the need of growing diversity in services and user behavior marks the future of wireless communication systems. Intelligent wireless communications aims at enabling the system to perceive and assess the available resources, to autonomously learn to adapt to the perceived wireless environment, and to reconfigure its operating mode to maximize the utility of the available resources. The perception capability and reconfigurability are the essential features of cognitive radio while modern machine learning techniques project great potential in system adaptation. In this paper, we discuss the development of the cognitive radio technology and machine learning techniques and emphasize their roles in improving spectrum and energy utility of wireless communication systems. We describe the state-of-the-art of relevant techniques, covering spectrum sensing and access approaches and powerful machine learning algorithms that enable spectrum- and energy-efficient communications in dynamic wireless environments. We also present practical applications of these techniques and identify further research challenges in cognitive radio and machine learning as applied to the existing and future wireless communication systems

From 4G to 5G: Self-organized Network Management meets Machine Learning

In this paper, we provide an analysis of self-organized network management, with an end-to-end perspective of the network. Self-organization as applied to cellular networks is usually referred to Self-organizing Networks (SONs), and it is a key driver for improving Operations, Administration, and Maintenance (OAM) activities. SON aims at reducing the cost of installation and management of 4G and future 5G networks, by simplifying operational tasks through the capability to configure, optimize and heal itself. To satisfy 5G network management requirements, this autonomous management vision has to be extended to the end to end network. In literature and also in some instances of products available in the market, Machine Learning (ML) has been identified as the key tool to implement autonomous adaptability and take advantage of experience when making decisions. In this paper, we survey how network management can significantly benefit from ML solutions. We review and provide the basic concepts and taxonomy for SON, network management and ML. We analyse the available state of the art in the literature, standardization, and in the market. We pay special attention to 3rd Generation Partnership Project (3GPP) evolution in the area of network management and to the data that can be extracted from 3GPP networks, in order to gain knowledge and experience in how the network is working, and improve network performance in a proactive way. Finally, we go through the main challenges associated with this line of research, in both 4G and in what 5G is getting designed, while identifying new directions for research.Comment: 23 pages, 3 figures, Surve

A QoS Guarantee Strategy for Multimedia Conferencing based on Bayesian Networks

Service Oriented Architecture (SOA) is commonly employed in the design and implementation of web service systems. The key technology to enable media communications in the context of SOA is the Service Oriented Communication. To exploit the advantage of SOA, we design and implement a web-based multimedia conferencing system that provides users with a hybrid orchestration of web and communication services. As the current SOA lacks effective QoS guarantee solutions for multimedia services, the user satisfaction is greatly challenged with QoS violations, e.g., low video PSNR (Peak Signal-to-Noise Ratio) and long playback delay. Motivated by addressing the critical problem, we firstly employ the Business Process Execution Language (BPEL) service engine for the hybrid services orchestration and execution. Secondly, we propose a novel context-aware approach to quantify and leverage the causal relationships between QoS metrics and available contexts based on Bayesian networks (CABIN). This approach includes three phases: (1) information discretization, (2) causal relationship profiling, and (3) optimal context tuning. We implement CABIN in a real-life multimedia conferencing system and compare its performance with existing delay and throughput oriented schemes. Experimental results show that CABIN outperforms the competing approaches in improving the video quality in terms of PSNR. It also provides a one-stop shop controls both the web and communication services.Comment: 9 page