Learning Density-Based Correlated Equilibria for Markov Games

Correlated Equilibrium (CE) is a well-established solution concept that captures coordination among agents and enjoys good algorithmic properties. In real-world multi-agent systems, in addition to being in an equilibrium, agents' policies are often expected to meet requirements with respect to safety, and fairness. Such additional requirements can often be expressed in terms of the state density which measures the state-visitation frequencies during the course of a game. However, existing CE notions or CE-finding approaches cannot explicitly specify a CE with particular properties concerning state density; they do so implicitly by either modifying reward functions or using value functions as the selection criteria. The resulting CE may thus not fully fulfil the state-density requirements. In this paper, we propose Density-Based Correlated Equilibria (DBCE), a new notion of CE that explicitly takes state density as selection criterion. Concretely, we instantiate DBCE by specifying different state-density requirements motivated by real-world applications. To compute DBCE, we put forward the Density Based Correlated Policy Iteration algorithm for the underlying control problem. We perform experiments on various games where results demonstrate the advantage of our CE-finding approach over existing methods in scenarios with state-density concerns

Auction-Based Efficient Online Incentive Mechanism Designs in Wireless Networks

Recently, wide use of mobile devices and applications, such as YouTube and Twitter, has facilitated every aspect of our daily lives. Meanwhile, it has also posed great challenges to enable resource-demanding users to successfully access networks. Thus, in order to enlarge network capacity and fully make use of vacant resources, new communication architectures emerge, such as D2D communications, edge computing, and crowdsourcing, all of which ask for involvement of end mobile users in assisting transmission, computation, or network management. However, end mobile users are not always willing to actively provide such sharing services if no reimbursements are provided as they need to consume their own computation and communication resources. Besides, since mobile users are not always stationary, they can opt-in and opt-out the network for their own convenience. Thus, an important practical characteristic of wireless networks, i.e., the mobility of mobile users cannot be ignored, which means that the demands of mobile users span over a period of time. As one of promising solutions, the online incentive mechanism design has been introduced in wireless networks in order to motivate the participation of more mobile users under a dynamic environment. In this thesis, with the analyses of each stakeholder's economic payoffs in wireless networks, the auction-based online incentive mechanisms are proposed to achieve resource allocations, participant selections, and payment determinations in two wireless networks, i.e., Crowdsensing and mobile edge computing. In particular, i) an online incentive mechanism is designed to guarantee Quality of Information of each arriving task in mobile crowdsensing networks, followed by an enhanced online strategy which could further improves the competitive ratio; ii) an online incentive mechanism jointly considering communication and computation resource allocations in collaborative edge computing networks is proposed based on the primal-dual theory; iii) to deal with the nonlinear issue in edge computing networks, an nonlinear online incentive mechanism under energy budget constraints of mobile users is designed based on the Maximal-in-Distributional Range framework; and iv) inspired by the recent development of deep learning techniques, a deep incentive mechanism with the budget balance of each mobile user is proposed to maximize the net revenue of service providers by leveraging the multi-task machine learning model. Both theoretical analyses and numerical results demonstrate the effectiveness of the designed mechanisms

Learning a Correlated Equilibrium with Perturbed Regret Minimization

International audienceIn this paper, we consider the problem of learning a correlated equilibrium of a finite non-cooperative game and show a new adaptive heuristic, called Correlated Perturbed Regret Minimization (CPRM) for this purpose. CPRM combines regret minimization to approach the set of correlated equilibria and a simple device suggesting actions to the players to further stabilize the dynamic. Numerical experiments support the hypothesis of the pointwise convergence of the empirical distribution over action profiles to an approximate correlated equilibrium with all players following the devices' suggestions. Additional simulation results suggest that CPRM is adaptive to changes in the game such as departures or arrivals of players

A Survey on Mobile Crowdsensing Systems: Challenges, Solutions, and Opportunities

Mobile crowdsensing (MCS) has gained significant attention in recent years and has become an appealing paradigm for urban sensing. For data collection, MCS systems rely on contribution from mobile devices of a large number of participants or a crowd. Smartphones, tablets, and wearable devices are deployed widely and already equipped with a rich set of sensors, making them an excellent source of information. Mobility and intelligence of humans guarantee higher coverage and better context awareness if compared to traditional sensor networks. At the same time, individuals may be reluctant to share data for privacy concerns. For this reason, MCS frameworks are specifically designed to include incentive mechanisms and address privacy concerns. Despite the growing interest in the research community, MCS solutions need a deeper investigation and categorization on many aspects that span from sensing and communication to system management and data storage. In this paper, we take the research on MCS a step further by presenting a survey on existing works in the domain and propose a detailed taxonomy to shed light on the current landscape and classify applications, methodologies, and architectures. Our objective is not only to analyze and consolidate past research but also to outline potential future research directions and synergies with other research areas

PLATFORM-DRIVEN CROWDSOURCED MANUFACTURING FOR MANUFACTURING AS A SERVICE

Platform-driven crowdsourced manufacturing is an emerging manufacturing paradigm to instantiate the adoption of the open business model in the context of achieving Manufacturing-as-a-Service (MaaS). It has attracted attention from both industries and academia as a powerful way of searching for manufacturing solutions extensively in a smart manufacturing era. In this regard, this work examines the origination and evolution of the open business model and highlights the trends towards platform-driven crowdsourced manufacturing as a solution for MaaS. Platform-driven crowdsourced manufacturing has a full function of value capturing, creation, and delivery approach, which is fulfilled by the cooperation among manufacturers, open innovators, and platforms. The platform-driven crowdsourced manufacturing workflow is proposed to organize these three decision agents by specifying the domains and interactions, following a functional, behavioral, and structural mapping model. A MaaS reference model is proposed to outline the critical functions and inter-relationships. A series of quantitative, qualitative, and computational solutions are developed for fulfilling the outlined functions. The case studies demonstrate the proposed methodologies and can pace the way towards a service-oriented product fulfillment process. This dissertation initially proposes a manufacturing theory and decision models by integrating manufacturer crowds through a cyber platform. This dissertation reveals the elementary conceptual framework based on stakeholder analysis, including dichotomy analysis of industrial applicability, decision agent identification, workflow, and holistic framework of platform-driven crowdsourced manufacturing. Three stakeholders require three essential service fields, and their cooperation requires an information service system as a kernel. These essential functions include contracting evaluation services for open innovators, manufacturers' task execution services, and platforms' management services. This research tackles these research challenges to provide a technology implementation roadmap and transition guidebook for industries towards crowdsourcing.Ph.D

Performance Modeling of Vehicular Clouds Under Different Service Strategies

The amount of data being generated at the edge of the Internet is rapidly rising as a result of the Internet of Things (IoT). Vehicles themselves are contributing enormously to data generation with their advanced sensor systems. This data contains contextual information; it's temporal and needs to be processed in real-time to be of any value. Transferring this data to the cloud is not feasible due to high cost and latency. This has led to the introduction of edge computing for processing of data close to the source. However, edge servers may not have the computing capacity to process all the data. Future vehicles will have significant computing power, which may be underutilized, and they may have a stake in the processing of the data. This led to the introduction of a new computing paradigm called vehicular cloud (VC), which consists of interconnected vehicles that can share resources and communicate with each other. The VCs may process the data by themselves or in cooperation with edge servers. Performance modeling of VCs is important, as it will help to determine whether it can provide adequate service to users. It will enable determining appropriate service strategies and the type of jobs that may be served by the VC such that Quality of service (QoS) requirements are met. Job completion time and throughput of VCs are important performance metrics. However, performance modeling of VCs is difficult because of the volatility of resources. As vehicles join and leave the VC, available resources vary in time. Performance evaluation results in the literature are lacking, and available results mostly pertain to stationary VCs formed from parked vehicles. This thesis proposes novel stochastic models for the performance evaluation of vehicular cloud systems that take into account resource volatility, composition of jobs from multiple tasks that can execute concurrently under different service strategies. First, we developed a stochastic model to analyze the job completion time in a VC system deployed on a highway with service interruption. Next, we developed a model to analyze the job completion time in a VC system with a service interruption avoidance strategy. This strategy aims to prevent disruptions in task service by only assigning tasks to vehicles that can complete the tasks’ execution before they leave the VC. In addition to analyzing job completion time, we evaluated the computing capacity of VC systems with a service interruption avoidance strategy, determining the number of jobs a VC system can complete during its lifetime. Finally, we studied the computing capacity of a robotaxi fleet, analyzing the average number of tasks that a robotaxi fleet can serve to completion during a cycle. By developing these models, conducting various analyses, and comparing the numerical results of the analyses to extensive Monte Carlo simulation results, we gained insights into job completion time, computing capacity, and overall performance of VC systems deployed in different contexts

Performance Modeling of Vehicular Clouds Under Different Service Strategies

The amount of data being generated at the edge of the Internet is rapidly rising as a result of the Internet of Things (IoT). Vehicles themselves are contributing enormously to data generation with their advanced sensor systems. This data contains contextual information; it's temporal and needs to be processed in real-time to be of any value. Transferring this data to the cloud is not feasible due to high cost and latency. This has led to the introduction of edge computing for processing of data close to the source. However, edge servers may not have the computing capacity to process all the data. Future vehicles will have significant computing power, which may be underutilized, and they may have a stake in the processing of the data. This led to the introduction of a new computing paradigm called vehicular cloud (VC), which consists of interconnected vehicles that can share resources and communicate with each other. The VCs may process the data by themselves or in cooperation with edge servers. Performance modeling of VCs is important, as it will help to determine whether it can provide adequate service to users. It will enable determining appropriate service strategies and the type of jobs that may be served by the VC such that Quality of service (QoS) requirements are met. Job completion time and throughput of VCs are important performance metrics. However, performance modeling of VCs is difficult because of the volatility of resources. As vehicles join and leave the VC, available resources vary in time. Performance evaluation results in the literature are lacking, and available results mostly pertain to stationary VCs formed from parked vehicles. This thesis proposes novel stochastic models for the performance evaluation of vehicular cloud systems that take into account resource volatility, composition of jobs from multiple tasks that can execute concurrently under different service strategies. First, we developed a stochastic model to analyze the job completion time in a VC system deployed on a highway with service interruption. Next, we developed a model to analyze the job completion time in a VC system with a service interruption avoidance strategy. This strategy aims to prevent disruptions in task service by only assigning tasks to vehicles that can complete the tasks’ execution before they leave the VC. In addition to analyzing job completion time, we evaluated the computing capacity of VC systems with a service interruption avoidance strategy, determining the number of jobs a VC system can complete during its lifetime. Finally, we studied the computing capacity of a robotaxi fleet, analyzing the average number of tasks that a robotaxi fleet can serve to completion during a cycle. By developing these models, conducting various analyses, and comparing the numerical results of the analyses to extensive Monte Carlo simulation results, we gained insights into job completion time, computing capacity, and overall performance of VC systems deployed in different contexts