HCI Model with Learning Mechanism for Cooperative Design in Pervasive Computing Environment

This paper presents a human-computer interaction model with a three layers learning mechanism in a pervasive environment. We begin with a discussion around a number of important issues related to human-computer interaction followed by a description of the architecture for a multi-agent cooperative design system for pervasive computing environment. We present our proposed three- layer HCI model and introduce the group formation algorithm, which is predicated on a dynamic sharing niche technology. Finally, we explore the cooperative reinforcement learning and fusion algorithms; the paper closes with concluding observations and a summary of the principal work and contributions of this paper

Cost-efficient Cooperative Video Caching Over Edge Networks

Cooperative caching has emerged as an efficient way to alleviate backhaul traffic and enhance user experience by proactively prefetching popular videos at the network edge. However, it is challenging to achieve the optimal design of video caching, sharing, and delivery within storage-limited edge networks due to the growing diversity of videos, unpredictable video requirements, and dynamic user preferences. To address this challenge, this work explores cost-efficient cooperative video caching via video compression techniques while considering unknown video popularity. Firstly, we formulate the joint video caching, sharing, and delivery problem to capture a balance between user delay and system operative cost under unknown time-varying video popularity. To solve this problem, we develop a two-layer decentralized reinforcement learning algorithm, which effectively reduces the action space and tackles the coupling among video caching, sharing, and delivery decisions compared to the conventional algorithms. Specifically, the outer layer produces the optimal decisions for video caching and communication resource allocation by employing a multi-agent deep deterministic policy gradient algorithm. Meanwhile, the optimal video sharing and computation resource allocation are determined in each agent’s inner layer using the alternating optimization algorithm. Numerical results show that the proposed algorithm outperforms benchmarks in terms of the cache hit rate, delay of users and system operative cost, and effectively strikes a trade-off between system operative cost and users’ delay

Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios

Communication technologies enable coordination among connected and autonomous vehicles (CAVs). However, it remains unclear how to utilize shared information to improve the safety and efficiency of the CAV system. In this work, we propose a framework of constrained multi-agent reinforcement learning (MARL) with a parallel safety shield for CAVs in challenging driving scenarios. The coordination mechanisms of the proposed MARL include information sharing and cooperative policy learning, with Graph Convolutional Network (GCN)-Transformer as a spatial-temporal encoder that enhances the agent's environment awareness. The safety shield module with Control Barrier Functions (CBF)-based safety checking protects the agents from taking unsafe actions. We design a constrained multi-agent advantage actor-critic (CMAA2C) algorithm to train safe and cooperative policies for CAVs. With the experiment deployed in the CARLA simulator, we verify the effectiveness of the safety checking, spatial-temporal encoder, and coordination mechanisms designed in our method by comparative experiments in several challenging scenarios with the defined hazard vehicles (HAZV). Results show that our proposed methodology significantly increases system safety and efficiency in challenging scenarios.Comment: This paper has been accepted by the 2023 IEEE International Conference on Robotics and Automation (ICRA 2023). 6 pages, 5 figure

Efficient Core-selecting Incentive Mechanism for Data Sharing in Federated Learning

Federated learning is a distributed machine learning system that uses participants' data to train an improved global model. In federated learning, participants cooperatively train a global model, and they will receive the global model and payments. Rational participants try to maximize their individual utility, and they will not input their high-quality data truthfully unless they are provided with satisfactory payments based on their data quality. Furthermore, federated learning benefits from the cooperative contributions of participants. Accordingly, how to establish an incentive mechanism that both incentivizes inputting data truthfully and promotes stable cooperation has become an important issue to consider. In this paper, we introduce a data sharing game model for federated learning and employ game-theoretic approaches to design a core-selecting incentive mechanism by utilizing a popular concept in cooperative games, the core. In federated learning, the core can be empty, resulting in the core-selecting mechanism becoming infeasible. To address this, our core-selecting mechanism employs a relaxation method and simultaneously minimizes the benefits of inputting false data for all participants. However, this mechanism is computationally expensive because it requires aggregating exponential models for all possible coalitions, which is infeasible in federated learning. To address this, we propose an efficient core-selecting mechanism based on sampling approximation that only aggregates models on sampled coalitions to approximate the exact result. Extensive experiments verify that the efficient core-selecting mechanism can incentivize inputting high-quality data and stable cooperation, while it reduces computational overhead compared to the core-selecting mechanism

The Role of Group Learning in Implementation of a Personnel Management System in a Hospital

A new HR system was introduced in a Dutch hospital. The system implied collaborative work among its users. The project planning seemed to be reasonably straightforward: the system's introduction was intended to take place gradually, including pilots in different departments and appropriate feedback. After some time, the system was successfully adopted by one group of users, but failed with another. We conceptualize the implementation process of groupware as group learning to frame the adoption of the system, and analyze the qualitative data collected during the longitudinal case study. We found that in the user group with strong group learning, adoption of the system occurred effectively and on time. In another user group with rather weak group learning, the use of the system was blocked after a short time. The results provided a first confirmation of our assumption about the importance of group learning processes in the implementation of groupware

Cost-efficient Cooperative Video Caching Over Edge Networks

Cooperative caching has emerged as an efficient way to alleviate backhaul traffic and enhance user experience by proactively prefetching popular videos at the network edge. However, it is challenging to achieve the optimal design of video caching, sharing, and delivery within storage-limited edge networks due to the growing diversity of videos, unpredictable video requirements, and dynamic user preferences. To address this challenge, this work explores cost-efficient cooperative video caching via video compression techniques while considering unknown video popularity. First, we formulate the joint video caching, sharing, and delivery problem to capture a balance between user delay and system operative cost under unknown time-varying video popularity. To solve this problem, we develop a two-layer decentralized reinforcement learning algorithm, which effectively reduces the action space and tackles the coupling among video caching, sharing, and delivery decisions compared to the conventional algorithms. Specifically, the outer layer produces the optimal decisions for video caching and communication resource allocation by employing a multiagent deep deterministic policy gradient algorithm. Meanwhile, the optimal video sharing and computation resource allocation are determined in each agent’s inner layer using the alternating optimization algorithm. Numerical results show that the proposed algorithm outperforms benchmarks in terms of the cache hit rate, delay of users and system operative cost, and effectively strikes a tradeoff between system operative cost and users’ delay