Effective Multi-Agent Deep Reinforcement Learning Control with Relative Entropy Regularization

In this paper, a novel Multi-agent Reinforcement Learning (MARL) approach, Multi-Agent Continuous Dynamic Policy Gradient (MACDPP) was proposed to tackle the issues of limited capability and sample efficiency in various scenarios controlled by multiple agents. It alleviates the inconsistency of multiple agents' policy updates by introducing the relative entropy regularization to the Centralized Training with Decentralized Execution (CTDE) framework with the Actor-Critic (AC) structure. Evaluated by multi-agent cooperation and competition tasks and traditional control tasks including OpenAI benchmarks and robot arm manipulation, MACDPP demonstrates significant superiority in learning capability and sample efficiency compared with both related multi-agent and widely implemented signal-agent baselines and therefore expands the potential of MARL in effectively learning challenging control scenarios

Conflict-Aware Real-Time Routing for Industrial Wireless Sensor-Actuator Networks

Process industries are adopting wireless sensor-actuator networks (WSANs) as the communication infrastructure. WirelessHART is an open industrial standard for WSANs that have seen world-wide deployments. Real-time scheduling and delay analysis have been studied for WSAN extensively. End-to-end delay in WSANs highly depends on routing, which is still open problem. This paper presents the first real-time routing design for WSAN. We first discuss end-to-end delays of WSANs, then present our real-time routing design. We have implemented and experimented our routing designs on a wireless testbed of 69 nodes. Both experimental results and simulations show that our routing design can improve the real-time performance significantly

Exploring Semantic Attributes from A Foundation Model for Federated Learning of Disjoint Label Spaces

Conventional centralised deep learning paradigms are not feasible when data from different sources cannot be shared due to data privacy or transmission limitation. To resolve this problem, federated learning has been introduced to transfer knowledge across multiple sources (clients) with non-shared data while optimising a globally generalised central model (server). Existing federated learning paradigms mostly focus on transferring holistic high-level knowledge (such as class) across models, which are closely related to specific objects of interest so may suffer from inverse attack. In contrast, in this work, we consider transferring mid-level semantic knowledge (such as attribute) which is not sensitive to specific objects of interest and therefore is more privacy-preserving and scalable. To this end, we formulate a new Federated Zero-Shot Learning (FZSL) paradigm to learn mid-level semantic knowledge at multiple local clients with non-shared local data and cumulatively aggregate a globally generalised central model for deployment. To improve model discriminative ability, we propose to explore semantic knowledge augmentation from external knowledge for enriching the mid-level semantic space in FZSL. Extensive experiments on five zeroshot learning benchmark datasets validate the effectiveness of our approach for optimising a generalisable federated learning model with mid-level semantic knowledge transfer.Comment: Under Revie

Enterprise’s Strategies to Deal with Epidemic Crisis Based on Super-Dynamic Capability Theory

In this paper, the supply chain management risks arising from 2019-novel coronavirus (hereinafter referred to as “COVID-19”) outbreak was proposed, and they were further analyzed from three main aspects such as change in demand conditions of domestic customers, change in domestic supply market, impact on domestic logistics industry. Besides, multiple feasible strategies for coping with such epidemic situation were proposed for enterprises based on the super-dynamic capability theory. The research in this paper has powerful theoretical value and practical significance for the current development of enterprises, especially the reorganization of enterprises under the current epidemic crisis in China

A new experience mining approach for improving low carbon city development

Developing low carbon city (LCC) has been widely appreciated as an important strategy for sustainable development. In line with this, an increasing number of cities globally have launched low carbon practices in recent years and gained various types of experience. However, it appears that existing studies do not present methods of how to use these valuable LCC experience in solving new problems. This study therefore introduces an experience mining approach to assist decision‐makers in reusing previous experience when tailoring LCC development strategies. The mining approach consists of three processes, namely, collecting historical cases which have been experiencing LCC, establishing LCC experience base, and mining similar experience cases. This study innovates the existing experience mining approach by introducing a two‐step mining process with considering the perspective of problem‐based urban characteristics (PBUCs) and the perspective of solution‐based urban characteristics (SBUCs). The application of the introduced mining approach has been demonstrated by a case study, where Shenyang’s energy structure is adopted as the target problem. The new experience mining approach provides a valuable reference for decision‐makers to retrieve similar cases for improving LCC development with the consideration of city characteristics.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/156189/2/sd2046_am.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/156189/1/sd2046.pd

Peaks of transportation CO2 emissions of 119 countries for sustainable development: Results from carbon Kuznets curve

Transportation has significantly boomed energy consumption and carbon dioxide (CO2) emissions. Understanding and forecasting the dynamic statuses of transportation CO2 emissions is a necessary step before making strategies to decrease CO2 emissions. Carbon Kuznets curve (CKC) hypothesis has been frequently validated properly to present the changing statuses of CO2 emissions in the literature. This study tests the CKC hypothesis using the data recording the CO2 emissions of transportation sectors of 119 countries over the period of 1995–2014, then turning points (TPs) are calculated for the countries where CKC hypothesis is turned out supported. Based on the CKC models, this study identifies different types of TPs, i.e. TP of carbon intensity (TPCI), TP of per capita CO2 emissions (TPPC), and TP of total CO2 emissions (TPTC) of the countries whose data support the CKC hypothesis. According to the earliness of the turning years (TYs) (TYCI, TYPC and TYTC) – the years when CO2 emissions peak – of individual countries, this study identified a step‐wise decoupling strategy for different countries, i.e. (1) first to reach the TPCI, (2) then to reach the TPPC, and (3) finally to reach the TPTC. As a result, the CKC hypothesis was supported by the data of 58 countries, among which, there are still seven countries having not reached any of the three TPs, 23 countries have reached the first‐step TP (TPCI), 9 countries have reached the second‐step TP (TPPC), and 19 countries have reached the third‐step TP (TPTC).Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/156198/2/sd2008.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/156198/1/sd2008_am.pd

First Detailed Analysis of a Relatively Deep, Low Mass-ratio Contact Binary: ATO J108.6991+27.8306

We present the first detailed photometric analysis of ATO J108.6991+27.8306 (hereinafter as J108). The short-period close binary J108 was observed by the Nanshan 1 m Wide Field Telescope of the Xinjiang Astronomical Observatory. The obtained BVRI-band light curves were used to determine the photometric solution by using the 2003 version of the Wilson-Devinney code. J108 is a typical deep ( f > 50%), low mass ratio (q < 0.25) overcontact binary system with a mass ratio of q = 0.1501 and a fill-out factor of f = 50.1 %, suggesting that it is in the late evolutionary stage of contact binary systems. We found the target to be a W-type W UMa binary and provided evidence for the presence of starspots on both components. From the temperature-luminosity diagram, the main component is the evolved main sequence star with an evolutionary age of about 7.94 Gyr.Comment: 7 pages, 6 figure

Maximizing Network Lifetime of Wireless Sensor-Actuator Networks under Graph Routing

Process industries are adopting wireless sensor-actuator networks (WSANs) as the communication infrastructure. The dynamics of industrial environments and stringent reliability requirements necessitate high degrees of fault tolerance in routing. WirelessHART is an open industrial standard for WSANs that have seen world-wide deployments. WirelessHART employs graph routing schemes to achieve network reliability through multiple paths. Since many industrial devices operate on batteries in harsh environments where changing batteries are prohibitively labor-intensive, WSANs need to achieve long network lifetime. To meet industrial demand for long-term reliable communication, this paper studies the problem of maximizing network lifetime for WSANs under graph routing. We formulate the network lifetime maximization problem for WirelessHART networks under graph routing. Then, we propose the optimal algorithm and two more efficient algorithms to prolong the network lifetime of WSANs. Experiments in a physical testbed and simulations show our linear programming relaxation and greedy heuristics can improve the network lifetime by up to 50% while preserving the reliability benefits of graph routing