Optimized state estimation for nonlinear dynamical networks subject to fading measurements and stochastic coupling strength: An event-triggered communication mechanism

summary:This paper is concerned with the design of event-based state estimation algorithm for nonlinear complex networks with fading measurements and stochastic coupling strength. The event-based communication protocol is employed to save energy and enhance the network transmission efficiency, where the changeable event-triggered threshold is adopted to adjust the data transmission frequency. The phenomenon of fading measurements is described by a series of random variables obeying certain probability distribution. The aim of the paper is to propose a new recursive event-based state estimation strategy such that, for the admissible linearization error, fading measurements and stochastic coupling strength, a minimum upper bound of estimation error covariance is given by designing the estimator gain. Furthermore, the monotonicity relationship between the trace of the upper bound of estimation error covariance and the fading probability is pointed out from the theoretical aspect. Finally, a simulation example is used to show the effectiveness of developed state estimation algorithm

Efficient Deep Reinforcement Learning via Adaptive Policy Transfer

Transfer Learning (TL) has shown great potential to accelerate Reinforcement Learning (RL) by leveraging prior knowledge from past learned policies of relevant tasks. Existing transfer approaches either explicitly computes the similarity between tasks or select appropriate source policies to provide guided explorations for the target task. However, how to directly optimize the target policy by alternatively utilizing knowledge from appropriate source policies without explicitly measuring the similarity is currently missing. In this paper, we propose a novel Policy Transfer Framework (PTF) to accelerate RL by taking advantage of this idea. Our framework learns when and which source policy is the best to reuse for the target policy and when to terminate it by modeling multi-policy transfer as the option learning problem. PTF can be easily combined with existing deep RL approaches. Experimental results show it significantly accelerates the learning process and surpasses state-of-the-art policy transfer methods in terms of learning efficiency and final performance in both discrete and continuous action spaces.Comment: Accepted by IJCAI'202

Multi-Agent Game Abstraction via Graph Attention Neural Network

In large-scale multi-agent systems, the large number of agents and complex game relationship cause great difficulty for policy learning. Therefore, simplifying the learning process is an important research issue. In many multi-agent systems, the interactions between agents often happen locally, which means that agents neither need to coordinate with all other agents nor need to coordinate with others all the time. Traditional methods attempt to use pre-defined rules to capture the interaction relationship between agents. However, the methods cannot be directly used in a large-scale environment due to the difficulty of transforming the complex interactions between agents into rules. In this paper, we model the relationship between agents by a complete graph and propose a novel game abstraction mechanism based on two-stage attention network (G2ANet), which can indicate whether there is an interaction between two agents and the importance of the interaction. We integrate this detection mechanism into graph neural network-based multi-agent reinforcement learning for conducting game abstraction and propose two novel learning algorithms GA-Comm and GA-AC. We conduct experiments in Traffic Junction and Predator-Prey. The results indicate that the proposed methods can simplify the learning process and meanwhile get better asymptotic performance compared with state-of-the-art algorithms.Comment: Accepted by AAAI202

Upgrading the quality of recycled aggregates from construction and demolitionwaste by using a novel brick separation and surface treatment method

Mixed recycled aggregates (MRA) from construction and demolition waste (CDW) with high-purity and environmental performance are required for highway construction application in base layer and precast concrete curbs. The main problematic constituents that reduce the quality level of the recycled aggregates applications are brick components, flaky particles, and attached mortar, which make up a large proportion of CDW in some countries. This paper studies the potential of brick separation technology based on shape characteristics in order to increase the recycled concrete aggregates (RCA) purity for MRA quality improvement. MRA after purification was also processed with surface treatment experiment by rotating in a cylinder to improve the shape characteristics and to remove the attached mortar. The purity, strength property, densities, water absorption ratio, shape index, and mortar removal ratio of MRA were studied before and after the use of the brick separation and surface treatment proposed in this study. Finally, the recycled aggregates upgradation solution was adopted in a stationary recycling plant designed for a length of 113 km highway construction. The properties of CDW mixed concrete for precast curbs manufacturing were conducted. The results indicate that problematic fractions (brick components, particle shape, and surface weakness) in the MRA were significantly reduced by using brick separation and surface treatment solution. Above all, it is very important that the proposed brick separation method was verified to be practically adopted in CDW recycling plant for highway base layer construction and concrete curbs manufacturing at a low cost