Communication-Efficient Decentralized Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control

Connected and autonomous vehicles (CAVs) promise next-gen transportation systems with enhanced safety, energy efficiency, and sustainability. One typical control strategy for CAVs is the so-called cooperative adaptive cruise control (CACC) where vehicles drive in platoons and cooperate to achieve safe and efficient transportation. In this study, we formulate CACC as a multi-agent reinforcement learning (MARL) problem. Diverging from existing MARL methods that use centralized training and decentralized execution which require not only a centralized communication mechanism but also dense inter-agent communication, we propose a fully-decentralized MARL framework for enhanced efficiency and scalability. In addition, a quantization-based communication scheme is proposed to reduce the communication overhead without significantly degrading the control performance. This is achieved by employing randomized rounding numbers to quantize each piece of communicated information and only communicating non-zero components after quantization. Extensive experimentation in two distinct CACC settings reveals that the proposed MARL framework consistently achieves superior performance over several contemporary benchmarks in terms of both communication efficiency and control efficacy.Comment: 11 pages, 7 figure

Explicit gain equations for hybrid graphene-quantum-dot photodetectors

Graphene is an attractive material for broadband photodetection but suffers from weak light absorption. Coating graphene with quantum dots can significantly enhance light absorption and create extraordinarily high photo gain. This high gain is often explained by the classical gain theory which is unfortunately an implicit function and may even be questionable. In this work, we managed to derive explicit gain equations for hybrid graphene-quantum-dot photodetectors. Due to the work function mismatch, lead sulfide (PbS) quantum dots coated on graphene will form a surface depletion region near the interface of quantum dots and graphene. Light illumination narrows down the surface depletion region, creating a photovoltage that gates the graphene. As a result, high photo gain in graphene is observed. The explicit gain equations are derived from the theoretical gate transfer characteristics of graphene and the correlation of the photovoltage with the light illumination intensity. The derived explicit gain equations fit well with the experimental data, from which physical parameters are extracted.Comment: 14 pages, 6 figure

Reconstruction of the event vertex in the PandaX-III experiment with convolution neural network

The tracks left by charged particles in a gaseous time projection chamber~(TPC) incorporate important information about the interaction process and drift of electrons in gas. The electron diffusion information carried by the tracks is an effective signature to reconstruct $z_0$, the vertex position in drift direction at which the event takes place. In this paper, we propose to reconstruct $z_0$ with convolution neural network~(CNN) in the PandaX-III experiment. A CNN model VGGZ0net is built and validated with Monte Carlo simulation data. It gives $z_0$ with a 11~cm precision for the events above 2~MeV uniformly distributed along a drift distance of 120~cm, and then the electron lifetime can be deduced. The energy resolution of detector is significantly improved after the electron lifetime correction, i.e., from 10.1\% to 4.0\% FWHM at the Q-value of double beta decay of $^{136}$Xe for the scenario with an electron lifetime of 6.5~ms. The CNN model is also successfully applied to the experimental data of the PandaX-III prototype detector for $z_0$ reconstruction

Signal identification with Kalman Filter towards background-free neutrinoless double beta decay searches in gaseous detectors

Particle tracks and differential energy loss measured in high pressure gaseous detectors can be exploited for event identification in neutrinoless double beta decay~($0\nu \beta \beta$) searches. We develop a new method based on Kalman Filter in a Bayesian formalism (KFB) to reconstruct meandering tracks of MeV-scale electrons. With simulation data, we compare the signal and background discrimination power of the KFB method assuming different detector granularities and energy resolutions. Typical background from $^{232}$Th and $^{238}$U decay chains can be suppressed by another order of magnitude than that in published literatures, approaching the background-free regime. For the proposed PandaX-III experiment, the $0\nu \beta \beta$ search half-life sensitivity at the 90\% confidence level would reach $2.7 \times 10^{26}$~yr with 5-year live time, a factor of 2.7 improvement over the initial design target