Graph Reinforcement Learning Application to Co-operative Decision-Making in Mixed Autonomy Traffic: Framework, Survey, and Challenges

Proper functioning of connected and automated vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaboration decision-making for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) has been widely used in solving decision-making problems. However, the existing DRL-based methods have been mainly focused on solving the decision-making of a single CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot accurately represent the mutual effects of vehicles and model dynamic traffic environments. To address these shortcomings, this article proposes a graph reinforcement learning (GRL) approach for multi-agent decision-making of CAVs in mixed autonomy traffic. First, a generic and modular GRL framework is designed. Then, a systematic review of DRL and GRL methods is presented, focusing on the problems addressed in recent research. Moreover, a comparative study on different GRL methods is further proposed based on the designed framework to verify the effectiveness of GRL methods. Results show that the GRL methods can well optimize the performance of multi-agent decision-making for CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges and future research directions are summarized. This study can provide a valuable research reference for solving the multi-agent decision-making problems of CAVs in mixed autonomy traffic and can promote the implementation of GRL-based methods into intelligent transportation systems. The source code of our work can be found at https://github.com/Jacklinkk/Graph_CAVs.Comment: 22 pages, 7 figures, 10 tables. Currently under review at IEEE Transactions on Intelligent Transportation System

High-speed tensor tomography: iterative reconstruction tensor tomography (IRTT) algorithm

The recent advent of tensor tomography techniques has enabled tomographic investigations of the 3D nanostructure organization of biological and material science samples. These techniques extended the concept of conventional X-ray tomography by reconstructing not only a scalar value such as the attenuation coefficient per voxel, but also a set of parameters that capture the local anisotropy of nanostructures within every voxel of the sample. Tensor tomography data sets are intrinsically large as each pixel of a conventional X-ray projection is substituted by a scattering pattern, and projections have to be recorded at different sample angular orientations with several tilts of the rotation axis with respect to the X-ray propagation direction. Currently available reconstruction approaches for such large data sets are computationally expensive. Here, a novel, fast reconstruction algorithm, named iterative reconstruction tensor tomography (IRTT), is presented to simplify and accelerate tensor tomography reconstructions. IRTT is based on a second-rank tensor model to describe the anisotropy of the nanostructure in every voxel and on an iterative error backpropagation reconstruction algorithm to achieve high convergence speed. The feasibility and accuracy of IRTT are demonstrated by reconstructing the nanostructure anisotropy of three samples: a carbon fiber knot, a human bone trabecula specimen and a fixed mouse brain. Results and reconstruction speed were compared with those obtained by the small-angle scattering tensor tomography (SASTT) reconstruction method introduced by Liebi et al. [Nature (2015), 527, 349–352]. The principal orientation of the nanostructure within each voxel revealed a high level of agreement between the two methods. Yet, for identical data sets and computer hardware used, IRTT was shown to be more than an order of magnitude faster. IRTT was found to yield robust results, it does not require prior knowledge of the sample for initializing parameters, and can be used in cases where simple anisotropy metrics are sufficient, i.e. the tensor approximation adequately captures the level of anisotropy and the dominant orientation within a voxel. In addition, by greatly accelerating the reconstruction, IRTT is particularly suitable for handling large tomographic data sets of samples with internal structure or as a real-time analysis tool during the experiment for online feedback during data acquisition. Alternatively, the IRTT results might be used as an initial guess for models capturing a higher complexity of structural anisotropy such as spherical harmonics based SASTT in Liebi et al. (2015), improving both overall convergence speed and robustness of the reconstruction

Research on Teaching Reform of Artificial Intelligence Course Based on CDIO

In view of the problem of how to set up general undergraduate artificial intelligence courses, on the basis of carefully combing and summarizing years of teaching exploration and practice, it is proposed to set Artificial Intelligence (AI) courses in the lower grades of the university. Taking the teaching practice carried out by Liaoning Institute of Science and Engineering as an example, the “12365” principle is proposed based on the CDIO concept, and corresponding teaching reform and practice are carried out

A Wireless Transient Attenuated-exponential Overpressure Beamforming with for Far-field Blast Source Localization

Time-domain beamforming is more suitable for blast wave transient signal than frequency-domain beamformer because wide-band spectrum of noise makes the beamforming image less clear. To avoid the gust effects and enable the location of blast source accurately, this paper proposes a new one-dimensional Far-field delay-and-sum (DAS) beamforming method with an attenuate exponential function model for wireless overpressure transient signal. In addition, we also design wireless overpressure peak and root-mean-square (RMS) directional estimators to assess the performance of the proposed new DAS beamforming method. Furthermore, the effects of the wireless pressure sensor node (WPSL) spacing, the number of WPSLs and side lobe level brought from noise on the beam width are investigated in the two estimators. The proposed formula is verified by a uniformly spaced linear sensing array, and the results verify the feasibility of the proposed method in blast source localization. This paper is conducted to provide new insight into blast source localization algorithm, and further open a door for transient blast overpressure source localization scenarios in future

Mapping the 3D orientation of nanocrystals and nanostructures in human bone: Indications of novel structural features

Bone is built from collagen fibrils and biomineral nanoparticles. In humans, they are organized in lamellar twisting patterns on the microscale. It has been a central tenet that the biomineral nanoparticles are co-aligned with the bone nanostructure. Here, we reconstruct the three-dimensional orientation in human lamellar bone of both the nanoscale features and the biomineral crystal lattice from small-angle x-ray scattering and wide-angle x-ray scattering, respectively. While most of the investigated regions show well-aligned nanostructure and crystal structure, consistent with current bone models, we report a localized difference in orientation distribution between the nanostructure and the biomineral crystals in specific bands. Our results show a robust and systematic, but localized, variation in the alignment of the two signals, which can be interpreted as either an additional mineral fraction in bone, a preferentially aligned extrafibrillar fraction, or the result of transverse stacking of mineral particles over several fibrils

Exploring the inorganic and organic nitrate aerosol formation regimes at a suburban site on the North China Plain

Nitrate-driven aerosol pollution frequently occurs during winter over the North China Plain (NCP). Extensive studies have focused on inorganic nitrate formation, but few have focused on organic nitrates in China, preduding a thorough understanding of the nitrogen cycle and nitrate aerosol formation. Here, the inorganic (NO3,inorg) and organic nitrate (NO3,org) formation regimes under aerosol liquid water (ALW) and aerosol acidity (pH) influences were investigated during winter over the NCP based on data derived from an Aerodyne high-resolution time-of-flight aerosol mass spectrometer (HR-ToF-AMS). The campaign-averaged concentration of the total nitrate was 53 mu g m(3), with a 13% contribution from NO3,org, which exhibited a significantly decreased contribution with increasing haze episode evolution. The diurnal cycles of NO3,inorg and NO3,org were similar, with high concentrations during the nighttime at a high ALW level, revealing the important role of aqueous-phase processes. However, the correlations between the aerosol pH and NO3,inorg (R-2 = 0.13, P <0.01) and NO3,org (R-2 = 0.63, P <0.01) during polluted periods indicated a contrasting effect of aerosol pH on inorganic and organic nitrate formation. Our results provide a useful reference for smog chamber studies and promote a better understanding of organic nitrate formation via a nthropogenic emissions. (C) 2021 Elsevier B.V. All rights reserved.Peer reviewe

Two-year continuous measurements of carbonaceous aerosols in urban Beijing, China: temporal variations, characteristics and source analyses

Organic carbon (OC) and elemental carbon (EC) in the PM2.5 of urban Beijing were measured hourly with a semi-continuous thermal-optical analyzer from Jan 1, 2013 to Dec 31, 2014. The annual average OC and EC concentrations in Beijing were 17.0 ± 12.4 and 3.4 ± 2.0 μg/m3 for 2013, and 16.8 ± 14.5 and 3.5 ± 2.9 μg/m3 for 2014. It is obvious that the annual average concentrations of OC and EC in 2014 were not less than those in 2013 while the annual average PM2.5 concentration (89.4 μg/m3) in 2014 was slightly reduced as compared to that (96.9 μg/m3) in 2013. Strong seasonality of the OC and EC concentrations were found with high values during the heating seasons and low values during the non-heating seasons. The diurnal cycles of OC and EC characterized by higher values at night and in the morning were caused by primary emissions, secondary transformation and stable meteorological condition. Due to increasing photochemical activity, the OC peaks were observed at approximately noon. No clear weekend effects were observed. Interestingly, in the early mornings on weekends in the autumn and winter, the OC and EC concentrations were close to or higher than those on weekdays. Our data also indicate that high OC and EC concentrations were closely associated with their potential source areas which were determined based on the potential source contribution function analysis. High potential source areas were identified and were mainly located in the south of Beijing and the plain of northern China. A much denser source region was recorded in the winter than in the other seasons, indicating that local and regional transport over regional scales are the most important. These results demonstrate that both regional transport from the southern regions and local accumulation could lead to the enhancements of OC and EC and likely contribute to the severe haze pollution in Beijing

Highly time-resolved chemical characterization and implications of regional transport for submicron aerosols in the North China Plain

To investigate the regional transport and formation mechanisms of submicron aerosols in the North China Plan (NCP), for the first time, we conducted simultaneous combined observations of the non-refractory submicron aerosols (NR-PM1) chemical compositions using aerosol mass spectrometer at urban Beijing (BJ) and at regional background area of the NCP (XL), from November 2018 to January 2019. During the observation period, average mass concentrations of PM1 in BJ and XL were 26.6 +/- 31.7 and 16.0 +/- 18.7 mu g m(-3) respectively. The aerosol composition in XL showed a lower contribution of organic aerosol (33% vs. 43%) and higher fractions of nitrate (35% vs. 30%), ammonium (16% vs. 13%), and chlorine (2% vs. 1%) than in BJ. Additionally, a higher contribution of secondary organic aerosol (SOA) was also observed in XL, suggesting low primary emissions and highly oxidized OA in the background area. Nitrate displayed a significantly enhanced contribution with the aggravation of aerosol pollution in both BJ and XL, which was completely neutralized by excess ammonium at both sites, that the abundant ammonia emissions in the NCP favor nitrate formation on a regional scale. In addition, a higher proportion of nitrate in XL can be attributed to the more neutral and higher oxidation capacity of the background atmosphere. Heterogeneous aqueous reaction plays an important role in sulfate and SOA formation, and is more efficient in BJ which can be attributed to the higher aerosol surface areas at urban site. Regional transport from the southwestern regions of NCP showed a significant impact on the formation of haze episodes. Beside the invasion of transported pollutants, the abundant water vapor associated with the air mass to the downwind background area further enhanced local secondary transformation and expanded the regional scope of the haze pollution in the NCP. (C) 2019 Elsevier B.V. All rights reserved.Peer reviewe