A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

The optimization of traffic signal control (TSC) is critical for an efficient transportation system. In recent years, reinforcement learning (RL) techniques have emerged as a popular approach for TSC and show promising results for highly adaptive control. However, existing RL-based methods suffer from notably poor real-world applicability and hardly have any successful deployments. The reasons for such failures are mostly due to the reliance on over-idealized traffic simulators for policy optimization, as well as using unrealistic fine-grained state observations and reward signals that are not directly obtainable from real-world sensors. In this paper, we propose a fully Data-Driven and simulator-free framework for realistic Traffic Signal Control (D2TSC). Specifically, we combine well-established traffic flow theory with machine learning to construct a reward inference model to infer the reward signals from coarse-grained traffic data. With the inferred rewards, we further propose a sample-efficient offline RL method to enable direct signal control policy learning from historical offline datasets of real-world intersections. To evaluate our approach, we collect historical traffic data from a real-world intersection, and develop a highly customized simulation environment that strictly follows real data characteristics. We demonstrate through extensive experiments that our approach achieves superior performance over conventional and offline RL baselines, and also enjoys much better real-world applicability.Comment: 15 pages, 6 figure

Chronic PPARγ Stimulation Shifts Amyloidosis to Higher Fibrillarity but Improves Cognition.

We undertook longitudinal β-amyloid positron emission tomography (Aβ-PET) imaging as a translational tool for monitoring of chronic treatment with the peroxisome proliferator-activated receptor gamma (PPARγ) agonist pioglitazone in Aβ model mice. We thus tested the hypothesis this treatment would rescue from increases of the Aβ-PET signal while promoting spatial learning and preservation of synaptic density. Here, we investigated longitudinally for 5 months PS2APP mice (N = 23; baseline age: 8 months) and App NL-G-F mice (N = 37; baseline age: 5 months) using Aβ-PET. Groups of mice were treated with pioglitazone or vehicle during the follow-up interval. We tested spatial memory performance and confirmed terminal PET findings by immunohistochemical and biochemistry analyses. Surprisingly, Aβ-PET and immunohistochemistry revealed a shift toward higher fibrillary composition of Aβ-plaques during upon chronic pioglitazone treatment. Nonetheless, synaptic density and spatial learning were improved in transgenic mice with pioglitazone treatment, in association with the increased plaque fibrillarity. These translational data suggest that a shift toward higher plaque fibrillarity protects cognitive function and brain integrity. Increases in the Aβ-PET signal upon immunomodulatory treatments targeting Aβ aggregation can thus be protective

Enhancing protective microglial activities with a dual function TREM2 antibody to the stalk region

Triggering receptor expressed on myeloid cells 2 (TREM2) is essential for the transition of homeostatic microglia to a disease‐associated microglial state. To enhance TREM2 activity, we sought to selectively increase the full‐length protein on the cell surface via reducing its proteolytic shedding by A Disintegrin And Metalloproteinase (i.e., α‐secretase) 10/17. We screened a panel of monoclonal antibodies against TREM2, with the aim to selectively compete for α‐secretase‐mediated shedding. Monoclonal antibody 4D9, which has a stalk region epitope close to the cleavage site, demonstrated dual mechanisms of action by stabilizing TREM2 on the cell surface and reducing its shedding, and concomitantly activating phospho‐SYK signaling. 4D9 stimulated survival of macrophages and increased microglial uptake of myelin debris and amyloid β‐peptide in vitro. In vivo target engagement was demonstrated in cerebrospinal fluid, where nearly all oluble TREM2 was 4D9‐bound. Moreover, in a mouse model for Alzheimer's disease‐related pathology, 4D9 reduced amyloidogenesis, enhanced microglial TREM2 expression, and reduced a homeostatic marker, suggesting a protective function by driving microglia toward a disease‐associated state

Remediation of Anthracene-Contaminated Soil with Sophorolipids-SDBS-Na2SiO3 and Treatment of Eluting Wastewater

The soil pollution of polycyclic aromatic hydrocarbons (PAHs) is serious in China, which not only affects the living and growing environment of plants and animals but also has a great impact on people’s health. The use of hydrophobic organic compounds to make use of surfactant ectopic elution processing is more convenient and cheaper as a repair scheme and can effectively wash out the polycyclic aromatic hydrocarbons in the soil. Therefore, we mixed sophorolipids:sodium dodecylbenzene sulfonate (SDBS):Na2SiO3 according to the mass ratio of 1:15:150. We explored the influencing factors of high and low concentrations of PAH-contaminated soil using a single factor test and four factors at a two-level factorial design. Then, the elution wastewater was treated by ultrasonic oxidation technology and the alkali-activated sodium persulfate technology. The results showed that: (1) In the single factor test, when the elution time is 8 h, the concentration of the compounded surfactant is 1200 mg/L, the particle size is 60 mesh, the concentration of NaCl is 100 mmol/L, and the concentration of KCl is 50 mmol/L, and the effect of the PAH-contaminated soil eluted by the composite surfactant is the best. Externally added NaCl and KCl salt ions have a more obvious promotion effect on the polycyclic aromatic hydrocarbon-contaminated soil; (2) in the interaction experiment, single factor B (elution time) and D (NaCl concentration) have a significant main effect. There is also a certain interaction between factor A (concentration agent concentration) and factor D, factor B, and factor C (KCl concentration); (3) the treatment of anthracene in the eluate by ultrasonic completely mineralizes the organic pollutants by the thermal and chemical effects produced by the ultrasonic cavitation phenomenon, so that the organic pollutants in the eluate are oxidized and degraded into simple environmentally friendly small molecular substances. When the optimal ultrasonic time is 60 min and the ratio of oxidant to activator is 1:2, the removal rate of contaminants in the eluent can reach 63.7%. At the same time, the turbidity of the eluent is significantly lower than that of the liquid after centrifugal separation, indicating that oxidants can not only remove the pollutants in elution water but also remove the residual soil particulate matter; and (4) by comparing the infrared spectrum of the eluted waste liquid before and after oxidation, it can be seen that during the oxidation process, the inner part of eluent waste liquid underwent a ring-opening reaction, and the ring-opening reaction also occurred in the part of the cyclic ester group of the surfactant, which changed from a ring to non-ring

Study on Dynamic Failure Characteristics of Coal and Rock in Stope Induced by Dynamic Load of Steep Horizontal Sublevel Mining

In this paper, through a similar simulation experiment, the roof rock breaking situation of horizontal sublevel caving method in extremely thick and steep inclined coal seam is studied, and the response characteristics of coal and rock mass under different dynamic load strength are analyzed. The mechanical response mechanisms of different mining positions under the influence of dynamic load and the law characteristics of the surrounding rock from elastic deformation to impact failure are revealed. The study shows that with the increase in the horizontal sublevel mining depth, the roof is gradually broken from the suspended state, and the broken block fills the goaf. The thin rock strata form a granular structure, and the thick rock strata form a block splicing and occlusion structure. On the same side of the coal seam floor, the concentrated stress gradually increases, and the closer the distance from the goaf is, the more obvious the stress change in the coal body below is. With the increase in dynamic load energy, the dynamic instability and failure of the dynamic load side of the roadway occur, and the stress of the bottom coal in the intake roadway of the working face increases due to the influence of the lateral abutment pressure of the goaf, while the stress of the bottom coal in the middle of the working face and the return airway decreases due to the mining of the upper section, which reveals that the dynamic load-induced dynamic behavior requires the critical energy. Due to vibration waves, the dynamic load effect is short, and the reflective stretching is generated on the surface of the roadway, resulting in the failure of the roadway. The peak acceleration increases linearly with the increase in source energy, indicating that the stronger the dynamic load energy is, the higher the impact risk is. When the dynamic load intensity is constant, the peak acceleration decreases with the increase in propagation distance, indicating that the vibration wave has the dominant propagation direction, and there are certain directional differences in the effect of coal and rock. When the dynamic load is applied, the impact failure of the coal body has a critical displacement. When the displacement of the roadway surface is less than this critical displacement, the impact failure will not occur

Numerical Simulation to Determine the Largest Confining Stress in Longitudinal Tests of Cable Bolts

Bolt support is an economic method of roadway support. However, due to the influence of mining disturbance, the stress of roadway-surrounding rock changes, thus resulting in varying degrees of confining pressure in the radial direction of bolt. In this manuscript, a numerical solution was proposed to determine the largest confining stress in longitudinal tests of cable tendons. FLAC3D was selected to simulate the longitudinal process of cable tendons. The structural pile element was selected to simulate the cable tendon. The loading behavior of the cable was controlled by the cohesive and the frictional behavior of the cable/grout surface. To confirm the credibility of this numerical solution, the loading behavior of a normal cable and an improved cable was simulated. Experimental longitudinal tests were selected to validate the numerical results, showing that there was a satisfactory agreement between numerical and experimental results. The loading behavior of normal cables and improved cables was numerically simulated. Under the same test conditions, when the improved cable was used, the confining medium can generate much higher confining stress compared with normal cable tendons. Consequently, higher confining stress can result in a larger loading capacity of cable tendons

Numerical Simulation to Determine the Largest Confining Stress in Longitudinal Tests of Cable Bolts

Bolt support is an economic method of roadway support. However, due to the influence of mining disturbance, the stress of roadway-surrounding rock changes, thus resulting in varying degrees of confining pressure in the radial direction of bolt. In this manuscript, a numerical solution was proposed to determine the largest confining stress in longitudinal tests of cable tendons. FLAC3D was selected to simulate the longitudinal process of cable tendons. The structural pile element was selected to simulate the cable tendon. The loading behavior of the cable was controlled by the cohesive and the frictional behavior of the cable/grout surface. To confirm the credibility of this numerical solution, the loading behavior of a normal cable and an improved cable was simulated. Experimental longitudinal tests were selected to validate the numerical results, showing that there was a satisfactory agreement between numerical and experimental results. The loading behavior of normal cables and improved cables was numerically simulated. Under the same test conditions, when the improved cable was used, the confining medium can generate much higher confining stress compared with normal cable tendons. Consequently, higher confining stress can result in a larger loading capacity of cable tendons

Determination of Narrow Coal Pillar Width and Roadway Surrounding Rock Support Technology in Gob Driving Roadway

In order to determine the rational width of coal pillars and study the surrounding rock control technology of gob side entry driving with a narrow coal pillar, this paper first calculates the width of narrow coal pillar according to limit equilibrium theory; after that, the lateral support pressure and plastic zone development of the goaf is analyzed by numerical simulation to determine the rational width of reserved coal pillar; finally, through the ring breaking deformation regulation of surrounding rock of the gob side entry, the deformation and failure characteristics of the gob-side roadway during excavation and the influence of mining on the deformation and failure of the gob-side roadway are analyzed. The research results show that, combined with theoretical analysis and numerical simulation, the width of narrow coal pillar is decided to be 10 m; the deformation of the coal pillar side of the gob side roadway is much bigger than the roof subsidence, the deformation of the solid coal wall and the floor deformation; after the bolt support design of the gob side roadway, the deformation and damage of the gob side roadway during the driving period is small; during mining, the deformation of the narrow coal pillar wall is the key factor to determine the stability of the gob roadway; under the bolt support scheme, the overall deformation and failure of surrounding rock of the goaf roadway is small, and the control effect of the surrounding rock of the goaf roadway is good

Highly Oriented Low-Dimensional Tin Halide Perovskites with Enhanced Stability and Photovoltaic Performance

The low toxicity and a near-ideal choice of bandgap make tin perovskite an attractive alternative to lead perovskite in low cost solar cells. However, the development of Sn perovskite solar cells has been impeded by their extremely poor stability when exposed to oxygen. We report low-dimensional Sn perovskites that exhibit markedly enhanced air stability in comparison with their 3D counterparts. The reduced degradation under air exposure is attributed to the improved thermodynamic stability after dimensional reduction, the encapsulating organic ligands, and the compact perovskite film preventing oxygen ingress. We then explore these highly oriented low-dimensional Sn perovskite films in solar cells. The perpendicular growth of the perovskite domains between electrodes allows efficient charge carrier transport, leading to power conversion efficiencies of 5.94% without the requirement of further device structure engineering. We tracked the performance of unencapsulated devices over 100 h and found no appreciable decay in efficiency. These findings raise the prospects of pure Sn perovskites for solar cells application