Causal Deep Reinforcement Learning using Observational Data

Deep reinforcement learning (DRL) requires the collection of plenty of interventional data, which is sometimes expensive and even unethical in the real world, such as in the autonomous driving and the medical field. Offline reinforcement learning promises to alleviate this issue by exploiting the vast amount of observational data available in the real world. However, observational data may mislead the learning agent to undesirable outcomes if the behavior policy that generates the data depends on unobserved random variables (i.e., confounders). In this paper, we propose two deconfounding methods in DRL to address this problem. The methods first calculate the importance degree of different samples based on the causal inference technique, and then adjust the impact of different samples on the loss function by reweighting or resampling the offline dataset to ensure its unbiasedness. These deconfounding methods can be flexibly combined with the existing model-free DRL algorithms such as soft actor-critic and deep Q-learning, provided that a weak condition can be satisfied by the loss functions of these algorithms. We prove the effectiveness of our deconfounding methods and validate them experimentally

Agent-based Models in Flood Simulation - A Case Study for New Brunswick’s Flooding Events in 2018

This thesis introduces an Agent-Based Modeling (ABM) framework for flood simulation and inversion modeling in flood-prone areas, aiming to improve our understanding of the complex dynamics of flooding and provide valuable insights for flood management. The developed model incorporates multi-source terrain datasets, and integrates water flow and meteorological conditions from remote sensing data sources. It takes into account factors such as precipitation patterns, geographical features, and Digital Elevation Model (DEM) resolution to accurately represent flood characteristics. The model's parameter settings are derived from extensive experimentation, allowing for effective control and meaningful results. By considering the impact of precipitation and the presence of rivers, the model demonstrates its ability to simulate flood inundation with a reasonable level of accuracy. Overall, this comprehensive model provides a valuable tool for flood simulation and offers insights into flood dynamics for effective flood management and mitigation strategies

GANN: Graph Alignment Neural Network for Semi-Supervised Learning

Graph neural networks (GNNs) have been widely investigated in the field of semi-supervised graph machine learning. Most methods fail to exploit adequate graph information when labeled data is limited, leading to the problem of oversmoothing. To overcome this issue, we propose the Graph Alignment Neural Network (GANN), a simple and effective graph neural architecture. A unique learning algorithm with three alignment rules is proposed to thoroughly explore hidden information for insufficient labels. Firstly, to better investigate attribute specifics, we suggest the feature alignment rule to align the inner product of both the attribute and embedding matrices. Secondly, to properly utilize the higher-order neighbor information, we propose the cluster center alignment rule, which involves aligning the inner product of the cluster center matrix with the unit matrix. Finally, to get reliable prediction results with few labels, we establish the minimum entropy alignment rule by lining up the prediction probability matrix with its sharpened result. Extensive studies on graph benchmark datasets demonstrate that GANN can achieve considerable benefits in semi-supervised node classification and outperform state-of-the-art competitors

Route Identification Method for On-Ramp Traffic at Adjacent Intersections of Expressway Entrance

To determine the control strategy at intersections adjacent to the expressway on-ramp, a route identification method based on empirical mode decomposition (EMD) and dynamic time warping (DTW) is established. First, the de-noise function of EMD method is applied to eliminate disturbances and extract features and trends of traffic data. Then, DTW is used to measure the similarity of traffic volume time series between intersection approaches and expressway on-ramp. Next, a three-dimensional feature vector is built for every intersection approach traffic flow, including DTW distance, space distance between on-ramp and intersection approach, and intersection traffic volume. Fuzzy C-means clustering method is employed to cluster intersection approaches into classifications and identify critical routes carrying the most traffic to the on-ramp. The traffic data are collected by inductive loops at Xujiahui on-ramp of North and South Viaduct Expressway and surrounding intersections in Shanghai, China. The result shows that the proposed method can achieve route classification among intersections for different time periods in one day, and the clustering result is significantly influenced by three dimensions of traffic flow feature vector. As an illustrative example, micro-simulation models are built with different control strategies. The simulation shows that the coordinated control of critical routes identified by the proposed method has a better performance than coordinated control of arterial roads. Conclusions demonstrated that the proposed route identification method could provide a theoretical basis for the coordinated control of traffic signals among intersections and on-ramp. Document type: Articl

Multi-Effects Coupled Nanogenerators for Simultaneously Harvesting Solar, Thermal, and Mechanical Energies

As a result of the widespread use of small-scale and low-power electronic devices, the demand for micro-energy sources has increased, in particular the potential to harvest the wide variety of energy sources present in their surrounding environment. In this paper, a novel coupled nanogenerator that can realize energy harvesting for multiple energy sources is reported. Based on the unique electrical properties of ferroelectric Bi 0.5Na 0.5TiO 3 (BNT) materials, it is possible to combine a photovoltaic cell, pyroelectric nanogenerator, and triboelectric-piezoelectric nanogenerator in a single element to harvest light, heat, and mechanical energy simultaneously. To evaluate the effectiveness of coupling for different materials, a Yang coupling factor (k C,Q) is defined in terms of transferred charge, where BNT has the largest k C,Q of 1.29 during heating, indicating that BNT has the best coupling enhancement compared to common ferroelectric materials. This new criterion and novel device structure therefore provide a new basis for the future development of coupled nanogenerators which are capable of harvesting multiple sources of energy.</p

Social support and self-efficacy multiply mediate the relationship between medical coping style and resilience in patients with type A aortic dissection

BackgroundPrevious research has shown that medical coping modes are associated with resilience in cardiovascular disease patients. However, postoperatively, the mechanism underlying this association in Stanford type A aortic dissection patients is poorly understood.ObjectiveThis study investigated the mediating effects of social support and self-efficacy on the relationship between medical coping modes and resilience in Stanford type A aortic dissection patients postoperatively.MethodsWe assessed 125 patients after surgery for Stanford type A aortic dissection using the Medical Coping Modes Questionnaire, the General Self-Efficacy Scale, the Social Support Rating Scale, and the Connor–Davidson Resilience Scale. Structural equation modeling with AMOS (v.24) was used to test the hypothesized model with multiple mediators. Both direct and mediational effects (through social support and self-efficacy) of medical coping modes on resilience outcomes were examined.ResultsThe mean Connor–Davidson Resilience Scale score was 63.78 ± 12.29. Confrontation, social support, and self-efficacy correlated with resilience (r = 0.40, 0.23, 0.72, respectively; all p &lt; 0.01). In multiple mediation models, social support independently (effect = 0.11; 95% confidence interval [CI], 0.04–0.27) and social support and self-efficacy serially (effect = 0.06; 95% CI, 0.02–0.14) mediated the association of confrontation with resilience maintenance, accounting for 57.89 and 10.53% of the total effect, respectively.ConclusionSocial support and self-efficacy were multiple mediators of the relationship between confrontation and resilience. Interventions designed to facilitate confrontation and subsequently increase social support and self-efficacy may be useful to increase resilience in Stanford type A aortic dissection patients