157 research outputs found

    Tackling Non-Stationarity in Reinforcement Learning via Causal-Origin Representation

    Full text link
    In real-world scenarios, the application of reinforcement learning is significantly challenged by complex non-stationarity. Most existing methods attempt to model changes in the environment explicitly, often requiring impractical prior knowledge. In this paper, we propose a new perspective, positing that non-stationarity can propagate and accumulate through complex causal relationships during state transitions, thereby compounding its sophistication and affecting policy learning. We believe that this challenge can be more effectively addressed by tracing the causal origin of non-stationarity. To this end, we introduce the Causal-Origin REPresentation (COREP) algorithm. COREP primarily employs a guided updating mechanism to learn a stable graph representation for states termed as causal-origin representation. By leveraging this representation, the learned policy exhibits impressive resilience to non-stationarity. We supplement our approach with a theoretical analysis grounded in the causal interpretation for non-stationary reinforcement learning, advocating for the validity of the causal-origin representation. Experimental results further demonstrate the superior performance of COREP over existing methods in tackling non-stationarity

    An Advanced Quantum-Resistant Signature Scheme for Cloud Based on Eisenstein Ring

    Get PDF
    The authors wish to express their appreciation to the reviewers for their helpful suggestions which greatly improved the presentation of this paper. This work was supported by the Major Program of National Natural Science Foundation of China (11290141).Peer reviewe

    A Virtual Motion Camouflage Approach for Cooperative Trajectory Planning of Multiple UCAVs

    Get PDF
    This paper investigates cooperative trajectory planning of multiple unmanned combat aerial vehicles (multi-UCAV) in performing autonomous cooperative air-to-ground target attack missions. By integrating an approximate allowable attack region model, several constraint models, and a multicriterion objective function, the problem is formulated as a cooperative trajectory optimal control problem (CTOCP). Then, a virtual motion camouflage (VMC) for cooperative trajectory planning of multi-UCAV, combining with the differential flatness theory, Gauss pseudospectral method (GPM), and nonlinear programming, is designed to solve the CTOCP. In particular, the notion of the virtual time is introduced to the VMC problem formulation to handle the temporal cooperative constraints. The simulation experiments validate that the CTOCP can be effectively solved by the cooperative trajectory planning algorithm based on VMC which integrates the spatial and temporal constraints on the trajectory level, and the comparative experiments illustrate that VMC based algorithm is more efficient than GPM based direct collocation method in tackling the CTOCP

    miR-499-5p Attenuates Mitochondrial Fission and Cell Apoptosis via p21 in Doxorubicin Cardiotoxicity

    Get PDF
    Doxorubicin (DOX) is a broad-spectrum anti-tumor drug, but its cardiotoxicity limits its clinical application. A better understanding of the molecular mechanisms underlying DOX cardiotoxicity will benefit clinical practice and remedy heart failure. Our present study observed that DOX caused cardiomyocyte (H9c2) apoptosis via the induction of abnormal mitochondrial fission. Notably, the expression levels of p21 increased in DOX-treated cardiomyocytes, and the silencing of p21 using siRNA greatly attenuated mitochondrial fission and apoptosis in cardiomyocytes. We also found that miR-499-5p could directly target p21 and attenuated DOX-induced mitochondrial fission and apoptosis. The role of the miR-499-5p-p21 axis in the prevention of DOX cardiotoxicity was also validated in the mice model. DOX treatment induced an upregulation of p21, which induced subsequent abnormal mitochondrial fission and myocardial apoptosis in mouse heart. Adenovirus-harboring miR-499-5p-overexpressing mice exhibited significantly reduced p21 expression, mitochondrial fission and myocardial apoptosis in hearts following DOX administration. The miR-499-5p-overexpressing mice also exhibited improved cardiomyocyte hypertrophy and cardiac function after DOX treatment. However, miR-499-5p was not involved in the DOX-induced apoptosis of cancer cells. Taken together, these findings reveal an emerging role of p21 in the regulation of mitochondrial fission program. miR-499-5p attenuated mitochondrial fission and DOX cardiotoxicity via the targeting of p21. These results provide new evidence for the miR-499-5p-p21 axis in the attenuation of DOX cardiotoxicity. The development of new therapeutic strategies based on the miR-499-5p-p21 axis is a promising path to overcome DOX cardiotoxicity as a chemotherapy for cancer treatment

    Patterns and mechanisms of coseismic and postseismic slips of the 2011 M W 7.1 Van (Turkey) earthquake revealed by multi-platform synthetic aperture radar interferometry

    Full text link
    On 23rd October 2011, a MW 7.1 reverse slip earthquake occurred in the Bardakçı-Saray thrust fault zone in the Van region, Eastern Turkey. Earlier geodetic studies have found different slip distributions in terms of both magnitude and pattern. In this paper, we present several COSMO-SkyMED (CSK), Envisat ASAR and RADARSAT-2 interferograms spanning different time intervals, showing that significant postseismic signals can be observed in the first three days after the mainshock. Using observations that combine coseismic and postseismic signals is shown to significantly underestimate coseismic slip. We hence employed the CSK pair with the minimum postseismic signals to generate one conventional interferogram and one along-track interferogram for further coseismic modelling. Our best-fit coseismic slip model suggests that: (1) this event is associated with a buried NNW dipping fault with a preferable dip angle of 49° and a maximum slip of 6.5 m at a depth of 12 km; and (2) two unequal asperities can be observed, consistent with previous seismic solutions. Significant oblique aseismic slip with predominant left-lateral slip components above the coseismic rupture zone within the first 3 days after the mainshock is also revealed by a postseismic CSK interferogram, indicating that the greatest principal stress axis might have rotated due to a significant stress drop during the coseismic rupture

    A New Fusion Fault Diagnosis Method for Fiber Optic Gyroscopes

    Full text link
    The fiber optic gyroscope (FOG) is a high precision inertial navigation device, and it is necessary to ensure its reliability for effective use. However, the extracted fault features are easily distorted due to the interference of vibrations when the FOG is in operation. In order to minimize the influence of vibrations to the greatest extent, a fusion diagnosis method was proposed in this paper. It extracted features from fault data with Fast Fourier Transform (FFT) and wavelet packet decomposition (WPD), and built a strong diagnostic classifier with a sparse auto encoder (SAE) and a neural network (NN). Then, a fusion neural network model was established based on the diagnostic output probabilities of the two primary classifiers, which improved the diagnostic accuracy and the anti-vibration capability. Then, five fault types of the FOG under random vibration conditions were established. Fault data sets were collected and generated for experimental comparison with other methods. The results showed that the proposed fusion fault diagnosis method could perform effective and robust fault diagnosis for the FOG under vibration conditions with a high diagnostic accuracy

    A New Fusion Fault Diagnosis Method for Fiber Optic Gyroscopes

    Full text link
    The fiber optic gyroscope (FOG) is a high precision inertial navigation device, and it is necessary to ensure its reliability for effective use. However, the extracted fault features are easily distorted due to the interference of vibrations when the FOG is in operation. In order to minimize the influence of vibrations to the greatest extent, a fusion diagnosis method was proposed in this paper. It extracted features from fault data with Fast Fourier Transform (FFT) and wavelet packet decomposition (WPD), and built a strong diagnostic classifier with a sparse auto encoder (SAE) and a neural network (NN). Then, a fusion neural network model was established based on the diagnostic output probabilities of the two primary classifiers, which improved the diagnostic accuracy and the anti-vibration capability. Then, five fault types of the FOG under random vibration conditions were established. Fault data sets were collected and generated for experimental comparison with other methods. The results showed that the proposed fusion fault diagnosis method could perform effective and robust fault diagnosis for the FOG under vibration conditions with a high diagnostic accuracy
    corecore