Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments

It is quite challenging to ensure the safety of reinforcement learning (RL) agents in an unknown and stochastic environment under hard constraints that require the system state not to reach certain specified unsafe regions. Many popular safe RL methods such as those based on the Constrained Markov Decision Process (CMDP) paradigm formulate safety violations in a cost function and try to constrain the expectation of cumulative cost under a threshold. However, it is often difficult to effectively capture and enforce hard reachability-based safety constraints indirectly with such constraints on safety violation costs. In this work, we leverage the notion of barrier function to explicitly encode the hard safety constraints, and given that the environment is unknown, relax them to our design of \emph{generative-model-based soft barrier functions}. Based on such soft barriers, we propose a safe RL approach that can jointly learn the environment and optimize the control policy, while effectively avoiding unsafe regions with safety probability optimization. Experiments on a set of examples demonstrate that our approach can effectively enforce hard safety constraints and significantly outperform CMDP-based baseline methods in system safe rate measured via simulations.Comment: 13 pages, 7 figure

Case report: Late in-stent thrombosis in a patient with vertebrobasilar dolichoectasia after stent-assisted coil embolization due to the discontinuation of antiplatelet therapy

Vertebrobasilar dolichoectasia (VBD) is a rare type of cerebrovascular disorder with high rates of morbidity and mortality. Due to the distinct pathological characteristics that fragmented internal elastic lamina and multiple dissections, VBD is difficult to treat and cured. Stent-assisted coil embolization is one of the main treatment modalities for such lesions. However, the duration of healing remained questionable, and there were no effective measures for evaluating endothelial coverage. Before complete endothelial coverage, the discontinuation of antiplatelet therapy may lead to fatal in-stent thrombosis; however, continued antiplatelet therapy could also result in bleeding complications. Thus, we present an autopsy case of late in-stent thrombosis due to the discontinuation of antiplatelet therapy and systematically review the literature to provide a reference for endovascular treatment and antiplatelet regimen of VBD

TLS2trees: A scalable tree segmentation pipeline for TLS data

1. Above-ground biomass (AGB) is an important metric used to quantify the mass of carbon stored in terrestrial ecosystems. For forests, this is routinely estimated at the plot scale (typically 1 ha) using inventory measurements and allometry. In recent years, terrestrial laser scanning (TLS) has appeared as a disruptive technology that can generate a more accurate assessment of tree and plot scale AGB; however, operationalising TLS methods has had to overcome a number of challenges. One such challenge is the segmentation of individual trees from plot level point clouds that are required to estimate woody volume, this is often done manually (e.g. with interactive point cloud editing software) and can be very time consuming. / 2. Here we present TLS2trees, an automated processing pipeline and set of Python command line tools that aims to redress this processing bottleneck. TLS2trees consists of existing and new methods and is specifically designed to be horizontally scalable. The processing pipeline is demonstrated on 7.5 ha of TLS data captured across 10 plots of seven forest types; from open savanna to dense tropical rainforest. / 3. A total of 10,557 trees are segmented with TLS2trees: these are compared to 1281 manually segmented trees. Results indicate that TLS2trees performs well, particularly for larger trees (i.e. the cohort of largest trees that comprise 50% of total plot volume), where plot-wise tree volume bias is ±0.4 m3 and %RMSE is 60%. Segmentation performance decreases for smaller trees, for example where DBH ≤10 cm; a number of reasons are suggested including performance of semantic segmentation step. / 4. The volume and scale of TLS data captured in forest plots is increasing. It is suggested that to fully utilise this data for activities such as monitoring, reporting and verification or as reference data for satellite missions an automated processing pipeline, such as TLS2trees, is required. To facilitate improvements to TLS2trees, as well as modification for other laser scanning modes (e.g. mobile and UAV laser scanning), TLS2trees is a free and open-source software

Model of Heat and Mass Exchange between a Downcast Shaft and the Air Flow to the Mine

Mining activities have increased owing to the rise in the social demand for minerals. Thermal hazards have become a major health and safety consideration in mines. The thermal environment of a working face is related to the air parameters at the bottom of shaft. The objective of this study is to accurately predict the air temperature at the bottom of a shaft in a mine with the ventilation time over 3 years. For this purpose, a mathematical model of the heat and mass exchange between the surrounding rock of the shaft and the downcast air is established by utilizing the finite volume method. The C++ languages are used for numerical calculations. The results are in agreement with the measured data. The effects of the relative humidity of air at the inlet of the shaft, the surface moisture coefficient of the shaft surface, and the physical parameters of the rock on the air parameters at the shaft bottom are studied in detail. Equations for calculating the enthalpy increase of air per 100 m in shaft with the depth of 1300 m were established by using cluster analysis. This equation provides a theoretical basis for predicting the air parameters along the shaft with the ventilation time over 3 years

An Improved Second-Order Blind Identification (SOBI) Signal De-Noising Method for Dynamic Deflection Measurements of Bridges Using Ground-Based Synthetic Aperture Radar (GBSAR)

Ground-based synthetic aperture radar (GBSAR) technology has been widely used for bridge dynamic deflection measurements due to its advantages of non-contact measurements, high frequency, and high accuracy. To reduce the influence of noise in dynamic deflection measurements of bridges using GBSAR—especially for noise of the instantaneous vibrations of the instrument itself caused by passing vehicles—an improved second-order blind identification (SOBI) signal de-noising method is proposed to obtain the de-noised time-series displacement of bridges. First, the obtained time-series displacements of three adjacent monitoring points in the same time domain are selected as observation signals, and the second-order correlations among the three time-series displacements are removed using a whitening process. Second, a mixing matrix is calculated using the joint approximation diagonalization technique for covariance matrices and to further obtain three separate signal components. Finally, the three separate signal components are converted in the frequency domain using the fast Fourier transform (FFT) algorithm, and the noise signal components are identified using a spectrum analysis. A new, independent, separated signal component matrix is generated using a zeroing process for the noise signal components. This process is inversely reconstructed using a mixing matrix to recover the original amplitude of the de-noised time-series displacement of the middle monitoring point among three adjacent monitoring points. The results of both simulated and on-site experiments show that the improved SOBI method has a powerful signal de-noising ability

Melittin-Based Nano-Delivery Systems for Cancer Therapy

Melittin (MEL) is a 26-amino acid polypeptide with a variety of pharmacological and toxicological effects, which include strong surface activity on cell lipid membranes, hemolytic activity, and potential anti-tumor properties. However, the clinical application of melittin is restricted due to its severe hemolytic activity. Different nanocarrier systems have been developed to achieve stable loading, side effects shielding, and tumor-targeted delivery, such as liposomes, cationic polymers, lipodisks, etc. In addition, MEL can be modified on nano drugs as a non-selective cytolytic peptide to enhance cellular uptake and endosomal/lysosomal escape. In this review, we discuss recent advances in MEL’s nano-delivery systems and MEL-modified nano drug carriers for cancer therapy

Deformation Activity Analysis of a Ground Fissure Based on Instantaneous Total Energy

This study proposes a novel instantaneous total energy method to perform an activity analysis of ground fissures deformation, which is calculated by integrating the extreme-point symmetric mode decomposition (ESMD) method and kinetic energy based on the time-series displacement acquired by shape acceleration array (SAA) sensors. The proposed method is tested on the Xiwang Road fissure in Beijing, China. First, to fully monitor the hanging wall and footwall of the monitored ground fissure, a 4 m-long SAA in the vertical direction and an 8 m-long SAA in the horizontal direction were embedded in a ground fissure to obtain an accurate time-series displacement with an accuracy of ±1.5 mm/32 m and a displacement acquisition frequency of once an hour. Second, to improve the accuracy of the activity analysis, the ESMD method and Spearman’s rho are applied to perform signal denoising of the original time-series displacement obtained by the SAA sensors. Finally, the instantaneous total energy is obtained to analyze the activity of the monitored ground fissure. The results demonstrate that the proposed method is more reliable to reflect the activity of a monitored ground fissure compared to the time-series displacement

Preparation of palladium nanoparticles and carboxylated MWNTs modified glassy carbon electrode and its application for the electrochemical determination of Cr (VI)

This study described a simple and rapid method for the electrochemical determination of Cr (VI) based on palladium nanoparticles and carboxylated multi-walled carbon nanotubes (MWNTs) modified glassy carbon electrode (GCE) (written as PdNPs-MWNT/GCE) .The prepared PdNPs-MWNT/GCE was characterized by FE-SEM,CV,EIS and DPV.It was found that many other metal ions such as Cr3+,Pd2+,Zn2+,Pb2+,Cu2+,Al3+,Ba2+,Ca2+,Mg2+ had no influence on Cr (VI) determination.Under the optimal conditions,the reduction current of Cr (VI) increased linearly with increasing the concentration of Cr (VI) in the range of 8×10-7~ 5×10-8mol/L with a correlation coefficient of 0.9960,and the detection limit of 3×10-8mol/L (based on S/N=3).The present method has the following advantages such as simple,rapid,good selectivity and sensitivity