CASOG: Conservative Actor-critic with SmOoth Gradient for Skill Learning in Robot-Assisted Intervention

Robot-assisted intervention has shown reduced radiation exposure to physicians and improved precision in clinical trials. However, existing vascular robotic systems follow master-slave control mode and entirely rely on manual commands. This paper proposes a novel offline reinforcement learning algorithm, Conservative Actor-critic with SmOoth Gradient (CASOG), to learn manipulation skills from human demonstrations on vascular robotic systems. The proposed algorithm conservatively estimates Q-function and smooths gradients of convolution layers to deal with distribution shift and overfitting issues. Furthermore, to focus on complex manipulations, transitions with larger temporal-difference error are sampled with higher probability. Comparative experiments in a pre-clinical environment demonstrate that CASOG can deliver guidewire to the target at a success rate of 94.00\% and mean backward steps of 14.07, performing closer to humans and better than prior offline reinforcement learning methods. These results indicate that the proposed algorithm is promising to improve the autonomy of vascular robotic systems.Comment: 13 pages, 5 figure, preprin

Recent Advances in Flame Retardant and Mechanical Properties of Polylactic Acid: A Review.

The large-scale application of ecofriendly polymeric materials has become a key focus of scientific research with the trend toward sustainable development. Mechanical properties and fire safety are two critical considerations of biopolymers for large-scale applications. Polylactic acid (PLA) is a flammable, melt-drop carrying, and strong but brittle polymer. Hence, it is essential to achieve both flame retardancy and mechanical enhancement to improve safety and broaden its application. This study reviews the recent research on the flame retardant functionalization and mechanical reinforcement of PLA. It classifies PLA according to the type of the flame retardant strategy employed, such as surface-modified fibers, modified nano/micro fillers, small-molecule and macromolecular flame retardants, flame retardants with fibers or polymers, and chain extension or crosslinking with other flame retardants. The functionalization strategies and main parameters of the modified PLA systems are summarized and analyzed. This study summarizes the latest advances in the fields of flame retardancy and mechanical reinforcement of PLA.pre-print3656 K

Loop-Mediated Isothermal Amplification Assay Targeting the MOMP Gene for Rapid Detection of Chlamydia psittaci Abortus Strain

For rapid detection of the Chlamydia psittaci abortus strain, a loop-mediated isothermal amplification (LAMP) assay was developed and evaluated in this study. The primers for the LAMP assay were designed on the basis of the main outer membrane protein (MOMP) gene sequence of C. psittaci. Analysis showed that the assay could detect the abortus strain of C. psittaci with adequate specificity. The sensitivity of the test was the same as that of the nested-conventional PCR and higher than that of chick embryo isolation. Testing of 153 samples indicated that the LAMP assay could detect the genome of the C. psittaci abortus strain effectively in clinical samples. This assay is a useful tool for rapid diagnosis of C. psittaci infection in sheep, swine and cattle

Memory effect and multifractality of cross-correlations in financial markets

An average instantaneous cross-correlation function is introduced to quantify the interaction of the financial market of a specific time. Based on the daily data of the American and Chinese stock markets, memory effect of the average instantaneous cross-correlations is investigated over different price return time intervals. Long-range time-correlations are revealed, and are found to persist up to a month-order magnitude of the price return time interval. Multifractal nature is investigated by a multifractal detrended fluctuation analysis.Comment: 11 pages, 4 figures

Erythromycin Enhances CD4+Foxp3+ Regulatory T-Cell Responses in a Rat Model of Smoke-Induced Lung Inflammation

Heavy smoking can induce airway inflammation and emphysema. Macrolides can modulate inflammation and effector T-cell response in the lungs. However, there is no information on whether erythromycin can modulate regulatory T-cell (Treg) response. This study is aimed at examining the impact of erythromycin on Treg response in the lungs in a rat model of smoking-induced emphysema. Male Wistar rats were exposed to normal air or cigarette smoking daily for 12 weeks and treated by gavage with 100 mg/kg of erythromycin or saline daily beginning at the forth week for nine weeks. The lung inflammation and the numbers of inflammatory infiltrates in bronchoalveolar lavage fluid (BALF) were characterized. The frequency, the number of Tregs, and the levels of Foxp3 expression in the lungs and IL-8, IL-35, and TNF-α in BALF were determined by flow cytometry, RT-PCR and ELISA, respectively. Treatment with erythromycin reduced smoking-induced inflammatory infiltrates, the levels of IL-8 and TNF-α in the BALF and lung damages but increased the numbers of CD4+Foxp3+ Tregs and the levels of Foxp3 transcription in the lungs, accompanied by increased levels of IL-35 in the BALF of rats. Our novel data indicated that erythromycin enhanced Treg responses, associated with the inhibition of smoking-induced inflammation in the lungs of rats

DOMAIN: MilDly COnservative Model-BAsed OfflINe Reinforcement Learning

Model-based reinforcement learning (RL), which learns environment model from offline dataset and generates more out-of-distribution model data, has become an effective approach to the problem of distribution shift in offline RL. Due to the gap between the learned and actual environment, conservatism should be incorporated into the algorithm to balance accurate offline data and imprecise model data. The conservatism of current algorithms mostly relies on model uncertainty estimation. However, uncertainty estimation is unreliable and leads to poor performance in certain scenarios, and the previous methods ignore differences between the model data, which brings great conservatism. Therefore, this paper proposes a milDly cOnservative Model-bAsed offlINe RL algorithm (DOMAIN) without estimating model uncertainty to address the above issues. DOMAIN introduces adaptive sampling distribution of model samples, which can adaptively adjust the model data penalty. In this paper, we theoretically demonstrate that the Q value learned by the DOMAIN outside the region is a lower bound of the true Q value, the DOMAIN is less conservative than previous model-based offline RL algorithms and has the guarantee of security policy improvement. The results of extensive experiments show that DOMAIN outperforms prior RL algorithms on the D4RL dataset benchmark, and achieves better performance than other RL algorithms on tasks that require generalization.Comment: 13 pages, 6 figure

Dynamics of Bid-ask Spread Return and Volatility of the Chinese Stock Market

Bid-ask spread is taken as an important measure of the financial market liquidity. In this article, we study the dynamics of the spread return and the spread volatility of four liquid stocks in the Chinese stock market, including the memory effect and the multifractal nature. By investigating the autocorrelation function and the Detrended Fluctuation Analysis (DFA), we find that the spread return is lack of long-range memory, while the spread volatility is long-range time correlated. Moreover, by applying the Multifractal Detrended Fluctuation Analysis (MF-DFA), the spread return is observed to possess a strong multifractality, which is similar to the dynamics of a variety of financial quantities. Differently from the spread return, the spread volatility exhibits a weak multifractal nature

CROP: Conservative Reward for Model-based Offline Policy Optimization

Offline reinforcement learning (RL) aims to optimize policy using collected data without online interactions. Model-based approaches are particularly appealing for addressing offline RL challenges due to their capability to mitigate the limitations of offline data through data generation using models. Prior research has demonstrated that introducing conservatism into the model or Q-function during policy optimization can effectively alleviate the prevalent distribution drift problem in offline RL. However, the investigation into the impacts of conservatism in reward estimation is still lacking. This paper proposes a novel model-based offline RL algorithm, Conservative Reward for model-based Offline Policy optimization (CROP), which conservatively estimates the reward in model training. To achieve a conservative reward estimation, CROP simultaneously minimizes the estimation error and the reward of random actions. Theoretical analysis shows that this conservative reward mechanism leads to a conservative policy evaluation and helps mitigate distribution drift. Experiments on D4RL benchmarks showcase that the performance of CROP is comparable to the state-of-the-art baselines. Notably, CROP establishes an innovative connection between offline and online RL, highlighting that offline RL problems can be tackled by adopting online RL techniques to the empirical Markov decision process trained with a conservative reward. The source code is available with https://github.com/G0K0URURI/CROP.git

Advantage of quantum coherence in postselected metrology

In conventional measurement, to reach the greatest accuracy of parameter estimation, all samples must be measured since each independent sample contains the same quantum Fisher information. In postselected metrology, postselection can concentrate the quantum Fisher information of the initial samples into a tiny post-selected sub-ensemble. It has been proven that this quantum advantage can not be realized in any classically commuting theory. In this work, we present that the advantage of postselection in weak value amplification (WVA) can not be achieved without quantum coherence. The quantum coherence of the initial system is closely related to the preparation costs and measurement costs in parameter estimation. With the increase of initial quantum coherence, the joint values of preparation costs and measurement costs can be optimized to smaller. Moreover, we derive an analytical tradeoff relation between the preparation, measurement and the quantum coherence. We further experimentally test the tradeoff relation in a linear optical setup. The experimental and theoretical results are in good agreement and show that the quantum coherence plays a key role in bounding the resource costs in the postselected metrology process

Association between decreased serum TBIL concentration and immediate memory impairment in schizophrenia patients

© 2019, The Author(s). Cognitive impairment is a core feature of schizophrenia (SCH). In addition to the toxic effect of Bilirubin (BIL), it has antioxidant properties that were associated with the psychopathology and cognitive impairment of psychiatric disorders. The aim of this study was to examine the correlation of serum total BIL (TBIL) concentration with cognitive impairment in SCH patients. We recruited 34 SCH patients and 119 healthy controls (HCs) in this case-control design. Cognition was assessed using the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). Serum TBIL concentration was measured using the immunoturbidimetric method. Serum TBIL concentration was significantly decreased in SCH patients compared to HCs after adjusting for age, gender, and education. Serum TBIL concentration in SCH patients was also positively correlated with the RBANS immediate memory score. Further stepwise multiple regression analysis confirmed the positive association between serum TBIL concentration and immediate memory score in SCH patients. Our findings supported that the decline in serum TBIL concentration was associated with the immediate memory impairment and psychopathology of SCH