Actor-Critic Reinforcement Learning for Control with Stability Guarantee

Reinforcement Learning (RL) and its integration with deep learning have achieved impressive performance in various robotic control tasks, ranging from motion planning and navigation to end-to-end visual manipulation. However, stability is not guaranteed in model-free RL by solely using data. From a control-theoretic perspective, stability is the most important property for any control system, since it is closely related to safety, robustness, and reliability of robotic systems. In this paper, we propose an actor-critic RL framework for control which can guarantee closed-loop stability by employing the classic Lyapunov's method in control theory. First of all, a data-based stability theorem is proposed for stochastic nonlinear systems modeled by Markov decision process. Then we show that the stability condition could be exploited as the critic in the actor-critic RL to learn a controller/policy. At last, the effectiveness of our approach is evaluated on several well-known 3-dimensional robot control tasks and a synthetic biology gene network tracking task in three different popular physics simulation platforms. As an empirical evaluation on the advantage of stability, we show that the learned policies can enable the systems to recover to the equilibrium or way-points when interfered by uncertainties such as system parametric variations and external disturbances to a certain extent.Comment: IEEE RA-L + IROS 202

Experimental Study of Tendon Failure Analysis for a TLP Floating Offshore Wind Turbine

This paper describes an experimental study conducted on a multi-column tension leg platform (TLP) floating offshore wind turbine (FOWT). A prototype model of the TLP FOWT supporting the NREL 5-MW wind turbine with a scale ratio of 1:50 is tested under various wind and wave conditions at the State Key Laboratory of Coastal and Offshore Engineering at Dalian University of Technology, China. This work has particularly focused on the tendon failure and its impact on the dynamic response of the FOWT. Free decay tests, regular wave tests, wind-wave combined tests and tendon failure tests are conducted using different environmental parameters.The results suggest that natural periods, dynamic responses of the platform, and forces in the tendons satisfy the design requirements. The analysis indicates that the impact of tendon failure on the platform surge, heave and pitch responses are found to be insignificant. When one of the tendons is broken, the adjacent tendons experience a significant increase in tensile force; and, the maximum tensile force in the remaining tendon is found to increase by about 130%. The overstepping of the minimum breaking load as recommended by the design standard DNV GL is not reached, and this indicates that the safety of the system is ensured even during the harshest failure condition

Dual Effects of the US-China Trade War and COVID-19 on United States Imports: Transfer of China's industrial chain?

The trade tension between the U.S. and China since 2018 has caused a steady decoupling of the world's two largest economies. The pandemic outbreak in 2020 complicated this process and had numerous unanticipated repercussions. This paper investigates how U.S. importers reacted to the trade war and worldwide lockdowns due to the COVID-19 pandemic. We examine the effects of the two incidents on U.S. imports separately and collectively, with various economic scopes. Our findings uncover intricate trading dynamics among the U.S., China, and Southeast Asia, through which businesses relocated portions of their global supply chain away from China to avoid high tariffs. Our analysis indicates that increased tariffs cause the U.S. to import less from China. Meanwhile, Southeast Asian exporters have integrated more into value chains centered on Chinese suppliers by participating more in assembling and completing products. However, the worldwide lockdowns over pandemic have reversed this trend as, over this period, the U.S. effectively imported more goods directly from China and indirectly through Southeast Asian exporters that imported from China.Comment: 30 pages, 6 figure

Purification and characterization of a novel defensin from the salivary glands of the black fly, Simulium bannaense

BACKGROUND: Black flies (Diptera: Simuliidae) are haematophagous insects that can cause allergic reactions and act as vectors of pathogens. Although their saliva has been thought to contain a diverse array of physiologically active molecules, little information is available on antimicrobial factors in black fly salivary glands, especially no defensins have been reported so far. METHODS: A novel cationic defensin designated SibaDef was purified using reverse phase high-performance liquid chromatography (RP-HPLC) from the salivary glands of the black fly Simulium bannaense. The amino acid sequence of SibaDef was determined by a combination method of automated Edman degradation and cDNA sequencing. The morphologic changes of Gram-positive bacteria Staphylococcus aureus or Bacillus subtilis treated with SibaDef were assessed by scanning electron microscopy (SEM). Quantitative PCR (qPCR) was performed to analyze the expression of SibaDef mRNA in whole bodies of insects after oral infection with the bacteria S. aureus or B. subtilis. RESULTS: Surprisingly, the phylogenetic analysis of defensin-related amino acid sequences demonstrated that SibaDef is most closely related to defensins from the human body louse Pediculus humanus corporis (Anoplura: Pediculidae), rather than to other dipteran defensins. SibaDef showed potent antimicrobial activities against Gram-positive bacteria with minimal inhibitory concentrations (MICs) ranging from 0.83 μM to 2.29 μM. SEM analysis indicated that SibaDef killed microorganisms through the disruption of cell membrane integrity. The transcript levels of SibaDef in the bacteria-immunized flies increased with the time course, reaching maximum at 36 h and then slowly decreased from that time point. CONCLUSIONS: Our results indicate that SibaDef is involved in the innate humoral response of the black fly S. bannaense, and it might play a significant role in the defence against microorganisms in both sugar and blood meals

Experimental Research on the Dynamic Characteristics of Offshore Wind Turbine Model Under the Action of Highly Nonlinear Waves

[Introduction] Monopile foundation is currently the most widely used support structure for offshore wind farms. For offshore wind farms in the medium water depth area such as Dafeng, Binhai and Sheyang in China, the design waves are very close to the wave breaking limit. Compared with the traveling wave, the focused waves can form an impact force on the pile column in a short time, which is larger than the conventional wave force, and undermines the operational performance and fatigue life of the offshore wind turbines. [Method] In this paper, NREL 5 MW monopile wind turbine model tests were carried out to investigate the hydrodynamic response of large offshore wind turbines to highly nonlinear waves and other harsh waves in combination with the marine environmental conditions of China's eastern coastal wind farms, focusing on exploring the test techniques for monopile wind turbines, and the mechanism of interaction between focused waves and monopiles under the action of focused waves. [Result] The results show that, the horizontal wave force on the pile foundation, which is significantly transient, will suddenly increase when the focused waves act on the monopile foundation, and receive the reverse impact force at the trough of focused wave fluctuation. [Conclusion] The results are of high theoretical value and have engineering prospect for the safe operation of monopile foundation

Genome-wide analysis and characterization of Dendrocalamus farinosus SUT gene family reveal DfSUT4 involvement in sucrose transportation in plants

Sucrose is the main transported form of photosynthetic products. Sucrose transporter (SUT) participates in the translocation of sucrose from source to sink, which is important for the growth and development of plants. Dendrocalamus farinosus is an important economic crop in southwestern China because of its high growth rate, high fiber content, and dual usage for food and timber, but the mechanism of sucrose transportation in D. farinosus is unclear. In this study, a total of 12 SUT transporter genes were determined in D. farinosus by whole-genome identification. DfSUT2, DfSUT7, and DfSUT11 were homologs of rice OsSUT2, while DfSUT4 was a homolog of OsSUT4, and these four DfSUT genes were expressed in the leaf, internode, node, and bamboo shoots of D. farinosus. In addition, DfSUT family genes were involved in photosynthetic product distribution, ABA/MeJA responses, and drought resistance, especially DfSUT4. The function of DfSUT4 was then verified in Nicotiana tabacum. DfSUT4 was localized mainly in the leaf mesophyll and stem phloem of pDfSUT4::GUS transgenic plant. The overexpression of DfSUT4 gene in transgenic plant showed increases of photosynthetic rate, above-ground biomass, thousand grain weight, and cellulose content. Our findings altogether indicate that DfSUT4 can be a candidate gene that can be involved in phloem sucrose transportation from the source leaves to the sink organs, phytohormone responses, abiotic stress, and fiber formation in plants, which is very important in the genetic improvement of D. farinosus and other crops

A novel method for objectively, rapidly and accurately evaluating burn depth via near infrared spectroscopy

The accurate and objective evaluation of burn depth is a significant challenge in burn wound care. Herein, we used near infrared spectroscopy (NIRS) technology to measure the different depth of thermal burns in ex vivo porcine models. Based on the intensity of the spectral signals and the diffuse reflection theory, we extracted the optical parameters involved in functional (total hemoglobin and water content) and structural (tissue scattered size and scattered particles) features that reflect the changes in burn depth. Next, we applied support vector regression to construct a model including the optical property parameters and the burn depth. Finally, we histologically verified the burn depth data collected via NIRS. The results showed that our inversion model could achieve an average relative error of about 7.63%, while the NIRS technology diagnostic accuracy was in the range of 50 μm. For the first time, this novel technique provides physicians with real-time burn depth information objectively and accurately