Partial Advantage Estimator for Proximal Policy Optimization

This paper proposes an innovative approach to the Generalized Advantage Estimator (GAE) to address the bias-variance trade-off in truncated roll-outs during reinforcement learning. In typical GAE implementations, the k-step advantage is estimated using a lambda-weighted average, until the terminal state. While this method provides constant bias-variance properties at any time step, it often necessitates truncated roll-outs with shorter horizons for faster learning and policy updates within a single episode. This study highlights an unexplored issue: the bias-variance properties differ for small versus considerable time steps within truncated roll-outs. Specifically, smaller time steps may have a significant bias, prompting a need for their increase. The proposed solution involves a partial GAE update, calculating the advantage estimates for all time steps but updating the policy only for a specified range. To prevent data wastage, the data from this range is retained for further processing and policy parameter updates. This partial GAE approach, despite the increased memory requirements, promises enhanced computation speed and optimal data utilization. Empirical validation was conducted on four MuJoCo tasks and microRTS. The results show a performance improvement trend with the partial GAE estimator, outperforming regular GAE in task completion speed in microRTS. These findings offer a promising direction for improving policy update efficiency in reinforcement learning

Hadron Masses in Medium and Neutron Star Properties

We investigate the properties of the neutron star with relativistic mean field models. We incorporate in the quantum hadrodynamics and in the quark-meson coupling models a possible reduction of meson masses in nuclear matter. The equation of state for neutron star matter is obtained and is employed in Oppenheimer-Volkov equation to extract the maximum mass of the stable neutron star. We find that the equation of state, the composition and the properties of the neutron stars are sensitive to the values of the meson masses in medium.Comment: 18 pages, 5 figures and 2 tables. To be published in EPJ

Shock wave boundary layer interaction controlled by surface arc plasma actuators

An array of 16 surface arc plasma actuators (SAPAs) is employed to control the shock wave boundary layer interaction (SWBLI) at a 26° compression ramp in a Mach 2.0 flow. A new electrical circuit is used to actuate all 16 SAPAs. The electrical measurement reveals significant augmentation in peak current (200 A) and an energy deposition of 1.05 J, which are the nominal characteristics of the setup. The SAPA array is later applied for SWBLI control. The actuator array is placed upstream of the SWBLI and operates at four different frequencies, namely, 500 Hz, 1 kHz, 2 kHz, and 5 kHz. In the wind tunnel experiment, high-speed schlieren at 25 000 frames per second is used for flow visualization. The shock wave system is modified significantly by the controlling gas blobs (CGBs) or controlling gas bulbs (CGBUs) generated by SAPAs. The foot portion of the separation shock wave disappears, and the oblique shock wave bifurcates when the CGBs pass through the interaction region. The shock weakening effect is further verified through the rms of the schlieren intensity of the same phase

Background-free in-vivo imaging of Vitamin C using time-gateable responsive probe

Sensitive optical imaging of active biomolecules in the living organism requires both a molecular probe specifically responsive to the target and a high-contrast approach to remove the background interference from autofluorescence and light scatterings. Here, a responsive probe for ascorbic acid (vitamin C) has been developed by conjugating two nitroxide radicals with a long-lived luminescent europium complex. The nitroxide radical withholds the probe on its "off" state (barely luminescent), until the presence of vitamin C will switch on the probe by forming its hydroxylamine derivative. The probe showed a linear response to vitamin C concentration with a detection limit of 9.1 nM, two orders of magnitude lower than that achieved using electrochemical methods. Time-gated luminescence microscopy (TGLM) method has further enabled real-time, specific and background-free monitoring of cellular uptake or endogenous production of vitamin C, and mapping of vitamin C in living Daphnia magna. This work suggests a rational design of lanthanide complexes for background-free small animal imaging of biologically functional molecules

Phantom for Evaluating Accuracy of Image Registration Software

Provided is a phantom for evaluating the accuracy of image registration software based on a result of matching tomograms of a predetermined position of the phantom, taken using two or more imaging apparatuses. Accordingly, it is possible to more efficiently evaluate the accuracy of the image registration software by comparing the tomograms with one another using a three-dimensional analysis. In addition, it is possible to facilitate the comparison of the tomograms with one another by installing a plurality of indicating bars in the phantom so that their cross sections can appear on each of the tomograms

Boty-like retrotransposons in the filamentous fungus Botrytis cinerea contain the additional antisense gene brtn

AbstractLong-terminal repeat (LTR) retrotransposons typically contain gag, pol, or gag–pol, and in some case env genes. In this work, we used data mining of the Botrytis cinerea genomic sequence and a molecular approach to identify Boty-like LTR retrotransposons in B. cinerea containing an antisense gene (brtn) between pol and the 3′-LTR. Reverse transcriptase PCR (RT-PCR) revealed that some brtn-like genes could be expressed, at least in B. cinerea T4. We conducted BLAST comparisons and conserved-domain analysis, but the function of putative BRTN is presently unknown. Boty-like LTR retrotransposons in Sclerotinia sclerotiorum, called ScscLRET and containing brtn homologs at positions similar to brtn, were detected by homology searches and data mining of the S. sclerotiorum 1980 genomic sequence. Thus, this study demonstrated that some fungal LTR retrotransposons contain additional antisense genes

Raman scattering study of NaFe0.53_{0.53}Cu0.47_{0.47}As

We use polarization-resolved Raman scattering to study lattice dynamics in NaFe$_{0.53}$Cu$_{0.47}$As single crystals. We identify 4 $A_{1g}$ phonon modes at 125, 172, 183 and 197 cm$^{-1}$, and 4 $B_{3g}$ phonon modes at 101, 138, 173, 226 cm$^{-1}$. The phonon spectra are consistent with the $Ibam$ group, which confirms that the Cu and Fe atoms form a stripe order. The temperature dependence of the phonon spectra suggests weak electron-phonon and magneto-elastic interactions