Self-Supervised Curriculum Generation for Autonomous Reinforcement Learning without Task-Specific Knowledge

A significant bottleneck in applying current reinforcement learning algorithms to real-world scenarios is the need to reset the environment between every episode. This reset process demands substantial human intervention, making it difficult for the agent to learn continuously and autonomously. Several recent works have introduced autonomous reinforcement learning (ARL) algorithms that generate curricula for jointly training reset and forward policies. While their curricula can reduce the number of required manual resets by taking into account the agent's learning progress, they rely on task-specific knowledge, such as predefined initial states or reset reward functions. In this paper, we propose a novel ARL algorithm that can generate a curriculum adaptive to the agent's learning progress without task-specific knowledge. Our curriculum empowers the agent to autonomously reset to diverse and informative initial states. To achieve this, we introduce a success discriminator that estimates the success probability from each initial state when the agent follows the forward policy. The success discriminator is trained with relabeled transitions in a self-supervised manner. Our experimental results demonstrate that our ARL algorithm can generate an adaptive curriculum and enable the agent to efficiently bootstrap to solve sparse-reward maze navigation tasks, outperforming baselines with significantly fewer manual resets.Comment: 8 pages, 5 figure

Application of Recent Developments in Deep Learning to ANN-based Automatic Berthing Systems

Previous studies on Artificial Neural Network (ANN)-based automatic berthing showed considerable increases in performance by training ANNs with a set of berthing datasets. However, the berthing performance deteriorated when an extrapolated initial position was given. To overcome the extrapolation problem and improve the training performance, recent developments in Deep Learning (DL) are adopted in this paper. Recent activation functions, weight initialization methods, input data-scaling methods, a higher number of hidden layers, and Batch Normalization (BN) are considered, and their effectiveness has been analyzed based on loss functions, berthing performance histories, and berthing trajectories. Finally, it is shown that the use of recent activation and weight initialization method results in faster training convergence and a higher number of hidden layers. This leads to a better berthing performance over the training dataset. It is found that application of the BN can overcome the extrapolated initial position problem

Increased Risk of Ischemic Stroke during Sleep in Apneic Patients.

BACKGROUND AND PURPOSE:The literature indicates that obstructive sleep apnea (OSA) increases the risk of ischemic stroke. However, the causal relationship between OSA and ischemic stroke is not well established. This study examined whether preexisting OSA symptoms affect the onset of acute ischemic stroke. METHODS:We investigated consecutive patients who were admitted with acute ischemic stroke, using a standardized protocol including the Berlin Questionnaire on symptoms of OSA prior to stroke. The collected stroke data included the time of the stroke onset, risk factors, and etiologic subtypes. The association between preceding OSA symptoms and wake-up stroke (WUS) was assessed using multivariate logistic regression analysis. RESULTS:We identified 260 subjects with acute ischemic strokes with a definite onset time, of which 25.8% were WUS. The presence of preexisting witnessed or self-recognized sleep apnea was the only risk factor for WUS (adjusted odds ratio=2.055, 95% confidence interval=1.035-4.083, p=0.040). CONCLUSIONS:Preexisting symptoms suggestive of OSA were associated with the occurrence of WUS. This suggests that OSA contributes to ischemic stroke not only as a predisposing risk factor but also as a triggering factor. Treating OSA might therefore be beneficial in preventing stroke, particularly that occurring during sleep

Thermally activated flux flow in superconducting epitaxial FeSe0.6Te0.4 thin film

AbstractThe thermally activated flux flow effect has been studied in epitaxial FeSe0.6Te0.4 thin film grown by a PLD method through the electrical resistivity measurement under various magnetic fields for B//c and B//ab. The results showed that the thermally activated flux flow effect is well described by the nonlinear temperature-dependent activation energy. The evaluated apparent activation energy U0(B) is one order larger than the reported results and showed the double-linearity in both magnetic field directions. Furthermore, the FeSe0.6Te0.4 thin film shows the anisotropy of 5.6 near Tc and 2D-like superconducting behavior in thermally activated flux flow region. In addition, the vortex glass transition and the temperature dependence of the high critical fields were determined

Low-temperature synthesis of CuO-interlaced nanodiscs for lithium ion battery electrodes

In this study, we report the high-yield synthesis of 2-dimensional cupric oxide (CuO) nanodiscs through dehydrogenation of 1-dimensional Cu(OH)2 nanowires at 60°C. Most of the nanodiscs had a diameter of approximately 500 nm and a thickness of approximately 50 nm. After further prolonged reaction times, secondary irregular nanodiscs gradually grew vertically into regular nanodiscs. These CuO nanostructures were characterized using X-ray diffraction, transmission electron microscopy, and Brunauer-Emmett-Teller measurements. The possible growth mechanism of the interlaced disc CuO nanostructures is systematically discussed. The electrochemical performances of the CuO nanodisc electrodes were evaluated in detail using cyclic voltammetry and galvanostatic cycling. Furthermore, we demonstrate that the incorporation of multiwalled carbon nanotubes enables the enhanced reversible capacities and capacity retention of CuO nanodisc electrodes on cycling by offering more efficient electron transport paths