Self-Supervised Curriculum Generation for Autonomous Reinforcement Learning without Task-Specific Knowledge

A significant bottleneck in applying current reinforcement learning algorithms to real-world scenarios is the need to reset the environment between every episode. This reset process demands substantial human intervention, making it difficult for the agent to learn continuously and autonomously. Several recent works have introduced autonomous reinforcement learning (ARL) algorithms that generate curricula for jointly training reset and forward policies. While their curricula can reduce the number of required manual resets by taking into account the agent's learning progress, they rely on task-specific knowledge, such as predefined initial states or reset reward functions. In this paper, we propose a novel ARL algorithm that can generate a curriculum adaptive to the agent's learning progress without task-specific knowledge. Our curriculum empowers the agent to autonomously reset to diverse and informative initial states. To achieve this, we introduce a success discriminator that estimates the success probability from each initial state when the agent follows the forward policy. The success discriminator is trained with relabeled transitions in a self-supervised manner. Our experimental results demonstrate that our ARL algorithm can generate an adaptive curriculum and enable the agent to efficiently bootstrap to solve sparse-reward maze navigation tasks, outperforming baselines with significantly fewer manual resets.Comment: 8 pages, 5 figure

Subcutaneous Sacrococcygeal Myxopapillary Ependymoma in Asian Female:A Case Report

Subcutaneous sacrococcygeal myxopapillary ependymoma is extremely rare tumor that has a tendency to develop in children and adolescents. There have been several case reports and sporadic reports in the literature. However, no case has been reported in an Asian patient, to the best of our knowledge. We describe a 25-year-old Asian female patient with a subcutaneous sacrococcygeal myxopapillary ependymoma that had been clinically diagnosed as a pilonidal cyst. The tumor was treated successfully by surgical excision and the patient is doing well without evidence of local recurrence or distant metastasis at 2 years after surgery.

Andro-Simnet: Android Malware Family Classification Using Social Network Analysis

While the rapid adaptation of mobile devices changes our daily life more conveniently, the threat derived from malware is also increased. There are lots of research to detect malware to protect mobile devices, but most of them adopt only signature-based malware detection method that can be easily bypassed by polymorphic and metamorphic malware. To detect malware and its variants, it is essential to adopt behavior-based detection for efficient malware classification. This paper presents a system that classifies malware by using common behavioral characteristics along with malware families. We measure the similarity between malware families with carefully chosen features commonly appeared in the same family. With the proposed similarity measure, we can classify malware by malware's attack behavior pattern and tactical characteristics. Also, we apply a community detection algorithm to increase the modularity within each malware family network aggregation. To maintain high classification accuracy, we propose a process to derive the optimal weights of the selected features in the proposed similarity measure. During this process, we find out which features are significant for representing the similarity between malware samples. Finally, we provide an intuitive graph visualization of malware samples which is helpful to understand the distribution and likeness of the malware networks. In the experiment, the proposed system achieved 97% accuracy for malware classification and 95% accuracy for prediction by K-fold cross-validation using the real malware dataset.Comment: 13 pages, 11 figures, dataset link: http://ocslab.hksecurity.net/Datasets/andro-simnet , demo video: https://youtu.be/JmfS-ZtCbg4 , In Proceedings of the 16th Annual Conference on Privacy, Security and Trust (PST), 201

Thermoelectric properties of graphene incorporated thermoelectric materials

Thermoelectric materials, which can change the waste heat into the usable electricity, are interested in various field of applications such as vehicle, ship, power plane, and so on. To enhance the thermoelectric properties, high electrical conductivity, high Seebeck coefficient, and low thermal conductivity should be conducted, however, the trade-off relation between electronic property and thermal property in terms of carrier concentration could be the bottle-neck on the enhancement of thermoelectric properties of the materials. In this presentation, we discuss with the graphene incorporation in the conventional thermoelectric materials, which could lead to independently control electric and thermal properties

Electrochemical performance of NixCo1-xMoO4 (0 ≤ x ≤ 1) nanowire anodes for lithium-ion batteries

NixCo1-xMoO4 (0 ≤ x ≤ 1) nanowire electrodes for lithium-ion rechargeable batteries have been synthesized via a hydrothermal method, followed by thermal post-annealing at 500°C for 2 h. The chemical composition of the nanowires was varied, and their morphological features and crystalline structures were characterized using field-emission scanning electron microscopy and X-ray powder diffraction. The reversible capacity of NiMoO4 and Ni0.75Co0.25MoO4 nanowire electrodes was larger (≈520 mA h/g after 20 cycles at a rate of 196 mA/g) than that of the other nanowires. This enhanced electrochemical performance of NixCo1-xMoO4 nanowires with high Ni content was ascribed to their larger surface area and efficient electron transport path facilitated by their one-dimensional nanostructure

Vitamin D3 Supplementation Reduces the Symptoms of Upper Respiratory Tract Infection during Winter Training in Vitamin D-Insufficient Taekwondo Athletes: A Randomized Controlled Trial

Vitamin D insufficiency may be associated with increased risk of upper respiratory tract infection (URTI) in athletes. This study examined the effects of vitamin D3 supplementation on salivary immune functions and symptoms of URTI in vitamin D-insufficient taekwondo athletes. Twenty-five male taekwondo athletes, aged 19–22 years with vitamin D insufficiency [serum 25-hydroxyvitamin-D concentrations (25(OH)D, 31.3 ± 1.39 nmol/L)], participated in this study. They were randomized to receive 5000 IU/day of vitamin D3 (n = 13) or placebo capsule (n = 12) during 4 weeks of winter training. Blood samples were collected two times (pre- and post-tests) for analyzing serum 25(OH)D concentration while salivary samples were obtained three times (pre-, mid-, and post-tests) for secretory immunoglobulin A (SIgA) and lactoferrin analyses. The symptoms of URTI were reported daily during the intervention. Serum 25(OH)D concentration significantly increased by 255.6% in the vitamin D group, whereas in the placebo group it did not change (p \u3c 0.001). While the significant increase in SIgA was observed in both groups (p \u3c 0.001), elevated salivary lactoferrin level in response to winter training was found only in the placebo group (p = 0.011). The change in serum 25(OH)D concentration was negatively associated with total URTI symptoms (r = −0.435, p = 0.015). Vitamin D3 supplementation may be effective in reducing the symptoms of URTI during winter training in vitamin D-insufficient taekwondo athletes

Tau functions as Widom constants

We define a tau function for a generic Riemann-Hilbert problem posed on a union of non-intersecting smooth closed curves with jump matrices analytic in their neighborhood. The tau function depends on parameters of the jumps and is expressed as the Fredholm determinant of an integral operator with block integrable kernel constructed in terms of elementary parametrices. Its logarithmic derivatives with respect to parameters are given by contour integrals involving these parametrices and the solution of the Riemann-Hilbert problem. In the case of one circle, the tau function coincides with Widom's determinant arising in the asymptotics of block Toeplitz matrices. Our construction gives the Jimbo-Miwa-Ueno tau function for Riemann-Hilbert problems of isomonodromic origin (Painlev\'e VI, V, III, Garnier system, etc) and the Sato-Segal-Wilson tau function for integrable hierarchies such as Gelfand-Dickey and Drinfeld-Sokolov.Comment: 26 pages, 6 figure