Strangeness-driven Exploration in Multi-Agent Reinforcement Learning

Efficient exploration strategy is one of essential issues in cooperative multi-agent reinforcement learning (MARL) algorithms requiring complex coordination. In this study, we introduce a new exploration method with the strangeness that can be easily incorporated into any centralized training and decentralized execution (CTDE)-based MARL algorithms. The strangeness refers to the degree of unfamiliarity of the observations that an agent visits. In order to give the observation strangeness a global perspective, it is also augmented with the the degree of unfamiliarity of the visited entire state. The exploration bonus is obtained from the strangeness and the proposed exploration method is not much affected by stochastic transitions commonly observed in MARL tasks. To prevent a high exploration bonus from making the MARL training insensitive to extrinsic rewards, we also propose a separate action-value function trained by both extrinsic reward and exploration bonus, on which a behavioral policy to generate transitions is designed based. It makes the CTDE-based MARL algorithms more stable when they are used with an exploration method. Through a comparative evaluation in didactic examples and the StarCraft Multi-Agent Challenge, we show that the proposed exploration method achieves significant performance improvement in the CTDE-based MARL algorithms.Comment: 9 pages, 7 figure

Prevalence of human parechovirus and enterovirus in cerebrospinal fluid samples in children in Jinju, Korea

PurposeHuman parechovirus (HPeV) and enterovirus (EV) are causative agents of a sepsis-like illness in neonates and of infections of the central nervous system in young children. The objectives of this study were to assess the prevalence of HPeV3 and EV infection in young children with a sepsis-like illness or with meningitis in Jinju, Korea.MethodsCerebrospinal fluid (CSF) samples were collected from 267 patients (age range, 1 day to 5 years) and assessed for HPeV and EV by performing reverse transcription polymerase chain reaction assay. Amplification products of the VP3/VP1 region of HPeV and of the VP1 region of EV were sequenced to identify the virus type.ResultsHPeV and EV were detected in 3.4% and 7.5% of the total CSF samples assessed, respectively. The age distribution of EV-positive patients (median age, 1.4 months) had a significantly broader range than that of HPeV-positive patients (median age, 7.8 months). The peak seasons for HPeV and EV infection were spring and summer, respectively. The clinical symptoms for HPeV and EV infection were similar, and fever was the most common symptom. Pleocytosis was detected in 22.2% of HPeV-positive patients and 35.5% of EV-positive patients. The VP3/VP1 gene sequence of the nine Korean strains clustered most closely with the Japanese strain (AB759202).ConclusionThe data indicate that HPeV infection is predominant in young infants (<6 months) and that meningitis without pleocytosis was caused by both HPeV and EV infection in children

A Target Coverage Scheduling Scheme Based on Genetic Algorithms in Directional Sensor Networks

As a promising tool for monitoring the physical world, directional sensor networks (DSNs) consisting of a large number of directional sensors are attracting increasing attention. As directional sensors in DSNs have limited battery power and restricted angles of sensing range, maximizing the network lifetime while monitoring all the targets in a given area remains a challenge. A major technique to conserve the energy of directional sensors is to use a node wake-up scheduling protocol by which some sensors remain active to provide sensing services, while the others are inactive to conserve their energy. In this paper, we first address a Maximum Set Covers for DSNs (MSCD) problem, which is known to be NP-complete, and present a greedy algorithm-based target coverage scheduling scheme that can solve this problem by heuristics. This scheme is used as a baseline for comparison. We then propose a target coverage scheduling scheme based on a genetic algorithm that can find the optimal cover sets to extend the network lifetime while monitoring all targets by the evolutionary global search technique. To verify and evaluate these schemes, we conducted simulations and showed that the schemes can contribute to extending the network lifetime. Simulation results indicated that the genetic algorithm-based scheduling scheme had better performance than the greedy algorithm-based scheme in terms of maximizing network lifetime

3,4-Dihydroxy­phenyl 3,4,5-trimethoxy­benzoate

In the title compound, C16H16O7, the dihedral angle between the two benzene rings is 82.02 (7)°. The crystal structure is stabilized by inter­molecular O—H⋯O hydrogen bonds, which link the mol­ecules into a two-dimensional network

Bilateral iliac and popliteal arterial thrombosis in a child with focal segmental glomerulosclerosis

Thromboembolic complications (TECs) are clinically important sequelae of nephrotic syndrome (NS). The incidence of TECs in children is approximately 2%–5%. The veins are the most commonly affected sites, particularly the deep veins in the legs, the inferior vena cava, the superior vena cava, and the renal veins. Arterial thrombosis, which is less common, typically occurs in the cerebral, pulmonary, and femoral arteries, and is associated with the use of steroids and diuretics. Popliteal artery thrombosis in children has been described in cases of traumatic dissection, osteochondroma, Mycoplasma pneumoniae infection, and fibromuscular dysplasia. We report of a 33-month-old girl with bilateral iliac and popliteal arterial thrombosis associated with steroid-resistant NS due to focal segmental glomerulosclerosis. Her treatment involved thrombectomy and intravenous heparinization, followed by oral warfarin for 8 months. Herein, we report a rare case of spontaneous iliac and popliteal arterial thrombosis in a young child with NS

Thoracic paravertebral block for breast surgery in a pregnant woman -A case report-

Non-obstetrical surgery during the first trimester is stressful to both the mother and the fetus. Anesthesiologists are also stressed, not only because of the effects of surgery itself, but also because of the uncertain influences of anesthesia thrown upon on the fetus. The authors present a case of breast surgery successfully performed on a woman 8 weeks pregnant requiring removal of breast abscess by the application of thoracic paravertebral block without any complications. Thoracic paravertebral block may be a safe anesthetic method for non-obstetric surgery during early pregnancy

Interferometric detection of prostate specific antigen based on enzyme immunoassay

AbstractInterferometric detection of Prostate-specific antigen (PSA) based on enzyme immunoassay are investigated. Refractive index changes of substrate are measured for PSA detection. Michelson scheme of optical interferometer was used so as to be applicable to a disposable fluidic chip. When interferometer is used for the measurements of refractive index changes, the detection is over 8 times more sensitive than that of absorbance changes for the same amount of target protein