Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution

Throughout long history, natural species have learned to survive by evolving their physical structures adaptive to the environment changes. In contrast, current reinforcement learning (RL) studies mainly focus on training an agent with a fixed morphology (e.g., skeletal structure and joint attributes) in a fixed environment, which can hardly generalize to changing environments or new tasks. In this paper, we optimize an RL agent and its morphology through ``morphology-environment co-evolution (MECE)'', in which the morphology keeps being updated to adapt to the changing environment, while the environment is modified progressively to bring new challenges and stimulate the improvement of the morphology. This leads to a curriculum to train generalizable RL, whose morphology and policy are optimized for different environments. Instead of hand-crafting the curriculum, we train two policies to automatically change the morphology and the environment. To this end, (1) we develop two novel and effective rewards for the two policies, which are solely based on the learning dynamics of the RL agent; (2) we design a scheduler to automatically determine when to change the environment and the morphology. In experiments on two classes of tasks, the morphology and RL policies trained via MECE exhibit significantly better generalization performance in unseen test environments than SOTA morphology optimization methods. Our ablation studies on the two MECE policies further show that the co-evolution between the morphology and environment is the key to the success

Predictive value of hyponatremia for short-term mortality in supratentorial spontaneous intracerebral hemorrhage: a single center study

BackgroundHyponatremia is a common electrolyte disturbance in patients with neurological disease; however, its predictive role for outcome in patients with supratentorial spontaneous intracerebral hemorrhage (sICH) is controversial. This study aims to explore the association between hyponatremia within 7 days after bleeding and 90-day mortality in patients with supratentorial sICH.MethodsA retrospective analysis was conducted at our institution. Patients with sICH meeting the inclusion criteria were enrolled in this study. Multivariate regression analyses were performed to determine the predictive value of hyponatremia (serum sodium <135 mmol/L) for 90-day mortality and functional outcome. Subgroup analysis was performed based on the degree and duration of hyponatremia and therapeutic strategies. The Spearman correlation test was performed to explore the relationship between hyponatremia severity and duration with variables in a multivariate regression model. Kaplan–Meier curve was depicted to reveal the relationship between hyponatremia and mortality. The receiver operating characteristic (ROC) curve was plotted to show the diagnostic effect of the minimum concentration of serum sodium (sodiummin) on 90-day mortality.ResultsA total of 960 patients were enrolled, 19.6% (188) of whom were patients with hyponatremia and 26.0% (250) had 90-day mortality. The incidence of hyponatremia was roughly 2.5 times in non-survivors compared with survivors (34.8% vs. 14.2%). Multivariate regression analysis revealed that hyponatremia was the independent predictor of 90-day mortality (OR 2.763, 95%CI 1.836–4.157) and adverse outcome (OR 3.579, 95%CI 2.332–6.780). Subgroup analysis indicated an increased trend in mortality risk with both duration (more or less than 48 h) and severity of hyponatremia (mild, moderate, and severe) and confirmed the predictive value of hyponatremia for mortality in patients undergoing surgical intervention (external ventricular drainage, craniotomy, and decompressive craniectomy; all p < 0.05). The Spearman correlation test indicated no moderate or strong relationship between hyponatremia severity and duration with other variables in the multivariate model (all |rs| < 0.4). The ROC curve suggested the moderate diagnostic performance of sodiummin for mortality in both general patients and subgroups of therapeutic method patients (AUC from 0.6475 to 0.7384).ConclusionHyponatremia occurring in the first 7 days after bleeding is an independent predictor of 90-day morality and adverse outcome. Rigorous electrolyte scrutiny in patients treated surgically is required

Resource Consumption for Supporting Federated Learning Enabled Network Edge Intelligence

Federated learning (FL) has recently become one of the hottest focuses in network edge intelligence. In the FL framework, user equipments (UEs) train local machine learning (ML) models and transmit the trained models to an aggregator where a global model is formed and then sent back to UEs, such that FL can enable collaborative model training. In large-scale and dynamic edge networks, both local model training and transmission may not be always successful due to constrained power and computing resources at mobile devices, wireless channel impairments, bandwidth limitations, etc., which directly degrades FL performance in terms of model accuracy and/or training time. On the other hand, we need to quantify the benefits and cost of deploying edge intelligence when we plan to improve network performance by using artificial intelligence (AI) techniques which definitely incur certain cost. Therefore, it is imperative to deeply understand the relationship between the required multiple-dimensional resources and FL performance to facilitate FL enabled edge intelligence. In this paper, we construct an analytical model for investigating the relationship between the accuracy of ML model and consumed network resources in FL enabled edge networks. Based on the analytical model, we can explicitly quantify the trained model accuracy given spatial-temporal domain distribution, available user computing and communication resources. Numerical results validate the effectiveness of our theoretical modeling and analysis. Our analytical model in this paper provides some useful guidelines for appropriately promoting FL enabled edge network intelligence

Biofilms as self-shaping growing nematics

Active nematics are the nonequilibrium analog of passive liquid crystals in which anisotropic units consume free energy to drive emergent behavior. Similar to liquid crystal (LC) molecules in displays, ordering and dynamics in active nematics are sensitive to boundary conditions; however, unlike passive liquid crystals, active nematics, such as those composed of living matter, have the potential to regulate their boundaries through self-generated stresses. Here, using bacterial biofilms confined by a hydrogel as a model system, we show how a three-dimensional, living nematic can actively shape itself and its boundary in order to regulate its internal architecture through growth-induced stresses. We show that biofilms exhibit a sharp transition in shape from domes to lenses upon changing environmental stiffness or cell-substrate friction, which is explained by a theoretical model considering the competition between confinement and interfacial forces. The growth mode defines the progression of the boundary, which in turn determines the trajectories and spatial distribution of cell lineages. We further demonstrate that the evolving boundary defines the orientational ordering of cells and the emergence of topological defects in the interior of the biofilm. Our findings reveal novel self-organization phenomena in confined active matter and provide strategies for guiding the development of programmed microbial consortia with emergent material properties

The site conditions of the Guo Shou Jing Telescope

The weather at Xinglong Observing Station, where the Guo Shou Jing Telescope (GSJT) is located, is strongly affected by the monsoon climate in north-east China. The LAMOST survey strategy is constrained by these weather patterns. In this paper, we present a statistics on observing hours from 2004 to 2007, and the sky brightness, seeing, and sky transparency from 1995 to 2011 at the site. We investigate effects of the site conditions on the survey plan. Operable hours each month shows strong correlation with season: on average there are 8 operable hours per night available in December, but only 1-2 hours in July and August. The seeing and the sky transparency also vary with seasons. Although the seeing is worse in windy winters, and the atmospheric extinction is worse in the spring and summer, the site is adequate for the proposed scientific program of LAMOST survey. With a Monte Carlo simulation using historical data on the site condition, we find that the available observation hours constrain the survey footprint from 22h to 16h in right ascension; the sky brightness allows LAMOST to obtain the limit magnitude of V = 19.5mag with S/N = 10.Comment: 10 pages, 8 figures, accepted for publication in RA

XMM-Newton Observations of NGC 247: X-ray Population and a Supersoft Ultraluminous X-ray Source

We report on a new XMM-Newton observation of NGC 247 from December 2009. The galaxy contains a supersoft, ultraluminous X-ray source (ULX) whose spectrum consists of a thermal component with a temperature about 0.1 keV and a power-law tail with a photon index around 2.5. The thermal emission is absolutely the dominant component, contributing 96% of the total luminosity in the 0.3-10 keV band. Variability is detected at timescales of 10^2 s and longer with a \nu^-1 power spectrum. These properties are consistent with black hole binaries in the thermal state and suggest the presence of an intermediate mass black hole of at least 600 solar masses. However, the integrated rms power is much higher than typically found in the thermal state. An alternative explanation of the emission could be a photosphere with a radius about 10^9 cm. A possible absorption feature around 1 keV is detected, which may be due to absorption of highly ionized winds. X-ray sources within the disk of NGC 247 have a luminosity function consistent with that found in low mass X-ray binaries. We confirm previous results that X-rays from the quasar PHL 6625 may be absorbed by gas in NGC 247, mainly at energies below 0.3 keV.Comment: accepted for publication in the Astrophysical Journa

Attacking practical quantum key distribution system with wavelength dependent beam splitter and multi-wavelength sources

Unconditional security of quantum key distribution protocol can be guaranteed by the basic property of quantum mechanics. Unfortunately, the practical quantum key distribution system always have some imperfections, and the practical system may be attacked if the imperfection can be controlled by the eavesdropper Eve. Applying the fatal security loophole introduced by the imperfect beam splitter's wavelength dependent optical property, we propose wavelength-dependent attacking model, which can be applied to almost all practical quantum key distribution systems with the passive state modulation and photon state detection after the practical beam splitter. Utilizing our attacking model, we experimentally demonstrate the attacking system based on practical polarization encoding quantum key distribution system with almost 100% success probability. Our result demonstrate that all practical devices require tightened security inspection for avoiding side channel attacks in practical quantum key distribution experimental realizations

Preoperative radiomic signature based on CT images for noninvasive evaluation of localized nephroblastoma in pediatric patients

BackgroundNephron sparing nephrectomy may not reduce the prognosis of nephroblastoma in the absence of involvement of the renal capsule, sinus vessels, and lymph nodes, However, there is no accurate preoperative noninvasive evaluation method at present.Materials and methods105 nephroblastoma patients underwent contrast-enhanced CT scan between 2013 and 2020 in our hospital were retrospectively collected, including 59 cases with localized stage and 46 cases with non-localized stage, and then were divided into training cohort (n= 73) and validation cohort (n= 32) according to the order of CT scanning time. After lesion segmentation and data preprocessing, radiomic features were extracted from each volume of interest. The multi-step procedure including Pearson correlation analysis and sequential forward floating selection was performed to produce radiomic signature. Prediction model was constructed using the radiomic signature and Logistic Regression classifier for predicting the localized nephroblastoma in the training cohort. Finally, the model performance was validated in the validation cohort.ResultsA total of 1652 radiomic features have been extracted, from which TOP 10 features were selected as the radiomic signature. The area under the receiver operating characteristic curve, accuracy, sensitivity and specificity of the prediction model were 0.796, 0.795, 0.732 and 0.875 for the training cohort respectively, and 0.710, 0.719, 0.611 and 0.857 for the validation cohort respectively. The result comparison with prediction models composed of different machine learning classifiers and different parameters also manifest the effectiveness of our radiomic model.ConclusionA logistic regression model based on radiomic features extracted from preoperative CT images had good ability to noninvasively predict nephroblastoma without renal capsule, sinus vessel, and lymph node involvement