Bilevel Optimization without Lower-Level Strong Convexity from the Hyper-Objective Perspective

Bilevel optimization reveals the inner structure of otherwise oblique optimization problems, such as hyperparameter tuning and meta-learning. A common goal in bilevel optimization is to find stationary points of the hyper-objective function. Although this hyper-objective approach is widely used, its theoretical properties have not been thoroughly investigated in cases where the lower-level functions lack strong convexity. In this work, we take a step forward and study the hyper-objective approach without the typical lower-level strong convexity assumption. Our hardness results show that the hyper-objective of general convex lower-level functions can be intractable either to evaluate or to optimize. To tackle this challenge, we introduce the gradient dominant condition, which strictly relaxes the strong convexity assumption by allowing the lower-level solution set to be non-singleton. Under the gradient dominant condition, we propose the Inexact Gradient-Free Method (IGFM), which uses the Switching Gradient Method (SGM) as the zeroth order oracle, to find an approximate stationary point of the hyper-objective. We also extend our results to nonsmooth lower-level functions under the weak sharp minimum condition

Ultra-compact structure in intermediate-luminosity radio quasars: building a sample of standard cosmological rulers and improving the dark energy constraints up to z∼3z\sim 3

In this paper, we present a new compiled milliarcsecond compact radio data set of 120 intermediate-luminosity quasars in the redshift range $0.46< z <2.76$. These quasars show negligible dependence on redshifts and intrinsic luminosity, and thus represents, in the standard model of cosmology, a fixed comoving-length of standard ruler. We implement a new cosmology-independent technique to calibrate the linear size of of this standard ruler as $l_m= 11.03\pm0.25$ pc, which is the typical radius at which AGN jets become opaque at the observed frequency $\nu\sim 2$ GHz. In the framework of flat $\Lambda$CDM model, we find a high value of the matter density parameter, $\Omega_m=0.322^{+0.244}_{-0.141}$, and a low value of the Hubble constant, $H_0=67.6^{+7.8}_{-7.4}\; \rm{kms}^{-1}\rm{Mpc}^{-1}$, which is in excellent agreement with the CMB anisotropy measurements by \textit{Planck}. We obtain ${\Omega_m}=0.309^{+0.215}_{-0.151}$, $w=-0.970^{+0.500}_{-1.730}$ at 68.3% CL for the constant $w$ of a dynamical dark-energy model, which demonstrates no significant deviation from the concordance $\Lambda$CDM model. Consistent fitting results are also obtained for other cosmological models explaining the cosmic acceleration, like Ricci dark energy (RDE) or Dvali-Gabadadze-Porrati (DGP) brane-world scenario. While no significant change in $w$ with redshift is detected, there is still considerable room for evolution in $w$ and the transition redshift at which $w$ departing from -1 is located at $z\sim 2.0$. Our results demonstrate that the method extensively investigated in our work on observational radio quasar data can be used to effectively derive cosmological information. Finally, we find the combination of high-redshift quasars and low-redshift clusters may provide an important source of angular diameter distances, considering the redshift coverage of these two astrophysical probes.Comment: 36 pages, 5 tables, 16 figures, A&A, in pres

Iteratively Learn Diverse Strategies with State Distance Information

In complex reinforcement learning (RL) problems, policies with similar rewards may have substantially different behaviors. It remains a fundamental challenge to optimize rewards while also discovering as many diverse strategies as possible, which can be crucial in many practical applications. Our study examines two design choices for tackling this challenge, i.e., diversity measure and computation framework. First, we find that with existing diversity measures, visually indistinguishable policies can still yield high diversity scores. To accurately capture the behavioral difference, we propose to incorporate the state-space distance information into the diversity measure. In addition, we examine two common computation frameworks for this problem, i.e., population-based training (PBT) and iterative learning (ITR). We show that although PBT is the precise problem formulation, ITR can achieve comparable diversity scores with higher computation efficiency, leading to improved solution quality in practice. Based on our analysis, we further combine ITR with two tractable realizations of the state-distance-based diversity measures and develop a novel diversity-driven RL algorithm, State-based Intrinsic-reward Policy Optimization (SIPO), with provable convergence properties. We empirically examine SIPO across three domains from robot locomotion to multi-agent games. In all of our testing environments, SIPO consistently produces strategically diverse and human-interpretable policies that cannot be discovered by existing baselines

ScalingNet: Extracting features from raw EEG data for emotion recognition

Convolutional Neural Networks (CNNs) have achieved remarkable performance breakthroughs in a variety of tasks. Recently, CNN-based methods that are fed with hand-extracted EEG features have steadily improved their performance on the emotion recognition task. In this paper, we propose a novel convolutional layer, called the Scaling Layer, which can adaptively extract effective data-driven spectrogram-like features from raw EEG signals. Furthermore, it exploits convolutional kernels scaled from one data-driven pattern to exposed a frequency-like dimension to address the shortcomings of prior methods requiring hand-extracted features or their approximations. ScalingNet, the proposed neural network architecture based on the Scaling Layer, has achieved state-of-the-art results across the established DEAP and AMIGOS benchmark datasets

In Situ Measurements of the Mechanical Properties of Electrochemically Deposited Li₂CO₃ and Li₂O Nanorods

Solid-electrolyte interface (SEI) is “the most important but least understood (component) in rechargeable Li-ion batteries”. The ideal SEI requires high elastic strength and can resist the penetration of a Li dendrite mechanically, which is vital for inhibiting the dendrite growth in lithium batteries. Even though Li$_{2}$CO$_{3}$ and Li$_{2}$O are identified as the major components of SEI, their mechanical properties are not well understood. Herein, SEI-related materials such as Li$_{2}$CO$_{3}$ and Li$_{2}$O were electrochemically deposited using an environmental transmission electron microscopy (ETEM), and their mechanical properties were assessed by in situ atomic force microscopy (AFM) and inverse finite element simulations. Both Li$_{2}$CO$_{3}$ and Li$_{2}$O exhibit nanocrystalline structures and good plasticity. The ultimate strength of Li$_{2}$CO$_{3}$ ranges from 192 to 330 MPa, while that of Li$_{2}$O is less than 100 MPa. These results provide a new understanding of the SEI and its related dendritic problems in lithium batteries

Novel hypoxia-related gene signature for predicting prognoses that correlate with the tumor immune microenvironment in NSCLC

Background: Intratumoral hypoxia is widely associated with the development of malignancy, treatment resistance, and worse prognoses. The global influence of hypoxia-related genes (HRGs) on prognostic significance, tumor microenvironment characteristics, and therapeutic response is unclear in patients with non-small cell lung cancer (NSCLC).Method: RNA-seq and clinical data for NSCLC patients were derived from The Cancer Genome Atlas (TCGA) database, and a group of HRGs was obtained from the MSigDB. The differentially expressed HRGs were determined using the limma package; prognostic HRGs were identified via univariate Cox regression. Using the least absolute shrinkage and selection operator (LASSO) and multivariate Cox regression, an optimized prognostic model consisting of nine HRGs was constructed. The prognostic model’s capacity was evaluated by Kaplan‒Meier survival curve analysis and receiver operating characteristic (ROC) curve analysis in the TCGA (training set) and GEO (validation set) cohorts. Moreover, a potential biological pathway and immune infiltration differences were explained.Results: A prognostic model containing nine HRGs (STC2, ALDOA, MIF, LDHA, EXT1, PGM2, ENO3, INHA, and RORA) was developed. NSCLC patients were separated into two risk categories according to the risk score generated by the hypoxia model. The model-based risk score had better predictive power than the clinicopathological method. Patients in the high-risk category had poor recurrence-free survival in the TCGA (HR: 1.426; 95% CI: 0.997–2.042; p = 0.046) and GEO (HR: 2.4; 95% CI: 1.7–3.2; p &lt; 0.0001) cohorts. The overall survival of the high-risk category was also inferior to that of the low-risk category in the TCGA (HR: 1.8; 95% CI: 1.5–2.2; p &lt; 0.0001) and GEO (HR: 1.8; 95% CI: 1.4–2.3; p &lt; 0.0001) cohorts. Additionally, we discovered a notable distinction in the enrichment of immune-related pathways, immune cell abundance, and immune checkpoint gene expression between the two subcategories.Conclusion: The proposed 9-HRG signature is a promising indicator for predicting NSCLC patient prognosis and may be potentially applicable in checkpoint therapy efficiency prediction

Intelligent semantic acquisition and smart decision support system of coal mine safety hazards

In view of problems of lacking intelligent semantic extraction function and multi—attribute interconnection retrieval analysis and decision function，and low intelligence degree in existing intelligent semantic acquisition and decision system of coal mine safety hazards, a kind of intelligent semantic acquisition and smart decision support system of coal mine safety hazards based on improved convolutional neural network (CNN) and ant colony optimization (ACO) was designed. The system adopts CNN—based intelligent semantic acquisition model, and uses CNN algorithm to match the close semantic keywords with the highest similarity, and uses mapping table to concern the standard keywords, so as to solve problem of low matching accuracy of semantic keywords. The system adopts ACO—based intelligent retrieval model, and uses negative feedback and positive reinforcement method of ACO algorithm to mark high—frequency retrieval rules, so as to realize intelligent display of high—frequency retrieval rules. The experiment and application results show that the system can realize functions such as interconnection query of multi—attribute semantic keywords, intelligent display of high—frequency retrieval rules, real—time tracking of data related to hidden danger, and diversified display of data charts, intelligent generation of decision analysis and early warning reports