Proximally Constrained Methods for Weakly Convex Optimization with Weakly Convex Constraints

Optimization models with non-convex constraints arise in many tasks in machine learning, e.g., learning with fairness constraints or Neyman-Pearson classification with non-convex loss. Although many efficient methods have been developed with theoretical convergence guarantees for non-convex unconstrained problems, it remains a challenge to design provably efficient algorithms for problems with non-convex functional constraints. This paper proposes a class of subgradient methods for constrained optimization where the objective function and the constraint functions are are weakly convex. Our methods solve a sequence of strongly convex subproblems, where a proximal term is added to both the objective function and each constraint function. Each subproblem can be solved by various algorithms for strongly convex optimization. Under a uniform Slater's condition, we establish the computation complexities of our methods for finding a nearly stationary point

First-Order Methods for Convex Constrained Optimization under Error Bound Conditions with Unknown Growth Parameters

We propose first-order methods based on a level-set technique for convex constrained optimization that satisfies an error bound condition with unknown growth parameters. The proposed approach solves the original problem by solving a sequence of unconstrained subproblems defined with different level parameters. Different from the existing level-set methods where the subproblems are solved sequentially, our method applies a first-order method to solve each subproblem independently and simultaneously, which can be implemented with either a single or multiple processors. Once the objective value of one subproblem is reduced by a constant factor, a sequential restart is performed to update the level parameters and restart the first-order methods. When the problem is non-smooth, our method finds an $\epsilon$-optimal and $\epsilon$-feasible solution by computing at most $O(\frac{G^{2/d}}{\epsilon^{2-2/d}}\ln^3(\frac{1}{\epsilon}))$ subgradients where $G>0$ and $d\geq 1$ are the growth rate and the exponent, respectively, in the error bound condition. When the problem is smooth, the complexity is improved to $O(\frac{G^{1/d}}{\epsilon^{1-1/d}}\ln^3(\frac{1}{\epsilon}))$. Our methods do not require knowing $G$, $d$ and any problem dependent parameters

Machine Learning Based on Event-Related EEG of Sustained Attention Differentiates Adults with Chronic High-Altitude Exposure from Healthy Controls

(1) Objective: The aim of this study was to examine the effect of high altitude on inhibitory control processes that underlie sustained attention in the neural correlates of EEG data, and explore whether the EEG data reflecting inhibitory control contain valuable information to classify high-altitude chronic hypoxia and plain controls. (2) Methods: 35 chronic high-altitude hypoxic adults and 32 matched controls were recruited. They were required to perform the go/no-go sustained attention task (GSAT) using event-related potentials. Three machine learning algorithms, namely a support vector machine (SVM), logistic regression (LR), and a decision tree (DT), were trained based on the related ERP components and neural oscillations to build a dichotomous classification model. (3) Results: Behaviorally, we found that the high altitude (HA) group had lower omission error rates during all observation periods than the low altitude (LA) group. Meanwhile, the ERP results showed that the HA participants had significantly shorter latency than the LAs for sustained potential (SP), indicating vigilance to response-related conflict. Meanwhile, event-related spectral perturbation (ERSP) analysis suggested that lowlander immigrants exposed to high altitudes may have compensatory activated prefrontal cortexes (PFC), as reflected by slow alpha, beta, and theta frequency-band neural oscillations. Finally, the machine learning results showed that the SVM achieved the optimal classification F1 score in the later stage of sustained attention, with an F1 score of 0.93, accuracy of 92.54%, sensitivity of 91.43%, specificity of 93.75%, and area under ROC curve (AUC) of 0.97. The results proved that SVM classification algorithms could be applied to identify chronic high-altitude hypoxia. (4) Conclusions: Compared with other methods, the SVM leads to a good overall performance that increases with the time spent on task, illustrating that the ERPs and neural oscillations may provide neuroelectrophysiological markers for identifying chronic plateau hypoxia