Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

In this paper, we propose an accelerated quasi-Newton proximal extragradient (A-QPNE) method for solving unconstrained smooth convex optimization problems. With access only to the gradients of the objective, we prove that our method can achieve a convergence rate of ${O}\bigl(\min\{\frac{1}{k^2}, \frac{\sqrt{d\log k}}{k^{2.5}}\}\bigr)$, where $d$ is the problem dimension and $k$ is the number of iterations. In particular, in the regime where $k = {O}(d)$, our method matches the optimal rate of ${O}(\frac{1}{k^2})$ by Nesterov's accelerated gradient (NAG). Moreover, in the the regime where $k = \Omega(d \log d)$, it outperforms NAG and converges at a faster rate of ${O}\bigl(\frac{\sqrt{d\log k}}{k^{2.5}}\bigr)$. To the best of our knowledge, this result is the first to demonstrate a provable gain of a quasi-Newton-type method over NAG in the convex setting. To achieve such results, we build our method on a recent variant of the Monteiro-Svaiter acceleration framework and adopt an online learning perspective to update the Hessian approximation matrices, in which we relate the convergence rate of our method to the dynamic regret of a specific online convex optimization problem in the space of matrices.Comment: 44 pages, 1 figur

Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergence

Quasi-Newton algorithms are among the most popular iterative methods for solving unconstrained minimization problems, largely due to their favorable superlinear convergence property. However, existing results for these algorithms are limited as they provide either (i) a global convergence guarantee with an asymptotic superlinear convergence rate, or (ii) a local non-asymptotic superlinear rate for the case that the initial point and the initial Hessian approximation are chosen properly. In particular, no current analysis for quasi-Newton methods guarantees global convergence with an explicit superlinear convergence rate. In this paper, we close this gap and present the first globally convergent quasi-Newton method with an explicit non-asymptotic superlinear convergence rate. Unlike classical quasi-Newton methods, we build our algorithm upon the hybrid proximal extragradient method and propose a novel online learning framework for updating the Hessian approximation matrices. Specifically, guided by the convergence analysis, we formulate the Hessian approximation update as an online convex optimization problem in the space of matrices, and we relate the bounded regret of the online problem to the superlinear convergence of our method.Comment: 33 pages, 1 figure, accepted to COLT 202

Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem

In this paper, we study a class of stochastic bilevel optimization problems, also known as stochastic simple bilevel optimization, where we minimize a smooth stochastic objective function over the optimal solution set of another stochastic convex optimization problem. We introduce novel stochastic bilevel optimization methods that locally approximate the solution set of the lower-level problem via a stochastic cutting plane, and then run a conditional gradient update with variance reduction techniques to control the error induced by using stochastic gradients. For the case that the upper-level function is convex, our method requires $\tilde{\mathcal{O}}(\max\{1/\epsilon_f^{2},1/\epsilon_g^{2}\})$ stochastic oracle queries to obtain a solution that is $\epsilon_f$-optimal for the upper-level and $\epsilon_g$-optimal for the lower-level. This guarantee improves the previous best-known complexity of $\mathcal{O}(\max\{1/\epsilon_f^{4},1/\epsilon_g^{4}\})$. Moreover, for the case that the upper-level function is non-convex, our method requires at most $\tilde{\mathcal{O}}(\max\{1/\epsilon_f^{3},1/\epsilon_g^{3}\})$ stochastic oracle queries to find an $(\epsilon_f, \epsilon_g)$-stationary point. In the finite-sum setting, we show that the number of stochastic oracle calls required by our method are $\tilde{\mathcal{O}}(\sqrt{n}/\epsilon)$ and $\tilde{\mathcal{O}}(\sqrt{n}/\epsilon^{2})$ for the convex and non-convex settings, respectively, where $\epsilon=\min \{\epsilon_f,\epsilon_g\}$

Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems

One of the key challenges of learning an online recommendation model is the temporal domain shift, which causes the mismatch between the training and testing data distribution and hence domain generalization error. To overcome, we propose to learn a meta future gradient generator that forecasts the gradient information of the future data distribution for training so that the recommendation model can be trained as if we were able to look ahead at the future of its deployment. Compared with Batch Update, a widely used paradigm, our theory suggests that the proposed algorithm achieves smaller temporal domain generalization error measured by a gradient variation term in a local regret. We demonstrate the empirical advantage by comparing with various representative baselines

The predictive value of preoperative luteinizing hormone to follicle stimulating hormone ratio for ovulation abnormalities recovery after laparoscopic sleeve gastrectomy: A prospective cohort study

IntroductionObesity-related ovulation abnormalities (OA) affect fertility. LSG is the most frequent bariatric operation. However, no research has identified a reliable indicator for predicting OA recovery after LSG. The purpose of this research was to examine the prognostic usefulness of preoperative the luteinizing hormone (LH) to follicle-stimulating hormone (FSH) ratio (LFR).MethodsOur department conducted a prospective study from 2016 to 2021. Venous blood was typically tested 3 days before surgery to get the preoperative LFR. Descriptive data, preoperative and postoperative variables were also collected. Binary logistic regression related preoperative LFR with OA recovery. The receiver operating characteristic (ROC) curve evulated preoperative LFR’s predictive capability.ResultsA total of 157 women with a complete follow-up of one year were included. LFR was the only factor linked with OA (P < 0.001). AUC (area under the ROC curve) = 0.915, cutoff = 1.782, sensitivity = 0.93, and specificity = 0.82.DiscussionOverall, LSG has a favorable surgical result, with a %TWL of 66.082 ± 12.012 at 12 months postoperatively. Preoperative sexual hormone levels, as expressed by LFR, has the potential to predict the fate of OA following LSG at one year post-operatively

Towards high-quality biodiesel production from microalgae using original and anaerobically-digested livestock wastewater

In this study, we conducted proof-of-concept research towards the simultaneous treatment of livestock wastewater and the generation of high-quality biodiesel, through microalgae technology. Both original (OPE) and anaerobically-digested (DPE) piggery effluents were investigated for the culture of the microalgae, Desmodesmus sp. EJ8-10. After 14 days’ cultivation, the dry biomass from microalgae cultivated in OPE increased from an initial value of 0.01 g/L to 0.33-0.39 g/L, while those growing in DPE only achieved a final dried mass of 0.15-0.35 g/L, under similar initial ammonium nitrogen (NH4+-N) concentrations. The significantly higher microalgal biomass production achieved in the OPE medium may have been supported by the abundance of both macronutrient, such as phosphorus (P), and of micronutrients, such as trace elements, present in the OPE, which may not been present in similar quantities in the DPE. However, a higher lipid content was observed (19.4-28%) in microalgal cells from DPE cultures than those (18.7-22.3%) from OPE cultures. Moreover, the fatty acid compositions in the microalgae cultured in DPE contained high levels of monounsaturated fatty acids (MUFAs) and total C16-C18 acids, which would afford a superior potential for high-quality biodiesel production. The N/P ratio (15.4:1) in OPE was much closer to that indicated by previous studies to be the most suitable (16:1) for microalgae growth, when compared with that determined from the DPE culture medium. This may facilitate protein synthesis in the algal cells and induce a lower accumulation of lipids. Based on these findings, we proposed a new flowsheet for sustainable livestock waste managemen

Valorisation of microalgae residues after lipid extraction: Pyrolysis characteristics for biofuel production

As a promising source of renewable energy, biofuel from microalgae pyrolysis is seen as a competitive alternative to fossil fuels. However, currently, the widely applied pre-treatment process of lipid extraction results in large amounts of microalgae residues, which though with energy potential, being considered as process wastes and ignored of its re-utilization potential. In this study, a new workflow of biofuel generation from microalgae biomass through lipid extraction and pyrolysis of defatted microalgae residues was proposed and assessed. The effects of lipid extraction and pyrolysis temperature (350–750 ℃) on pyrolysis products were investigated, and pyrolysis pathways were postulated. To address the twin goals of lowering emission of pollutants and elevating energy products, an optimal pyrolysis temperature of 650 ℃ was suggested. After extraction of lipids, the relative contents of valuable products (aromatic, aliphatic hydrocarbons and fatty acids) and some harmful by-products, e.g., PAHs, significantly reduced, while other harmful substrates, e.g., nitrogen-compounds increased. Mechanistic investigations indicated that pyrolysis of proteins without the presence of lipids could promote higher production of nitrogen-containing organics and aromatics. These results reveal the effects of lipid extraction and variation of temperature on microalgal pyrolysis, and also provide a basis for full utilization of microalgae as an aid to alleviate many fossil energy problems