On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization

Extrapolation is a well-known technique for solving convex optimization and variational inequalities and recently attracts some attention for non-convex optimization. Several recent works have empirically shown its success in some machine learning tasks. However, it has not been analyzed for non-convex minimization and there still remains a gap between the theory and the practice. In this paper, we analyze gradient descent and stochastic gradient descent with extrapolation for finding an approximate first-order stationary point in smooth non-convex optimization problems. Our convergence upper bounds show that the algorithms with extrapolation can be accelerated than without extrapolation

Galantamine protects against hydrochloric acid aspirationinduced acute respiratory distress syndrome in rabbits

Purpose: To study the effect of galantamine on anti-inflammatory responses in acid aspiration-induced acute respiratory syndrome (ARDS), and the underlying mechanism.Methods: Six groups of male rabbits (156), i.e., control group, ARDS group, galantamine + ARDS (GAL) group, galantamine + ARDS + methyllycaconitine (MLA) group, galantamine + ARDS + vagotomy (Vag) group, and galantamine + ARDS + atropine sulfate (ATS) group. ARDS model was produced by acid aspiration. After 4 h, TNF-α, IL-6 and IL-1β were assayed in lung tissue, while corticosterone levels were determined in blood. Histopathological lesions and wet-to-dry (W/D) weight ratio of lung tissue (n = 10) were assessed. After 12 h, HMGB1 protein and corticosterone levels were determined in lung tissue and blood, respectively. Mortality rate was determined after 72 h.Results: Acid aspiration-induced ARDS induced disorganization of lung structure, and elevated TNF-α, IL-6, IL-1β, and HMGB1activities, and lung W/D weight ratio. The acid-induced ARDS, as well as increases in W/D weight ratio, pro-inflammatory cytokines and lung lesions were significantly decreased by galantamine pretreatment. Methyllycaconitin, vagotomy and atropine sulfate abolished the galantamine-induced suppression of acute inflammatory response, pathological changes in lungs, and W/D weight ratio. However, serum corticosterone levels were not significantly altered in each group.Conclusion: Galantamine reduces inflammation in acid aspiration-induced ARDS by the cholinergic anti-inflammatory pathway.Keywords: Galantamine, Hydrochloric acid, Aspiration-induced, Respiratory distress syndrome, Methyllycaconitin, Vagotomy, Atropine sulfat