A unified convergence analysis for shuffling-type gradient methods

In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works with any sampling without replacement strategy and covers many known variants such as randomized reshuffling, deterministic or randomized single permutation, and cyclic and incremental gradient schemes. We focus on two different settings: strongly convex and nonconvex problems, but also discuss the non-strongly convex case. Our main contribution consists of new non-asymptotic and asymptotic convergence rates for a wide class of shuffling-type gradient methods in both nonconvex and convex settings. We also study uniformly randomized shuffling variants with different learning rates and model assumptions. While our rate in the nonconvex case is new and significantly improved over existing works under standard assumptions, the rate on the strongly convex one matches the existing best-known rates prior to this paper up to a constant factor without imposing a bounded gradient condition. Finally, we empirically illustrate our theoretical results via two numerical examples: nonconvex logistic regression and neural network training examples. As byproducts, our results suggest some appropriate choices for diminishing learning rates in certain shuffling variants

Faktor Penguat Pada Peningkatan Kinerja Karyawan PT. Gading Murni Surabaya

The purpose of this study is to determine which factors as an amplifier to improve the performance of employees of PT. Gading Murni Surabaya, between the leadership style and compensation received by employees. This study uses a survey approach by collecting data using a questionnaire of 50 respondents then analyzed using quantitative methods. The regession equation is Y = 8.009 + 0,223 X1 + 0,240 X2. The results of this study concluded that the leadership style and compensation variables had a corrected item total correlation value exceeding r table = 0.284 and the reliability test of the leadership style variable, and the alpha cronbach's compensation results exceeded 0.060, which means that the variable was valid and reliable. Leadership and compensation styles also simultaneously have a significant effect on employee performance. And the independent variable that has the largest beta coefficient is the compensation variable (X2) with a beta coefficient of 0.240.   Keywords : Leadership style, Compesation and Employee Performance

Finite-Sum Smooth Optimization with SARAH

The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finite-sum smooth nonconvex objective function F(w)=1n∑ni=1fi(w) has been proven to be at least Ω(n−−√/ϵ) for n≤O(ϵ−2) where ϵ denotes the attained accuracy E[∥∇F(w~)∥2]≤ϵ for the outputted approximation w~ (Fang et al., 2018). In this paper, we provide a convergence analysis for a slightly modified version of the SARAH algorithm (Nguyen et al., 2017a;b) and achieve total complexity that matches the lower-bound worst case complexity in (Fang et al., 2018) up to a constant factor when n≤O(ϵ−2) for nonconvex problems. For convex optimization, we propose SARAH++ with sublinear convergence for general convex and linear convergence for strongly convex problems; and we provide a practical version for which numerical experiments on various datasets show an improved performance