Curvature-aided Incremental Aggregated Gradient Method

We propose a new algorithm for finite sum optimization which we call the curvature-aided incremental aggregated gradient (CIAG) method. Motivated by the problem of training a classifier for a d-dimensional problem, where the number of training data is $m$ and $m \gg d \gg 1$, the CIAG method seeks to accelerate incremental aggregated gradient (IAG) methods using aids from the curvature (or Hessian) information, while avoiding the evaluation of matrix inverses required by the incremental Newton (IN) method. Specifically, our idea is to exploit the incrementally aggregated Hessian matrix to trace the full gradient vector at every incremental step, therefore achieving an improved linear convergence rate over the state-of-the-art IAG methods. For strongly convex problems, the fast linear convergence rate requires the objective function to be close to quadratic, or the initial point to be close to optimal solution. Importantly, we show that running one iteration of the CIAG method yields the same improvement to the optimality gap as running one iteration of the full gradient method, while the complexity is $O(d^2)$ for CIAG and $O(md)$ for the full gradient. Overall, the CIAG method strikes a balance between the high computation complexity incremental Newton-type methods and the slow IAG method. Our numerical results support the theoretical findings and show that the CIAG method often converges with much fewer iterations than IAG, and requires much shorter running time than IN when the problem dimension is high.Comment: Final version submitted to Allerton Conference 2017 on Oct 8, 201

Incremental Clustering: The Case for Extra Clusters

The explosion in the amount of data available for analysis often necessitates a transition from batch to incremental clustering methods, which process one element at a time and typically store only a small subset of the data. In this paper, we initiate the formal analysis of incremental clustering methods focusing on the types of cluster structure that they are able to detect. We find that the incremental setting is strictly weaker than the batch model, proving that a fundamental class of cluster structures that can readily be detected in the batch setting is impossible to identify using any incremental method. Furthermore, we show how the limitations of incremental clustering can be overcome by allowing additional clusters

Improved rotor position estimation by signal injection in brushless AC motors, accounting for cross-coupling magnetic saturation

The paper presents an improved signal injection- based sensorless control method for permanent magnet brushless AC (BLAC) motors, accounting for the influence of cross-coupling magnetic saturation between the d- and q-axes. The d- and q-axis incremental self-inductances, the incremental mutual-inductance between the (d-axis and q-axis, and the cross-coupling factor are determined by finite element analysis. A method is also proposed for measuring the cross-coupling factor which can be used directly in the sensorless control scheme. Both measurements and predictions show that a significant improvement in the accuracy of the rotor position estimation can be achieved under both dynamic and steady-state operation, compared with that which is obtained with the conventional signal injection method