A Workflow for Statistical Inference in Stochastic Gradient Descent

Stochastic gradient descent (SGD) is an estimation tool for large data employed in machine learning and statistics. Due to the Markovian nature of the SGD process, inference is a challenging problem. An underlying asymptotic normality of the averaged SGD (ASGD) estimator allows for the construction of a batch-means estimator of the asymptotic covariance matrix. Instead of the usual increasing batch-size strategy employed in ASGD, we propose a memory efficient equal batch-size strategy and show that under mild conditions, the estimator is consistent. A key feature of the proposed batching technique is that it allows for bias-correction of the variance, at no cost to memory. Since joint inference for high dimensional problems may be undesirable, we present marginal-friendly simultaneous confidence intervals, and show through an example how covariance estimators of ASGD can be employed in improved predictions.Comment: 43 pages, 8 figure

A Computationally Efficient algorithm to estimate the Parameters of a Two-Dimensional Chirp Model with the product term

Chirp signal models and their generalizations have been used to model many natural and man-made phenomena in signal processing and time series literature. In recent times, several methods have been proposed for parameter estimation of these models. These methods however are either statistically sub-optimal or computationally burdensome, specially for two dimensional (2D) chirp models. In this paper, we consider the problem of parameter estimation of 2D chirp models and propose a computationally efficient estimator and establish asymptotic theoretical properties of the proposed estimators. And the proposed estimators are observed to have the same rates of convergence as the least squares estimators (LSEs). Furthermore, the proposed estimators of chirp rate parameters are shown to be asymptotically optimal. Extensive and detailed numerical simulations are conducted, which support theoretical results of the proposed estimators

On estimating parameters of a multi-component Chirp Model with equal chirp rates

Multi-component chirp signal models with equal chirp rates appear in various radar applications, e.g., synthetic aperture radar, echo signal of a rapid mobile target, etc. Many sub-optimal estimators have been developed for such models, however, these suffer from the problem of either identifiability or error propagation effect. In this paper, we have developed theoretical properties of the least squares estimators (LSEs) of the parameters of multi-component chirp model with equal chirp rates, where the model is contaminated with linear stationary errors. We also propose two computationally efficient estimators as alternative to LSEs, namely sequential combined estimators and sequential plugin estimators. Strong consistency and asymptotic normality of these estimators have been derived. Interestingly, it is observed that sequential combined estimator of the chirp rate parameter is asymptotically efficient. Extensive numerical simulations have been performed, which validate satisfactory computational and theoretical performance of all three estimators. {We have also analysed a simulated radar data with the help of our proposed estimators of multi-component chirp model with equal chirp rates, which performs efficiently in recovery of inverse synthetic aperture radar (ISAR) image of a target from a noisy data