Randomized algorithms, such as randomized sketching or projections, are a
promising approach to ease the computational burden in analyzing large
datasets. However, randomized algorithms also produce non-deterministic
outputs, leading to the problem of evaluating their accuracy. In this paper, we
develop a statistical inference framework for quantifying the uncertainty of
the outputs of randomized algorithms. We develop appropriate statistical
methods -- sub-randomization, multi-run plug-in and multi-run aggregation
inference -- by using multiple runs of the same randomized algorithm, or by
estimating the unknown parameters of the limiting distribution. As an example,
we develop methods for statistical inference for least squares parameters via
random sketching using matrices with i.i.d.entries, or uniform partial
orthogonal matrices. For this, we characterize the limiting distribution of
estimators obtained via sketch-and-solve as well as partial sketching methods.
The analysis of i.i.d. sketches uses a trigonometric interpolation argument to
establish a differential equation for the limiting expected characteristic
function and find the dependence on the kurtosis of the entries of the
sketching matrix. The results are supported via a broad range of simulations