research

Optimal linear estimation under unknown nonlinear transform

Abstract

Linear regression studies the problem of estimating a model parameter Ξ²βˆ—βˆˆRp\beta^* \in \mathbb{R}^p, from nn observations {(yi,xi)}i=1n\{(y_i,\mathbf{x}_i)\}_{i=1}^n from linear model yi=⟨xi,Ξ²βˆ—βŸ©+Ο΅iy_i = \langle \mathbf{x}_i,\beta^* \rangle + \epsilon_i. We consider a significant generalization in which the relationship between ⟨xi,Ξ²βˆ—βŸ©\langle \mathbf{x}_i,\beta^* \rangle and yiy_i is noisy, quantized to a single bit, potentially nonlinear, noninvertible, as well as unknown. This model is known as the single-index model in statistics, and, among other things, it represents a significant generalization of one-bit compressed sensing. We propose a novel spectral-based estimation procedure and show that we can recover Ξ²βˆ—\beta^* in settings (i.e., classes of link function ff) where previous algorithms fail. In general, our algorithm requires only very mild restrictions on the (unknown) functional relationship between yiy_i and ⟨xi,Ξ²βˆ—βŸ©\langle \mathbf{x}_i,\beta^* \rangle. We also consider the high dimensional setting where Ξ²βˆ—\beta^* is sparse ,and introduce a two-stage nonconvex framework that addresses estimation challenges in high dimensional regimes where p≫np \gg n. For a broad class of link functions between ⟨xi,Ξ²βˆ—βŸ©\langle \mathbf{x}_i,\beta^* \rangle and yiy_i, we establish minimax lower bounds that demonstrate the optimality of our estimators in both the classical and high dimensional regimes.Comment: 25 pages, 3 figure

    Similar works