Sign-Full Random Projections

The method of 1-bit ("sign-sign") random projections has been a popular tool for efficient search and machine learning on large datasets. Given two $D$-dim data vectors $u$, $v\in\mathbb{R}^D$, one can generate $x = \sum_{i=1}^D u_i r_i$, and $y = \sum_{i=1}^D v_i r_i$, where $r_i\sim N(0,1)$ iid. The "collision probability" is ${Pr}\left(sgn(x)=sgn(y)\right) = 1-\frac{\cos^{-1}\rho}{\pi}$, where $\rho = \rho(u,v)$ is the cosine similarity. We develop "sign-full" random projections by estimating $\rho$ from (e.g.,) the expectation $E(sgn(x)y)=\sqrt{\frac{2}{\pi}} \rho$, which can be further substantially improved by normalizing $y$. For nonnegative data, we recommend an interesting estimator based on $E\left(y_- 1_{x\geq 0} + y_+ 1_{x<0}\right)$ and its normalized version. The recommended estimator almost matches the accuracy of the (computationally expensive) maximum likelihood estimator. At high similarity ($\rho\rightarrow1$), the asymptotic variance of recommended estimator is only $\frac{4}{3\pi} \approx 0.4$ of the estimator for sign-sign projections. At small $k$ and high similarity, the improvement would be even much more substantial

Robustness of Sketched Linear Classifiers to Adversarial Attacks

Linear classifiers are well-known to be vulnerable to adversarial attacks: they may predict incorrect labels for input data that are adversarially modified with small perturbations. However, this phenomenon has not been properly understood in the context of sketch-based linear classifiers, typically used in memory-constrained paradigms, which rely on random projections of the features for model compression. In this paper, we propose novel Fast-Gradient-Sign Method (FGSM) attacks for sketched classifiers in full, partial, and black-box information settings with regards to their internal parameters. We perform extensive experiments on the MNIST dataset to characterize their robustness as a function of perturbation budget. Our results suggest that, in the full-information setting, these classifiers are less accurate on unaltered input than their uncompressed counterparts but just as susceptible to adversarial attacks. But in more realistic partial and black-box information settings, sketching improves robustness while having lower memory footprint.Peer reviewe

Optical M0bius Strips in Three Dimensional Ellipse Fields: Lines of Linear Polarization

The minor axes of, and the normals to, the polarization ellipses that surround singular lines of linear polarization in three dimensional optical ellipse fields are shown to be organized into Mobius strips and into structures we call rippled rings (r-rings). The Mobius strips have two full twists, and can be either right- or left-handed. The major axes of the surrounding ellipses generate cone-like structures. Three orthogonal projections that give rise to 15 indices are used to characterize the different structures. These indices, if independent, could generate 839,808 geometrically and topologically distinct lines; selection rules are presented that reduce the number of lines to 8,248, some 5,562 of which have been observed in a computer simulation. Statistical probabilities are presented for the most important index combinations in random fields. It is argued that it is presently feasible to perform experimental measurements of the Mobius strips, r-rings, and cones described here theoretically

Optical Mobius Strips in Three Dimensional Ellipse Fields: Lines of Circular Polarization

The major and minor axes of the polarization ellipses that surround singular lines of circular polarization in three dimensional optical ellipse fields are shown to be organized into Mobius strips. These strips can have either one or three half-twists, and can be either right- or left-handed. The normals to the surrounding ellipses generate cone-like structures. Two special projections, one new geometrical, and seven new topological indices are developed to characterize the rather complex structures of the Mobius strips and cones. These eight indices, together with the two well-known indices used until now to characterize singular lines of circular polarization, could, if independent, generate 16,384 geometrically and topologically distinct lines. Geometric constraints and 13 selection rules are discussed that reduce the number of lines to 2,104, some 1,150 of which have been observed in practice; this number of different C lines is ~ 350 times greater than the three types of lines recognized previously. Statistical probabilities are presented for the most important index combinations in random fields. It is argued that it is presently feasible to perform experimental measurements of the Mobius strips and cones described here theoretically

Sign Stable Projections, Sign Cauchy Projections and Chi-Square Kernels

The method of stable random projections is popular for efficiently computing the Lp distances in high dimension (where 0<p<=2), using small space. Because it adopts nonadaptive linear projections, this method is naturally suitable when the data are collected in a dynamic streaming fashion (i.e., turnstile data streams). In this paper, we propose to use only the signs of the projected data and analyze the probability of collision (i.e., when the two signs differ). We derive a bound of the collision probability which is exact when p=2 and becomes less sharp when p moves away from 2. Interestingly, when p=1 (i.e., Cauchy random projections), we show that the probability of collision can be accurately approximated as functions of the chi-square similarity. For example, when the (un-normalized) data are binary, the maximum approximation error of the collision probability is smaller than 0.0192. In text and vision applications, the chi-square similarity is a popular measure for nonnegative data when the features are generated from histograms. Our experiments confirm that the proposed method is promising for large-scale learning applications

Numerical shadow and geometry of quantum states

The totality of normalised density matrices of order N forms a convex set Q_N in R^(N^2-1). Working with the flat geometry induced by the Hilbert-Schmidt distance we consider images of orthogonal projections of Q_N onto a two-plane and show that they are similar to the numerical ranges of matrices of order N. For a matrix A of a order N one defines its numerical shadow as a probability distribution supported on its numerical range W(A), induced by the unitarily invariant Fubini-Study measure on the complex projective manifold CP^(N-1). We define generalized, mixed-states shadows of A and demonstrate their usefulness to analyse the structure of the set of quantum states and unitary dynamics therein.Comment: 19 pages, 5 figure