Intrinsic Universal Measurements of Non-linear Embeddings

A basic problem in machine learning is to find a mapping $f$ from a low dimensional latent space to a high dimensional observation space. Equipped with the representation power of non-linearity, a learner can easily find a mapping which perfectly fits all the observations. However such a mapping is often not considered as good as it is not simple enough and over-fits. How to define simplicity? This paper tries to make such a formal definition of the amount of information imposed by a non-linear mapping. This definition is based on information geometry and is independent of observations, nor specific parametrizations. We prove these basic properties and discuss relationships with parametric and non-parametric embeddings.Comment: work in progres

Consistent Basis Pursuit for Signal and Matrix Estimates in Quantized Compressed Sensing

This paper focuses on the estimation of low-complexity signals when they are observed through $M$ uniformly quantized compressive observations. Among such signals, we consider 1-D sparse vectors, low-rank matrices, or compressible signals that are well approximated by one of these two models. In this context, we prove the estimation efficiency of a variant of Basis Pursuit Denoise, called Consistent Basis Pursuit (CoBP), enforcing consistency between the observations and the re-observed estimate, while promoting its low-complexity nature. We show that the reconstruction error of CoBP decays like $M^{-1/4}$ when all parameters but $M$ are fixed. Our proof is connected to recent bounds on the proximity of vectors or matrices when (i) those belong to a set of small intrinsic "dimension", as measured by the Gaussian mean width, and (ii) they share the same quantized (dithered) random projections. By solving CoBP with a proximal algorithm, we provide some extensive numerical observations that confirm the theoretical bound as $M$ is increased, displaying even faster error decay than predicted. The same phenomenon is observed in the special, yet important case of 1-bit CS.Comment: Keywords: Quantized compressed sensing, quantization, consistency, error decay, low-rank, sparsity. 10 pages, 3 figures. Note abbout this version: title change, typo corrections, clarification of the context, adding a comparison with BPD

Small Width, Low Distortions: Quantized Random Embeddings of Low-complexity Sets

Under which conditions and with which distortions can we preserve the pairwise-distances of low-complexity vectors, e.g., for structured sets such as the set of sparse vectors or the one of low-rank matrices, when these are mapped in a finite set of vectors? This work addresses this general question through the specific use of a quantized and dithered random linear mapping which combines, in the following order, a sub-Gaussian random projection in $\mathbb R^M$ of vectors in $\mathbb R^N$, a random translation, or "dither", of the projected vectors and a uniform scalar quantizer of resolution $\delta>0$ applied componentwise. Thanks to this quantized mapping we are first able to show that, with high probability, an embedding of a bounded set $\mathcal K \subset \mathbb R^N$ in $\delta \mathbb Z^M$ can be achieved when distances in the quantized and in the original domains are measured with the $\ell_1$- and $\ell_2$-norm, respectively, and provided the number of quantized observations $M$ is large before the square of the "Gaussian mean width" of $\mathcal K$. In this case, we show that the embedding is actually "quasi-isometric" and only suffers of both multiplicative and additive distortions whose magnitudes decrease as $M^{-1/5}$ for general sets, and as $M^{-1/2}$ for structured set, when $M$ increases. Second, when one is only interested in characterizing the maximal distance separating two elements of $\mathcal K$ mapped to the same quantized vector, i.e., the "consistency width" of the mapping, we show that for a similar number of measurements and with high probability this width decays as $M^{-1/4}$ for general sets and as $1/M$ for structured ones when $M$ increases. Finally, as an important aspect of our work, we also establish how the non-Gaussianity of the mapping impacts the class of vectors that can be embedded or whose consistency width provably decays when $M$ increases.Comment: Keywords: quantization, restricted isometry property, compressed sensing, dimensionality reduction. 31 pages, 1 figur

Smart Nanostructures and Synthetic Quantum Systems

So far proposed quantum computers use fragile and environmentally sensitive natural quantum systems. Here we explore the notion that synthetic quantum systems suitable for quantum computation may be fabricated from smart nanostructures using topological excitations of a neural-type network that can mimic natural quantum systems. These developments are a technological application of process physics which is a semantic information theory of reality in which space and quantum phenomena are emergent.Comment: LaTex,14 pages 1 eps file. To be published in BioMEMS and Smart Nanostructures, Proceedings of SPIE Conference #4590, ed. L. B. Kis

Synthetic Quantum Systems

So far proposed quantum computers use fragile and environmentally sensitive natural quantum systems. Here we explore the new notion that synthetic quantum systems suitable for quantum computation may be fabricated from smart nanostructures using topological excitations of a stochastic neural-type network that can mimic natural quantum systems. These developments are a technological application of process physics which is an information theory of reality in which space and quantum phenomena are emergent, and so indicates the deep origins of quantum phenomena. Analogous complex stochastic dynamical systems have recently been proposed within neurobiology to deal with the emergent complexity of biosystems, particularly the biodynamics of higher brain function. The reasons for analogous discoveries in fundamental physics and neurobiology are discussed.Comment: 16 pages, Latex, 1 eps figure fil