Quantitative Coding and Complexity Theory of Compact Metric Spaces

Specifying a computational problem requires fixing encodings for input and output: encoding graphs as adjacency matrices, characters as integers, integers as bit strings, and vice versa. For such discrete data, the actual encoding is usually straightforward and/or complexity-theoretically inessential (up to polynomial time, say); but concerning continuous data, already real numbers naturally suggest various encodings with very different computational properties. With respect to qualitative computability, Kreitz and Weihrauch (1985) had identified ADMISSIBILITY as crucial property for 'reasonable' encodings over the Cantor space of infinite binary sequences, so-called representations [doi:10.1007/11780342_48]: For (precisely) these does the sometimes so-called MAIN THEOREM apply, characterizing continuity of functions in terms of continuous realizers. We rephrase qualitative admissibility as continuity of both the representation and its multivalued inverse, adopting from [doi:10.4115/jla.2013.5.7] a notion of sequential continuity for multifunctions. This suggests its quantitative refinement as criterion for representations suitable for complexity investigations. Higher-type complexity is captured by replacing Cantor's as ground space with Baire or any other (compact) ULTRAmetric space: a quantitative counterpart to equilogical spaces in computability [doi:10.1016/j.tcs.2003.11.012]

Global and local Complexity in weakly chaotic dynamical systems

In a topological dynamical system the complexity of an orbit is a measure of the amount of information (algorithmic information content) that is necessary to describe the orbit. This indicator is invariant up to topological conjugation. We consider this indicator of local complexity of the dynamics and provide different examples of its behavior, showing how it can be useful to characterize various kind of weakly chaotic dynamics. We also provide criteria to find systems with non trivial orbit complexity (systems where the description of the whole orbit requires an infinite amount of information). We consider also a global indicator of the complexity of the system. This global indicator generalizes the topological entropy, taking into account systems were the number of essentially different orbits increases less than exponentially. Then we prove that if the system is constructive (roughly speaking: if the map can be defined up to any given accuracy using a finite amount of information) the orbit complexity is everywhere less or equal than the generalized topological entropy. Conversely there are compact non constructive examples where the inequality is reversed, suggesting that this notion comes out naturally in this kind of complexity questions.Comment: 23 page

Complexity of distances: Theory of generalized analytic equivalence relations

We generalize the notion of analytic/Borel equivalence relations, orbit equivalence relations, and Borel reductions between them to their continuous and quantitative counterparts: analytic/Borel pseudometrics, orbit pseudometrics, and Borel reductions between them. We motivate these concepts on examples and we set some basic general theory. We illustrate the new notion of reduction by showing that the Gromov-Hausdorff distance maintains the same complexity if it is defined on the class of all Polish metric spaces, spaces bounded from below, from above, and from both below and above. Then we show that $E_1$ is not reducible to equivalences induced by orbit pseudometrics, generalizing the seminal result of Kechris and Louveau. We answer in negative a question of Ben-Yaacov, Doucha, Nies, and Tsankov on whether balls in the Gromov-Hausdorff and Kadets distances are Borel. In appendix, we provide new methods using games showing that the distance-zero classes in certain pseudometrics are Borel, extending the results of Ben Yaacov, Doucha, Nies, and Tsankov. There is a complementary paper of the authors where reductions between the most common pseudometrics from functional analysis and metric geometry are provided.Comment: Based on the feedback we received, we decided to split the original version into two parts. The new version is now the first part of this spli

Metric entropy, n-widths, and sampling of functions on manifolds

We first investigate on the asymptotics of the Kolmogorov metric entropy and nonlinear n-widths of approximation spaces on some function classes on manifolds and quasi-metric measure spaces. Secondly, we develop constructive algorithms to represent those functions within a prescribed accuracy. The constructions can be based on either spectral information or scattered samples of the target function. Our algorithmic scheme is asymptotically optimal in the sense of nonlinear n-widths and asymptotically optimal up to a logarithmic factor with respect to the metric entropy

Sample Complexity of Dictionary Learning and other Matrix Factorizations

Many modern tools in machine learning and signal processing, such as sparse dictionary learning, principal component analysis (PCA), non-negative matrix factorization (NMF), $K$-means clustering, etc., rely on the factorization of a matrix obtained by concatenating high-dimensional vectors from a training collection. While the idealized task would be to optimize the expected quality of the factors over the underlying distribution of training vectors, it is achieved in practice by minimizing an empirical average over the considered collection. The focus of this paper is to provide sample complexity estimates to uniformly control how much the empirical average deviates from the expected cost function. Standard arguments imply that the performance of the empirical predictor also exhibit such guarantees. The level of genericity of the approach encompasses several possible constraints on the factors (tensor product structure, shift-invariance, sparsity \ldots), thus providing a unified perspective on the sample complexity of several widely used matrix factorization schemes. The derived generalization bounds behave proportional to $\sqrt{\log(n)/n}$ w.r.t.\ the number of samples $n$ for the considered matrix factorization techniques.Comment: to appea