Estimation of instrinsic dimension via clustering

The problem of estimating the intrinsic dimension of a set of points in high dimensional space is a critical issue for a wide range of disciplines, including genomics, finance, and networking. Current estimation techniques are dependent on either the ambient or intrinsic dimension in terms of computational complexity, which may cause these methods to become intractable for large data sets. In this paper, we present a clustering-based methodology that exploits the inherent self-similarity of data to efficiently estimate the intrinsic dimension of a set of points. When the data satisfies a specified general clustering condition, we prove that the estimated dimension approaches the true Hausdorff dimension. Experiments show that the clustering-based approach allows for more efficient and accurate intrinsic dimension estimation compared with all prior techniques, even when the data does not conform to obvious self-similarity structure. Finally, we present empirical results which show the clustering-based estimation allows for a natural partitioning of the data points that lie on separate manifolds of varying intrinsic dimension

Fractal descriptors based on the probability dimension: a texture analysis and classification approach

In this work, we propose a novel technique for obtaining descriptors of gray-level texture images. The descriptors are provided by applying a multiscale transform to the fractal dimension of the image estimated through the probability (Voss) method. The effectiveness of the descriptors is verified in a classification task using benchmark over texture datasets. The results obtained demonstrate the efficiency of the proposed method as a tool for the description and discrimination of texture images.Comment: 7 pages, 6 figures. arXiv admin note: text overlap with arXiv:1205.282

A fractal dimension for measures via persistent homology

We use persistent homology in order to define a family of fractal dimensions, denoted $\mathrm{dim}_{\mathrm{PH}}^i(\mu)$ for each homological dimension $i\ge 0$, assigned to a probability measure $\mu$ on a metric space. The case of $0$-dimensional homology ($i=0$) relates to work by Michael J Steele (1988) studying the total length of a minimal spanning tree on a random sampling of points. Indeed, if $\mu$ is supported on a compact subset of Euclidean space $\mathbb{R}^m$ for $m\ge2$, then Steele's work implies that $\mathrm{dim}_{\mathrm{PH}}^0(\mu)=m$ if the absolutely continuous part of $\mu$ has positive mass, and otherwise $\mathrm{dim}_{\mathrm{PH}}^0(\mu)<m$. Experiments suggest that similar results may be true for higher-dimensional homology $0<i<m$, though this is an open question. Our fractal dimension is defined by considering a limit, as the number of points $n$ goes to infinity, of the total sum of the $i$-dimensional persistent homology interval lengths for $n$ random points selected from $\mu$ in an i.i.d. fashion. To some measures $\mu,$ we are able to assign a finer invariant, a curve measuring the limiting distribution of persistent homology interval lengths as the number of points goes to infinity. We prove this limiting curve exists in the case of $0$-dimensional homology when $\mu$ is the uniform distribution over the unit interval, and conjecture that it exists when $\mu$ is the rescaled probability measure for a compact set in Euclidean space with positive Lebesgue measure

Entropy computing via integration over fractal measures

We discuss the properties of invariant measures corresponding to iterated function systems (IFSs) with place-dependent probabilities and compute their Renyi entropies, generalized dimensions, and multifractal spectra. It is shown that with certain dynamical systems one can associate the corresponding IFSs in such a way that their generalized entropies are equal. This provides a new method of computing entropy for some classical and quantum dynamical systems. Numerical techniques are based on integration over the fractal measures.Comment: 14 pages in Latex, Revtex + 4 figures in .ps attached (revised version, new title, several changes, to appear in CHAOS

Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms

Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the dataset and the training algorithm must be taken into account in order to obtain meaningful generalization bounds, it is still theoretically not clear which properties of the data and the algorithm determine the generalization performance. In this study, we approach this problem from a dynamical systems theory perspective and represent stochastic optimization algorithms as random iterated function systems (IFS). Well studied in the dynamical systems literature, under mild assumptions, such IFSs can be shown to be ergodic with an invariant measure that is often supported on sets with a fractal structure. As our main contribution, we prove that the generalization error of a stochastic optimization algorithm can be bounded based on the `complexity' of the fractal structure that underlies its invariant measure. Leveraging results from dynamical systems theory, we show that the generalization error can be explicitly linked to the choice of the algorithm (e.g., stochastic gradient descent -- SGD), algorithm hyperparameters (e.g., step-size, batch-size), and the geometry of the problem (e.g., Hessian of the loss). We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden-layered neural networks) and algorithms (e.g., SGD and preconditioned variants), and obtain analytical estimates for our bound.For modern neural networks, we develop an efficient algorithm to compute the developed bound and support our theory with various experiments on neural networks.Comment: 34 pages including Supplement, 4 Figure

Fractal Shapes Generated by Iterated Function Systems

This thesis explores the construction of shapes and, in particular, fractal-type shapes as fixed points of contractive iterated function systems as discussed in Michael Barnsley\u27s 1988 book ``Fractals Everywhere. The purpose of the thesis is to serve as a resource for an undergraduate-level introduction to the beauty and core ideas of fractal geometry, especially with regard to visualizations of basic concepts and algorithms