Average Symmetry and Complexity of Binary Sequences

The concept of complexity as average symmetry is here formalised by introducing a general expression dependent on the relevant symmetry and a related discrete set of transformations. This complexity has hybrid features of both statistical complexities and of those related to algorithmic complexity. Like the former, random objects are not the most complex while they still are more complex than the more symmetric ones (as in the latter). By applying this definition to the particular case of rotations of binary sequences, we are able to find a precise expression for it. In particular, we then analyse the behaviour of this measure in different well-known automatic sequences, where we find interesting new properties. A generalisation of the measure to statistical ensembles is also presented and applied to the case of i.i.d. random sequences and to the equilibrium configurations of the one-dimensional Ising model. In both cases, we find that the complexity is continuous and differentiable as a function of the relevant parameters and agrees with the intuitive requirements we were looking for.Comment: 9 pages, 5 figure

Online Learning in Discrete Hidden Markov Models

We present and analyse three online algorithms for learning in discrete Hidden Markov Models (HMMs) and compare them with the Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of generalisation error we draw learning curves in simplified situations. The performance for learning drifting concepts of one of the presented algorithms is analysed and compared with the Baldi-Chauvin algorithm in the same situations. A brief discussion about learning and symmetry breaking based on our results is also presented.Comment: 8 pages, 6 figure

Measuring complexity through average symmetry

This work introduces a complexity measure which addresses some conflicting issues between existing ones by using a new principle - measuring the average amount of symmetry broken by an object. It attributes low (although different) complexity to either deterministic or random homogeneous densities and higher complexity to the intermediate cases. This new measure is easily computable, breaks the coarse graining paradigm and can be straightforwardly generalized, including to continuous cases and general networks. By applying this measure to a series of objects, it is shown that it can be consistently used for both small scale structures with exact symmetry breaking and large scale patterns, for which, differently from similar measures, it consistently discriminates between repetitive patterns, random configurations and self-similar structure

Bayesian online algorithms for learning in discrete Hidden Markov Models

We propose and analyze two different Bayesian online algorithms for learning in discrete Hidden Markov Models and compare their performance with the already known Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of generalization we draw learning curves in simplified situations for these algorithms and compare their performances

Typical kernel size and number of sparse random matrices over Galois fields: a statistical physics approach

Using methods of statistical physics, we study the average number and kernel size of general sparse random matrices over GF(q), with a given connectivity profile, in the thermodynamical limit of large matrices. We introduce a mapping of GF(q) matrices onto spin systems using the representation of the cyclic group of order q as the q-th complex roots of unity. This representation facilitates the derivation of the average kernel size of random matrices using the replica approach, under the replica symmetric ansatz, resulting in saddle point equations for general connectivity distributions. Numerical solutions are then obtained for particular cases by population dynamics. Similar techniques also allow us to obtain an expression for the exact and average number of random matrices for any general connectivity profile. We present numerical results for particular distributions