The complexity measure from Shiner et al. [Physical Review E 59, 1999, 1459-1464] (henceforth abbreviated as SDL-measure) has recently been the subject of a fierce debate. We discuss the properties and shortcomings of this measure, from the point of view of our recently constructed fundamental, statistical mechanics-based measures of complexity Cs(γ,β) [Stoop et al., J. Stat. Phys. 114, 2004, 1127-1137]. We show explicitly, what the shortcomings of the SDL-measure are: It is over-universal, and the implemented temperature dependence is trivial. We also show how the original SDL-approach can be modified to rule out these points of critique. Results of this modification are shown for the logistic parabol

Kern, A.

Steeb, W.-H

Stoop, N.

Stoop, R.

RERO DOC Digital Library

Nonlinear Dynamics (2006) 44: 213–218DOI: 10.1007/s11071-006-1971-z c© Springer 2006Comment on the Shiner–Davison–Landsberg MeasureR. STOOP1,3,∗, A. KERN1, W.-H. STEEB3,∗, and N. STOOP21Institute of Neuroinformatics, Swiss Federal Institute of Technology, ETHZ/UNIZH, Zu¨rich, Switzerland; 2PhysicsDepartment, Swiss Federal Institute of Technology, ETHZ/UNIZH, Zu¨rich, Switzerland; 3International School of ScientificComputing, University of Johannesburg, Johannesburg, South Africa;∗Authors for correspondence (e-mail: Ruedi@ini.phys.ethz.ch; fax: +41-44-6353025 whs@na.rau.ac.za; fax:+27-11-4892646)(Received: 15 September 2004; accepted: 25 March 2005)Abstract. The complexity measure from Shiner et al. [Physical Review E 59, 1999, 1459–1464] (henceforth abbreviated asSDL-measure) has recently been the subject of a fierce debate. We discuss the properties and shortcomings of this measure, fromthe point of view of our recently constructed fundamental, statistical mechanics-based measures of complexity Cs (γ, β) [Stoop etal., J. Stat. Phys. 114, 2004, 1127–1137]. We show explicitly, what the shortcomings of the SDL-measure are: It is over-universal,and the implemented temperature dependence is trivial. We also show how the original SDL-approach can be modified to ruleout these points of critique. Results of this modification are shown for the logistic parabola.Key words: complexity, complexity measures, thermodynamics1. IntroductionThe question of how to measure complexity has continually attracted interest over the last decade, fromthe theoretical and the experimental points of view [1–21]. The Kolmogorov/Solomonoff algorithmicviewpoint [3, 4, 18] has been the most influential concept of complexity. The algorithmic complexity Aof an object s is defined as the length of the shortest program P (in bits) that produces (prints) the object sA(s) = minC,P:C(P)=slog(length(P)), (1)where C is a computer. As there exists a universal computer, called the turing machine, which is ableto simulate any other computer, A(s) is a well-defined quantity.The problem with this measure is that it violates basic conceptions of complexity. For example, randomsequences are assigned the maximal complexity. However, computer-generated random sequences aregenerally the result of a simple random generator, which, obviously, has a finite algorithmic complexity.Moreover, using an intrinsic notion of complexity, truly random sequences appear no more complexthan any pseudo-random sequences, even though the latter have a much shorter description length.In fact, finding the shortest description length is not that easy, as there are at least infinitely manyprograms to be checked. This is why the product of the algorithmic complexity with the difficulty offinding the right program might provide a more adequate approximation to the perceived, human, notionof complexity. Contrasting a plot of a two-dimensional embedded (pseudo-)random generator outputagainst the output of the two-dimensional circle-standard map, illustrates the dilemma [8]: Whereasthe random output is dull and void of structure, the output of the standard map appears as “interesting”and “complex.” The reason for this inappropriateness as a natural perception of complexity, is that214 R. Stoop et al.the algorithmic complexity has been devised as a measure of the complexity of objects generated bycomputers, or computer programs. In this context, the world appears to be rational (in the sense ofrational numbers), allowing only a countable number of states to be distinguished. The real world, ase.g. generated by analog electronic circuits, however, is based on real numbers (even when rationalnumbers are measured), as is any biological or physical system. In fact, Go¨del’s Theorem [22] requiressmall digits in the measurements to be unpredictable per se (and not as a consequence of chaos theory),due to coupling to the rest of the world. Therefore, the need emerges to appropriately define measuresof complexity for natural and physical systems.2. The SDL-Measure Interpreted in Terms of the Fluctuation SpectrumOne recent approach to define complexity therefore builds on the requirement that the measure shouldbe zero for truly random, as well as completely ordered, objects. Moreover, this complexity measureshould be easy to evaluate; it should, in particular, not require hierarchical decomposition of the system.Starting from this position, Shiner et al. [1] defined their complexity measure asα,β = α(1 − )β, (2)where  = S/Smax, with S being the Boltzmann–Gibbs–Shannon entropy. SDL interpreted  as thedisorder, and (1 − ) as the order in the system. The rescaling by Smax maps measured order/disorderinto the unit interval. However, as has been pointed out by several authors [21, 23], the SDL-measurehas some important shortcomings. To elucidate the origin of these, and to point out ways to correctthem, is the main content of our contribution.3. A Recent Thermodynamic Formalism-Based Measure of ComplexityIn order to achieve this, we contrast SDL with our previously proposed measure of complexity [2].The latter is probably the most general statistical mechanics approach to complexity. For an observer-dependent variable ε, the fluctuation entropy spectrum S(ε) is derived, using the thermodynamic for-malism of dynamical systems [12]. This is achieved by a Legendre transform, applied to the free energyassociated with the natural partition sum induced by the temporal evolution of the system. In moredetail, the thermodynamic formalism departs from a partition function Z (n, β, ν), where n is the levelor depth of the partition and β can be viewed as an inverse temperature. With Z (n, β, ν), a free energyF(β) = limn→∞1nlog(Z (n, β, ν)) (3)is associated, where in F(β) we suppressed the dependence on the observable. β can be interpretedas an artificial temperature (that has no absolute zero, though). In the absence of phase transitions, anentropy function is obtained by means of the Legendre transformS(ν) = νβ − F(β). (4)Requirements that apply to entropy functions are strict convexity with infinite derivatives at the twoendpoints of the curve (in the absence of phase transition effects). From the large deviation entropyComment on the Shiner–Davison–Landsberg Measure 215Figure 1. Fluctuation spectrum of different maps and specific entropy measures SI and Smax, respectively. Thick full lines,filled dots: Convex entropy functions S(ε) obtained for two asymmetric tent maps of varying asymmetry. Dashed line, open dot:Numerical approximation of S(ε) obtained for the fully developed parabola (partition level n = 12), which slowly convergestowards the triangular function (thin full lines). In this case, SI and Smax coincide. In the presence of first-order phase transitions,piecewise linear parts of the graph emerge, as is demonstrated by the parabola.S(ε), the complexity measure (for details see [2]) is calculated asCs(γ, β) = ε02β ε1ε1 − κ∫Supp( ˜S)( ˜S(ε˜)/ε˜)γ d ε˜. (5)Here, κ denotes a potential escape rate (which is nonzero only for repellors). The fluctuation spectrumgenerally has the convex form shown in Figure 1 for the asymmetric tent maps. In the presence of first-order phase transitions, straight-line parts emerge, as is shown by the example of the fully developedparabola. For the measure, the graph of S(ε) versus ε has been rescaled by extracting the topologicallength scale ε0 on both axes, which is indicated in (5) by the tildes.In the context of the fluctuation entropy spectrum S(ε), the SDL-complexity measure obtains a simpleinterpretation: S in their formula corresponds to the observable measure SI in the fluctuation spectrum,defined by SI (ε) = ε, whereas Smax corresponds to the topological entropy (the maximum of S(ε)).Their quantity is therefore proportional to the product of SI (Smax − SI ). Geometrically, this measure isrepresented by the gray area of Figure 2.4. Properties of the SDL-MeasureIn particular, by construction the SDL-measure has the following unwanted properties: The measureis calculated from only two particularly significant points of the fluctuation spectrum: The naturalmeasure SI and the topological measure Smax (the latter sometimes also referred to as the balancedmeasure). The remaining shape of the spectrum S(ε) remains without influence. This leads to anover-universality in the following sense: Dynamical systems that have entirely different propertiesmay be attributed identical complexities. This feature is most dramatically illustrated by hyperbolicmaps versus maps displaying phase-transition phenomena; in particular, intermittent maps that can216 R. Stoop et al.Figure 2. Geometric meaning of the SDL-complexity measure: The gray area has the size of SI (Smax − SI ). For the fullydeveloped parabola (see Figure 1), area zero would be obtained.be made to have identical values of SI and Smax, and hence complexity. Obviously, however, theintermittent maps are much more difficult to predict than hyperbolic ones, yet they are mapped on theidentical family of complexity measures α,β . Therefore, the detailed behavior of the system cannot berecovered by this approach, and using the exponents α, β is not a remedy for this situation. Instead, theinformation contained in the two points of the spectrum is re-distributed in a nontrivial way over the realaxis.5. Modification of the SDL-MeasureIn order to obtain better measures based on the order–disorder approach, the trivial temperature de-pendence of the SDL-measure has to be replaced by a dependence that takes into account the wholespectrum. This can be achieved in the following way. First, the Stoop-complexity integrand( ˜S(ε˜)/ε˜)γ (6)may be viewed as being related to complexities based on a SDL-like product integrand of the form(1 − S(ε)/ε)α(S(ε)/ε)β. (7)To achieve a more appropriate measure, we propose to integrate this function over all possible lengthscales. The result of this calculation for the logistic map is shown in Figure 3 for α = β = 1.Note, however, that for this measure, the influence of the intermittent length scales will be reducedin comparison to our original complexity measure (5), something that we would prefer to avoid (forarguments, see [2]). As a more promising generalization of our measure, we could imagine consideringindependent exponentiation to the denominator of the integrand instead.Comment on the Shiner–Davison–Landsberg Measure 217Figure 3. Modified SDL-complexity CSh and Lyapunov exponent λ for the logistic map, as a function of the order parameter a.6. ResultsThis modification removes the criticized over-universality. A comparison of the SDL-complexity withthe one obtained in the proposed way clearly elucidates notable differences between the two results(compare our Figure 3 with Figure 3 of Shiner et al. [1]): Whereas for the original measure we obtainan overall monotonously decreasing function, this is no longer the case with the modified measure. Fur-thermore, the original complexity measure yields nonzero complexity for completely ordered systems(period-doubling cascade cases or period-3 window), which seems counter-intuitive to us. The resultsobtained by the modified measure do not share this defect. They appear to fulfill the requirements askedfor in the definition of a measure of complexity [1] much better: that it should be zero when either thesystem is completely ordered or random.Our proposed approach is very general. In addition to one-dimensional systems, the evaluation canbe performed for higher-dimensional systems as encountered in the description of electronic systems(usually dimensions 2–4), as well as for their time series. In the latter case, however, the approach requiressubstantially more efforts and care. It is hoped that by the evaluation of complexities along computationalpathways, insight could be gained into the deeper nature of natural or artificial computation.7. ConclusionsIn this way, the SDL-measure can be modified to invalidate the most pertinent critiques. The nontrivialtemperature dependence will remove the over-universality. In particular, the requirement by Binder andPerry [23], that at least some classes of systems known from hierarchical analysis should be discernible,is satisfied: Different dynamical systems will have distinct complexity measure families. In the commentby Crutchfield et al. [21], the authors criticized that any measure of complexity must be tied intrinsicallyto a process. In our modification, this is now indeed the case. Starting from the fundamental observationsof complexity based on order and disorder, we have arrived at a measure that is no longer subject to themost pertinent critiques, and whose construction is entirely transparent. However, it remains to be seenwhether a deeper significance can be attributed to the integrand of Equation (7), and, connected to thisquestion, how useful the measure could be for practical applications.218 R. Stoop et al.AcknowledgementsR.S. acknowledges stimulating discussions with J.S. Shiner, and partial support by the SNF.References1. Shiner, J. S., Davison, M., and Landsberg, P. T., ‘Simple measure for complexity’, Physical Review E 59, 1999, 1459–1464.2. Stoop, R., Stoop, N., and Bunimovich, L. A., ‘Complexity of dynamics as variability of prediction’, J. Stat. Phys. 114, 2004,1127–1137.3. Shannon, C. E., ‘A mathematical theory of communication’, Bell Syst. Technol. J. 27, 1948, 379–423.4. Solomonoff, R. J., ‘A preliminary report on a general theory of inductive inference’, Rep. ZTB-138, ZATOR Co., Cambridge,MA, 1960.5. Kolmogorov, A. N., ‘Three approaches to the quantitative definition of information’, Probl. Inf. Transm. Theory (1), 1965,1–7.6. Chaitin, G., ‘On the length of programs for computing finite binary sequences’, J. ACM 13, 1966, 547–569.7. Lempel, A. and Ziv, J., ‘On the complexity of finite sequences’, IEEE Trans. Inform. Theory 22, 1976, 75–81.8. Grassberger, P., ‘Towards a quantitative theory of self-generated complexity’, Int. J. Theor. Phys. 25, 1986, 907–938.9. Lloyd, S. and Pagels, H., ‘Complexity as thermodynamic depth’, Ann. Phys. (N.Y.) 188, 1988, 186–213.10. Bennett, C. H., Complexity, Entropy and the Physics of Information, W. H. Zurek (ed.), Addison-Wesley, Reading MA, 1990.11. McShea, D. W., ‘Complexity and evolution: What everybody knows’, Biol. Philos. 6, 1991, 303–324.12. Peinke, J., Parisi, J., Roessler, O. E., and Stoop, R., Encounter with Chaos, Springer-Verlag, Berlin, 1992.13. Bates, J. E. and Shepard, H. K., ‘Measuring complexity using information fluctuation’, Phys. Lett. A 172, 1993, 416–425.14. Atmanspacher, H., ‘Complexity and meaning as a bridge across the cartesian cut’, J. Conscious. Stud. 1, 1994, 168–181.15. Wackerbauer, R., Witt, A., Atmanspacher, H., Kurths, J., and Scheingraber, H., ‘Quantification of structural and dynamicalcomplexity’, Chaos Solitons Fractals 4, 1994, 133–173.16. Atmanspacher, H., Ra¨th, C., and Wiedenmann, C., ‘Statistics and meta-statistics in the concept of complexity’, Physica A234, 1997, 819–829.17. Steeb, W.-H., Solms, F., Shi, T. K., and Stoop, R., ‘Cubic map, complexity and Ljapunov exponent’, Phys. Scr. 55, 1997,520–522.18. Li, M. and Vita´nyi, M., An Introduction to Kolmogorov Complexity and its Applications, Springer-Verlag, Berlin, 1997.19. Crutchfield, J. P. and Shalizi, J. P., ‘Thermodynamic depth of causal states: Objective complexity via minimal representations’,Phys. Rev. E 59, 1999, 275–283.20. Landsberg, P. T. and Shiner, J. S., ‘Disorder and complexity in an ideal on-equilibrium fermi gas’, Phys. Lett. A 245, 1998,228–232.21. Crutchfield, J. P., Feldman, D. P., and Shalizi, C. R., ‘Comment I on simple measure for complexity’, Phys. Rev. E 62, 2000,2996–2997.22. Go¨del, K., Philosophy of Mathematics, Benacerraf, P. and Putnam, H. (eds.), Prentice-Hall, Englewood Cliffs, NJ, 1964.23. Binder, P.-M. and Perry, N., ‘Comment II on simple measure for complexity’, Phys. Rev. E 62, 2000, 2998–2999.

Comment on the Shiner-Davison-Landsberg Measure

http://doc.rero.ch/record/315088/files/11071_2006_Article_1971.pdf

Comment on the Shiner-Davison-Landsberg Measure

Abstract

Similar works

Full text

Available Versions

RERO DOC Digital Library