We present several application of simple topological arguments in problems of
Kolmogorov complexity. Basically we use the standard fact from topology that
the disk is simply connected. It proves to be enough to construct strings with
some nontrivial algorithmic properties.Comment: Extended versio

Romashchenko, Andrei

Shen, Alexander

English

arXiv

We present several application of simple topological arguments in problems of Kolmogorov complexity. Basically we use the standard fact from topology that the disk is simply connected. It proves to be enough to construct strings with some nontrivial algorithmic properties

Alexander Shen

Andrei Romashchenko

Directory of Open Access Journals

E. Formenti (Ed.): AUTOMATA and JAC 2012 conferencesEPTCS 90, 2012, pp. 127–132, doi:10.4204/EPTCS.90.10c© A. Shen and A. RomashchenkoThis work is licensed under theCreative Commons Attribution License.Topological arguments for Kolmogorov complexity∗Alexander Shen Andrei Romashchenkoalexander.shen@lirmm.fr andrei.romashchenko@lirmm.frLIRMM, CNRS & Montpellier II, on leave from IITP, MoscowWe present several application of simple topological arguments in problems of Kolmogorov com-plexity. Basically we use the standard fact from topology that the disk is simply connected. It provesto be enough to construct strings with some nontrivial algorithmic properties.1 IntroductionIn this paper we show that a very simple and intuitive topological technique can be surprisingly effectivein algorithmic information theory. We present several examples of “topological” proofs of existence ofstrings with some nontrivial algorithmic properties. We focus on technical aspects of the arguments,assuming that the reader is familiar with basic notions of algorithmic information theory such as Kol-mogorov complexity and the mutual information (see the classic textbook [1]).Let us start with an example. Consider a string x that has complexity n (we consider plain complexityC(x), but this does not matter for now). We want to find a string y such that C(x|y) ≈ n/2. This can bedone as follows: consider the shortest description p of x; it has length n; let y be the first half of thisdescription. Then it is easy to check that C(x|y) = n/2+O(logn).However, if we want C(x|y) to be close to n/2 with (maximal possible) precision O(1), one needsa different argument. Let us start with y = x (when C(x|y) ≈ 0) and delete bits (say, at the end) oneby one until we get y = Λ (the empty string) when C(x|y) ≈ n). When the last bit of y is deleted, theconditional complexity C(x|y) changes by at most O(1), so it cannot cross the threshold n/2 withoutvisiting O(1)-neighborhood of n/2.Topological arguments of this type can be used in two (and more) dimensions, though they becomea bit less trivial. We provide several examples of this type in the following sections.2 Vyugin’s result and its extensionsM. Vyugin [2] has shown that for every n and for every string x with C(x) ≥ 2n+O(1) one can find astring y such that both conditional complexities C(x|y) and C(y|x) are equal to n+O(1). This is provedwith a (rather ingenious) game argument. As we shall see in this paper, the condition C(x) ≥ 2n isstronger than necessary; it is enough to assume that C(x) ≥ n+ c logn for some c. This can be shown(unless C(x) is not very large) by a simple topological argument. (The game argument still seems to benecessary if C(x) is really big compared to n.)Similar reasoning allows us to construct y such that both complexities C(x|y) and C(y|x) has prescibedvalues with O(1)-precision, even if those values are different. (This question was discussed in Vyu-gin’s paper [2], but no positive result of this type is given there except for the already mentioned casem = n+O(1).) Again we need some restrictions that guarantee that prescribed values are not unreason-able large or small. Here is the exact statement.∗This work was supported in part by NAFIT ANR-08-EMER-008-01 grant.128 Topological arguments for Kolmogorov complexityTheorem 1. Let P be some polynomial. There exists a constant c such that for every string x and for allintegers m,n such that• n+ c logn≤ C(x)≤ P(n);• c logn≤ m≤ P(n),there exists a string y such that |C(x|y)−n| ≤ c and |C(y|x)−m| ≤ c.(Since C(x|y) does not exceed C(x), we need C(x) to be greater than n; the “safety margin” of sizeO(logn) is assumed in the statement. We also require m and C(x) to be polynomially bounded.)Proof. Let p be the shortest description of x; it is a string of length C(x). Consider also an incompressiblestring z of length sligthly greater than m, e.g., let |z| and C(z) be equal to 2m+O(1). Moreover, wetake z independent from p, so C(z|p) is also 2m+O(1). (Random z has these properties with positiveprobability; in fact, some other properties of z will be needed, see below.)The string y is then constructed as (the encoding of) a pair (p′,z′) where p′ and z′ are prefixes of pand z (respectively); it remains to decide how long p′ and z′ should be. In other words, we have twoparameters, |p′| and |z′|, and the space of the parameters is a rectangle. Each point in this rectangledetermines y = (p′,z′) and is mapped to the point (C(x|y),C(y|x)) at the right. We need to show thatsome point is mapped into (n,m) with O(1)-precision.|p′||z′|C(x)A BCD2mA′B′C′ D′C(x|y)C(y|x)C(x)2m(n,m)Figure 1: Each pair in the left rectangle determines y = (p′,z′) and is mapped into a pair C(x|y),C(y|x)in the right one.To show this, we note that the mapping is “continuous” in the sense that neighbor points on the leftare mapped into points at distance O(1) on the right1. Indeed, C(u|v) changes only by O(1) if u or vis changed by adding or deleting the last bit. Consider a path A–B–C–D–A that goes counterclockwisearound the rectangle on the left; as we shall see, the image path on the right will go clockwise (withlogarithmic precision) around the rectangle and makes one turn around the point (m,n). Then we cancontinuously transform the path on the left into one point (since rectangle is simply connected); if itsimage on the right never comes close to (m,n), we get a contradiction.Now we look closely at the path around the rectangle and its image. Note that our assumptionsguarantee that logC(x) = Θ(logn), so we write just O(log) having in mind O(logn) or O(logC(x)); notealso that O(logm)≤ O(log).1Technically, Lipschitz property is a better name here. However, we will use discrete versions of topological results aboutcontinuous mappings, so we keep the name.A. Shen and A. Romashchenko 129• Point A: here y = (Λ,Λ), so C(y|x) = O(1) and C(x|y) = C(x) + O(1), so the image is A′ =(C(x),0) with O(1)-precision.• Edge A–B: here y = (p′,Λ). Then C(y|x) = O(log) since C(p|x) = O(logC(x)) [the conditionalcomplexity of a shortest description of x given x is O(logC(x))], and the length of p′ can bedescribed by O(logC(x)) bits, too. And C(x|y) is somewhere between 0 and C(x)+O(1).• Point B: here y = (p,Λ), so C(y|x) = O(logC(x)) and C(x|y) = O(1); the image is B′ = (0,0)with O(log)-precision. So the edge A–B is mapped into a path along A′B′ going from A′ to B′ withO(log)-precision.• Edge B–C: here y = (p,z′), so C(y|x) is somewhere between 0 and 2m+O(log) (recall that thelength of z′ is between 0 and 2m and requires O(logm) = O(log) bits; C(p|x) is also O(log)). Onthe other hand, C(x|y) = O(1), since p determines x.• Point C: here y = (p,z), so C(x|y) = O(1) and C(y|x) = O(log)+C(z|x) = 2m+O(log). So theimage is C′ = (0,2m) with O(log)-precision.• Edge C–D: here y = (p′,z), so C(y|x) = 2m+O(log) and C(x|y) is between 0 and C(x)+O(log).• Point D: here y = (Λ,z), so C(y|x) = C(z|x)+O(1) = 2m+O(1) and C(x|y) = C(x|z) = C(x)+O(log) since x and z have only logarithmic mutual information. So the image of D is D′ =(C(x),2m) with O(log)-precision.• Edge D–A: here y = (Λ,z′), so C(y|x) is between 0 and 2m+O(log), and C(x|y) is O(log) (notethat z′ can have only O(log) bit of additional information compared to z).This analysis shows that the path on the right follows the trajectory A′–B′–C′–D′–A′ with O(log)-precision and therefore turns around the point (m,n) if this point is O(log)-far from the boundary of therectangle, and this is exactly what our assumption guarantees.3 Decreasing complexity by using an oracleLet a and b be two strings. They have some complexities C(a) and C(b). If a third string t is given, wecan consider the conditional complexities C(a|y) and C(b|y) which are (in general) smaller that C(a) andC(b). Now the question: can we describe the pairs (C(a|y),C(b|y)) that can be obtained by choosing anappropriate value of y? We answer this question for the case when a and b have small mutual information,and the answer is simple: we can get an arbitrary pair (α,β ) such that 0≤ α ≤ C(a) and 0≤ β ≤ C(b)and α,β are not too close to the endpoints of the corresponding intervals (the distance is big comparedto the logarithms of complexities and to the mutual information).Theorem 2. For some constant c the following statement holds: for every two strings a,b and for everyintegers α,β such that• α,β ≥ c(logC(a)+ logC(b)+ I(a : b));• α ≤ C(a)− c(logC(a)+ logC(b)+ I(a : b));• β ≤ C(b)− c(logC(a)+ logC(b)+ I(a : b)),there exists a string y such that |C(a|y)−α| ≤ c and |C(b|y)−β | ≤ c.Note that this statement is evidently true if instead of O(1)-precision we are satisfied with a precisionof O(logC(a)+ logC(b)+ I(a : b)). Indeed, we can consider the shortest descriptions p and q for a andb and then let y = (p′,q′) where p′ is p without α last bits, q′ is q without β last bits; the informationdistance between a, p and between b,q is logarithmic, p′ and q′ are independent with our precision, etc.130 Topological arguments for Kolmogorov complexityProof. To get O(1)-precision, we need to combine the construction above with topological argumentssimilar to the proof of Theorem 1. Consider the shortest descriptions p and q for a and b. Then |p|=C(a)and |q|= C(b). For every pair (u,v) of integers such that 0≤ u≤ |p| and 0≤ v≤ |q| considery(u,v) = (p without u last bits, q without v last bits).As we have discussed, C(a|y(u,v)) and C(b|y(u,v)) are close to u and v respectively; the distance isO(logC(a)+ logC(b)+ I(a : b)).In other terms, let us consider the mapping(u,v) 7→ (C(a|y(u,v)),C(b|y(u,v));it is defined on the rectangle [0,C(x)]× [0,C(y)] and is close to the identity mapping, the distance isO(logC(a)+ logC(b)+I(a : b)). This mapping is also continuous in the sense explained above. Now thetopological argument can be used to show that the image O(1)-covers the O(logC(a)+ logC(b)+ I(a :b))-interior of the rectangle.Remark. This argument can be generalized easily to three (or more) dimensions. For example, letus consider three strings a,b,c that are almost independent. In this case we get a mapping of a three-dimensional box to itself which is “continuous” and is close to identity. Then a topological argument(based on the fact that identity mapping of the two-dimensional sphere S2 is not homotopic to the constantmapping) shows that the image of this mapping cover (with O(1)-precision) the interior of the box.4 Combination with Muchnik’s techniqueFor the case when a and b are dependent, the result of Theorem 2 looks rather weak. We can extent thearea of pairs (α,β ) that can be covered, if we combine the topological technique with an argument basedon Muchnik’s theorem on conditional description.Theorem 3. For some constant c the following statement holds: for every two strings a,b and for everyintegers α,β such that• α ≤ C(a)− c logn,• β ≤ C(b)− c logn,• −C(a|b)+ c logn≤ β −α ≤ C(b|a)− c logn,(where n = C(a)+C(b)) there exists a string y such that |C(a|y)−α| ≤ c and |C(b|y)−β | ≤ c.Proof. The proof is based on the following lemma.Lemma 1. For all strings a and b there exist strings a′ and b′ such that• C(a′) = |a′|= C(a) and C(b′) = |b′|= C(b),• C(a′|a) = O(logn) and C(b′|b) = O(logn),• I(a′l : b′m) = max{0, l +m−C(a,b)}+O(logn) for each l = 1, . . . , |a′| and m = 1, . . . , |b′|,where n = C(a)+C(b), and a′l , b′m denote prefixes of string a′ and b′ of length l and m respectively.Proof of lemma: We use the method of conditional descriptions proposed by Andrei Muchnik. First, weapply Theorem 2 from [3] and get a conditional description of a given b and given the empty string.Muchnik’s theorem guarantees that there exists a string a′ such thatA. Shen and A. Romashchenko 131• |a′|= C(a),• C(a′|a) = O(logn),• C(a|a′) = O(logn),• C(a|a′′,b) = O(logn), where a′′ is the prefix of a′ of length C(a|b).Then, we apply the same theorem of Muchnik again: we construct a description of b conditional on|a′|+1 strings a′0,a′1, . . . ,a′|a′|. We get b′ such that• |b′|= C(b),• C(b′|b) = O(logn),• C(b|a′l,b′m) = O(logn) for all l ≤ |a′| and m≤ |b′| such that l +m = C(a,b).The constructed a′ and b′ satisfy the theorem.To prove the theorem we need to combine the argument from Theorem 2 and Lemma 1. Let a′ andb′ be the strings from Lemma. For every pair (l,m) of integers such that 0≤ l ≤ |a′| and 0≤m≤ |b′| wedefiney(l,m) = (first l bits of a′, first m bits of b′).Similarly to the proof of Theorem 2, we consider the mapping(l,m) 7→ (C(a|y(l,m)),C(b|y(l,m)),which is defined on the rectangle [0,C(a)]× [0,C(b)].lmC(a)C(a|b)U V WZC(b|a)YC(b)X1X2 X3C(a|y)C(b|y)C(a)C(b)C(b|a)I(a : b)I(a : b)C(a|b)U ′V ′W ′X ′ Y ′Z′(α,β )Figure 2: Each pair of integers on the left determines y = (a′l,b′m), which is mapped to a pair(C(a|y),C(b|y)) on the right.This mapping is “continuous” (neighbor points mapped into points at distance O(1)). Consider thepath U–V –W–X1–X2–Y –Z–U that goes counterclockwise around the pentagon on Fig. 2. We will showthat the image path will go clockwise (with logarithmic precision) around the hexagon U ′–V ′–W ′–X ′–Y ′–Z′–U ′ on Fig. 2. This path makes a turn around the point (α,β ). Hence, point (α,β ) (more precisely,some point in O(1)-neighborhood of (α,β )) has a pre-image (l,m) in the rectangle [0,C(x)]× [0,C(y)].Let us look closely at the path around the pentagon and its image.132 Topological arguments for Kolmogorov complexity• m = 0 and l = 0 . . .C(a|b): the image goes along U ′–V ′ with O(logn)-precision;• m = 0 and l = C(a|b) . . .C(a): the image goes along V ′–W ′ with O(logn)-precision;• l = C(a) and m = 0 . . .C(b|a): the image goes along W ′–X ′ with O(logn)-precision;• l =C(a)−λ and m=C(b|a)+λ for λ = 0 . . . I(a : b): the image remains in O(logn)-neighborhoodof X ′;• m = C(b) and l = C(a|b) . . .0: the image goes along X ′–Y ′ with O(logn)-precision;• l = 0 and m = C(b) . . .C(b|a): the image goes along Y ′–Z′ with O(logn)-precision;• l = 0 and m = C(b|a) . . .0: the image goes along Z′–U ′ with O(logn)-precision;Thus, the path on in the image follows the trajectory U ′–V ′–W ′–X ′–Y ′–Z′–U ′ with O(log)-precision.Therefore turns around the point (α,β ) if this point is O(logn)-far from the boundary of the hexagon,and this is exactly what our assumption guarantees.Remark. Instead of the path U–W–X1–X2–Y –U we could take another path U–W–X3–Y –U . Theshortcut X1–X2 is equivalent to the longer path X1–X3–X2 since all points of the triangle X1X2X3 aremapped into O(logn)-neighborhood of X ′ (if l +m ≥ C(a,b), then y(l,m) contains enough informationto reconstruct both a and b).From Theorem 3 it follows in particular that if C(a|b) and C(b|a) are not logarithmically negligible,one can cut by factor 2 the complexities of a and b while adding an oracle.Acknowledgements. The authors are grateful to Laurent Bienvenu who suggested to write down thissimple argument, Tarik Kaced, and all the colleagues in Escape/NAFIT/Kolmogorov seminar team.References[1] M. Li & P. Vitányi (2008): An introduction to Kolmogorov complexity and its applications. 3rd ed., Springer-Verlag, New York.[2] M. Vyugin (2002): Information distances and conditional complexities. Theoretical Computer Science271(1–2), pp. 145–150 doi:10.1016/S0304-3975(01)00037-8.[3] An. A. Muchnik (2002): Conditional complexity and codes. Theoretical Computer Science 271(1–2), pp.97–109 doi:10.1016/S0304-3975(01)00033-0.

Electronic Proceedings in Theoretical Computer Science

Topological arguments for Kolmogorov complexity

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.404.6560

Topological arguments for Kolmogorov complexity

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals