InformationTheory

InformationTheory.Shannon.Kolmogorov.OmegaNoncomputable

source

Chaitin's constant is not computably approximable #

Cover–Thomas (2nd ed.). Running the self-delimiting machine prefixUniversalEval with a finite budget yields a total function prefixEvaln, primitive recursive in the budget and the program, which is sound and complete for the unbounded machine. Summing 2 ^ (-|p|) over the programs that have already halted gives the stage approximations omegaApprox, which increase to the halting probability Ω: each is a subsum of the halting weight, and every finite subsum is already reached at some stage.

A computable approximation of Ω from above therefore lets one search for a stage whose weight is within 2 ^ (-n) of Ω, and that search is itself computable because every comparison involved is between natural numbers over a common power of two. Past that stage no program of length at most n can halt any more, since its weight would push the halting weight above Ω. So the halting set of the machine would be decidable, which it is not.

Main definitions #

  • IsComputableENNReal — computability of an extended nonnegative real by a computable sequence of dyadic rationals with error 2 ^ (-n).
  • IsFloorComputableENNReal — the dyadic floor variant of the same notion, which implies it.
  • prefixEvaln — the step-bounded evaluator of the self-delimiting machine.
  • omegaApprox — the stage-t approximation to Ω from below, with numerator omegaApproxNum over the denominator 2 ^ t.
  • searchPred — the stage-lookup predicate, phrased over natural numbers.

Main statements #

Implementation notes #

IsComputableENNReal states its two bounds additively, so truncated subtraction of extended reals never occurs. The standard notion of a computable real — a computable sequence of rationals converging to x with error 2 ^ (-n) — can be phrased in Lean, since is Primcodable through Denumerable. What is out of reach is a proof that it implies IsComputableENNReal: rounding a rational approximation to a multiple of 2 ^ (-n) needs computable division on , and Mathlib.Computability carries no Primrec arithmetic on . The comparison that is formalized is isComputableENNReal_of_floor, the implication from the dyadic floor form; since the floor form implies the additive one, negating the additive form is the stronger statement.

def

InformationTheory.Kolmogorov.IsComputableENNReal

source
(x : ENNReal) :

A computable nonnegative extended real: a computable sequence of dyadic numerators a n, whose value a n / 2 ^ n is within 2 ^ (-n) of x. The two bounds are stated additively, so no truncated subtraction occurs.

The standard notion — a computable sequence of rationals converging to x with error 2 ^ (-n) — yields such a numerator sequence by rounding, but that rounding step is not formalized here, for want of computable arithmetic on . The additive form is preferred over the dyadic floor form IsFloorComputableENNReal because no uniform transformation turns an approximation sequence into a floor sequence: that would mean deciding, from the approximations alone, which side of a dyadic grid point x falls on. @audit:ok

Equations
Instances For
    Used by
      def

      InformationTheory.Kolmogorov.IsFloorComputableENNReal

      source
      (x : ENNReal) :

      The dyadic floor form of computability: a n * 2 ^ (-n) sits in the cell of width 2 ^ (-n) below x. It implies IsComputableENNReal, as isComputableENNReal_of_floor records; there is no uniform converse, since producing a floor sequence from an approximation sequence would mean deciding which side of a dyadic grid point x lies on. @audit:ok

      Equations
      Instances For
        Used by
          theorem

          InformationTheory.Kolmogorov.isComputableENNReal_of_floor

          source
          Used by
            noncomputable def

            InformationTheory.Kolmogorov.prefixEvaln

            source
            (k : ) (p : List Bool) :

            The step-bounded evaluator: the self-delimiting machine run with budget k, following decodePayload_eq_dispatch with eval replaced by evaln k.

            Equations
            • One or more equations did not get rendered due to their size.
            Instances For
              Used by
                theorem

                InformationTheory.Kolmogorov.prefixEvaln_mono

                source
                {k₁ k₂ : } {p : List Bool} {x : } (h : k₁ k₂) (hx : x prefixEvaln k₁ p) :
                x prefixEvaln k₂ p
                Used by
                  theorem

                  InformationTheory.Kolmogorov.prefixEvaln_sound

                  source
                  {k : } {p : List Bool} {x : } (hx : x prefixEvaln k p) :
                  Used by
                    theorem

                    InformationTheory.Kolmogorov.prefixEvaln_complete

                    source
                    {p : List Bool} {x : } :

                    The bounded evaluator is sound and complete for the unbounded machine: x is an output of prefixUniversalEval p exactly when some finite budget k already produces it.

                    Used by
                      theorem

                      InformationTheory.Kolmogorov.prefixEvaln_dom_iff

                      source
                      Used by
                        theorem

                        InformationTheory.Kolmogorov.prefixEvaln_primrec

                        source
                        :
                        Primrec fun (a : × List Bool) => prefixEvaln a.1 a.2

                        Running the self-delimiting machine with a finite budget is primitive recursive in the budget and the program jointly.

                        Used by

                          Enumerating bit strings #

                          theorem

                          InformationTheory.Kolmogorov.primrec_list_sum

                          source
                          Used by
                            theorem

                            InformationTheory.Kolmogorov.primrec_list_filter

                            source
                            {α : Type u_1} {β : Type u_2} [Primcodable α] [Primcodable β] {f : αList β} {p : αβBool} (hf : Primrec f) (hp : Primrec₂ p) :
                            Primrec fun (a : α) => List.filter (p a) (f a)
                            Used by
                              theorem

                              InformationTheory.Kolmogorov.primrec_two_pow

                              source
                              :
                              Primrec fun (n : ) => 2 ^ n
                              Used by
                                def

                                InformationTheory.Kolmogorov.allBitStrings

                                source
                                :

                                All bit strings of length exactly n.

                                Equations
                                Instances For
                                  Used by
                                    theorem

                                    InformationTheory.Kolmogorov.mem_allBitStrings

                                    source
                                    {p : List Bool} {n : } :
                                    Used by
                                      theorem

                                      InformationTheory.Kolmogorov.allBitStrings_nodup

                                      source
                                      Used by
                                        theorem

                                        InformationTheory.Kolmogorov.allBitStrings_primrec

                                        source
                                        Used by
                                          def

                                          InformationTheory.Kolmogorov.allBitStringsLE

                                          source
                                          (t : ) :

                                          All bit strings of length at most t.

                                          Equations
                                          Instances For
                                            Used by
                                              theorem

                                              InformationTheory.Kolmogorov.mem_allBitStringsLE

                                              source
                                              {p : List Bool} {t : } :
                                              Used by
                                                theorem

                                                InformationTheory.Kolmogorov.allBitStringsLE_nodup

                                                source
                                                Used by
                                                  theorem

                                                  InformationTheory.Kolmogorov.allBitStringsLE_primrec

                                                  source
                                                  Used by

                                                    Approximating the halting probability from below #

                                                    noncomputable def

                                                    InformationTheory.Kolmogorov.haltingList

                                                    source
                                                    (t : ) :

                                                    The programs of length at most t on which the machine already halts within budget t.

                                                    Equations
                                                    Instances For
                                                      Used by
                                                        theorem

                                                        InformationTheory.Kolmogorov.mem_haltingList

                                                        source
                                                        Used by
                                                          theorem

                                                          InformationTheory.Kolmogorov.haltingList_nodup

                                                          source
                                                          (t : ) :
                                                          Used by
                                                            theorem

                                                            InformationTheory.Kolmogorov.haltingList_primrec

                                                            source
                                                            Used by
                                                              noncomputable def

                                                              InformationTheory.Kolmogorov.omegaApprox

                                                              source
                                                              (t : ) :

                                                              The stage-t approximation to Ω from below: the mass of the programs of length at most t on which the machine halts within budget t.

                                                              Equations
                                                              Instances For
                                                                Used by
                                                                  noncomputable def

                                                                  InformationTheory.Kolmogorov.omegaApproxNum

                                                                  source
                                                                  (t : ) :

                                                                  The numerator of omegaApprox t written over the denominator 2 ^ t.

                                                                  Equations
                                                                  Instances For
                                                                    Used by
                                                                      theorem

                                                                      InformationTheory.Kolmogorov.natCast_two_pow_sub_mul_inv_pow

                                                                      source
                                                                      {t : } (h : t) :
                                                                      ↑(2 ^ (t - )) * 2⁻¹ ^ t = 2⁻¹ ^
                                                                      Used by
                                                                        theorem

                                                                        InformationTheory.Kolmogorov.omegaApprox_eq_num

                                                                        source
                                                                        (t : ) :
                                                                        Used by
                                                                          theorem

                                                                          InformationTheory.Kolmogorov.omegaApprox_ne_top

                                                                          source
                                                                          Used by
                                                                            theorem

                                                                            InformationTheory.Kolmogorov.omegaApproxNum_computable

                                                                            source

                                                                            The numerators of the lower approximation form a computable sequence, so the stage approximations to Ω are computable over the denominators 2 ^ t.

                                                                            Used by

                                                                              Monotonicity and cofinality of the lower approximation #

                                                                              theorem

                                                                              InformationTheory.Kolmogorov.prefixEvaln_isSome_mono

                                                                              source
                                                                              {k₁ k₂ : } {p : List Bool} (h : k₁ k₂) (hp : (prefixEvaln k₁ p).isSome = true) :
                                                                              Used by
                                                                                theorem

                                                                                InformationTheory.Kolmogorov.haltingList_toFinset_subset

                                                                                source
                                                                                {t₁ t₂ : } (h : t₁ t₂) :
                                                                                Used by
                                                                                  theorem

                                                                                  InformationTheory.Kolmogorov.omegaApprox_mono

                                                                                  source
                                                                                  Used by
                                                                                    theorem

                                                                                    InformationTheory.Kolmogorov.sum_le_chaitinOmega

                                                                                    source
                                                                                    {s : Finset (List Bool)} (hs : ps, (prefixUniversalEval p).Dom) :
                                                                                    ps, 2⁻¹ ^ p.length chaitinOmega
                                                                                    Used by
                                                                                      theorem

                                                                                      InformationTheory.Kolmogorov.omegaApprox_le_chaitinOmega

                                                                                      source
                                                                                      Used by
                                                                                        theorem

                                                                                        InformationTheory.Kolmogorov.exists_sum_le_omegaApprox

                                                                                        source
                                                                                        (s : Finset { p : List Bool // (prefixUniversalEval p).Dom }) :
                                                                                        ∃ (t : ), qs, 2⁻¹ ^ (↑q).length omegaApprox t
                                                                                        Used by
                                                                                          theorem

                                                                                          InformationTheory.Kolmogorov.chaitinOmega_eq_iSup_sum

                                                                                          source
                                                                                          :
                                                                                          chaitinOmega = ⨆ (s : Finset { p : List Bool // (prefixUniversalEval p).Dom }), qs, 2⁻¹ ^ (↑q).length
                                                                                          Used by
                                                                                            theorem

                                                                                            InformationTheory.Kolmogorov.exists_omegaApprox_gt

                                                                                            source
                                                                                            {ε : ENNReal} ( : ε 0) :
                                                                                            ∃ (t : ), chaitinOmega < omegaApprox t + ε
                                                                                            Used by
                                                                                              theorem

                                                                                              InformationTheory.Kolmogorov.iSup_omegaApprox

                                                                                              source
                                                                                              :

                                                                                              The stage approximations increase to the halting probability: each is a subsum of the halting weight, and every finite subsum is reached at some stage, so their supremum is Ω.

                                                                                              Used by

                                                                                                Locating a stage from a computable approximation #

                                                                                                theorem

                                                                                                InformationTheory.Kolmogorov.inv_two_pow_add_self

                                                                                                source
                                                                                                (k : ) :
                                                                                                2⁻¹ ^ (k + 1) + 2⁻¹ ^ (k + 1) = 2⁻¹ ^ k
                                                                                                Used by
                                                                                                  theorem

                                                                                                  InformationTheory.Kolmogorov.natCast_mul_inv_pow_eq

                                                                                                  source
                                                                                                  (c : ) {d m : } (h : + d = m) :
                                                                                                  c * 2⁻¹ ^ = ↑(c * 2 ^ d) * 2⁻¹ ^ m
                                                                                                  Used by
                                                                                                    def

                                                                                                    InformationTheory.Kolmogorov.searchPred

                                                                                                    source
                                                                                                    (a : ) (n t : ) :

                                                                                                    The search predicate behind the stage lookup, written over the common denominator 2 ^ (n + 2 + t) so that every comparison is between natural numbers: a (n + 2) * 2 ^ (-(n+2)) ≤ omegaApprox t + 2 ^ (-(n+1)).

                                                                                                    Equations
                                                                                                    Instances For
                                                                                                      Used by
                                                                                                        noncomputable instance

                                                                                                        InformationTheory.Kolmogorov.instDecidableSearchPred

                                                                                                        source
                                                                                                        @[implicit_reducible]
                                                                                                        (a : ) (n t : ) :
                                                                                                        Equations
                                                                                                        Used by
                                                                                                          theorem

                                                                                                          InformationTheory.Kolmogorov.searchPred_iff

                                                                                                          source
                                                                                                          (a : ) (n t : ) :
                                                                                                          searchPred a n t (a (n + 2)) * 2⁻¹ ^ (n + 2) omegaApprox t + 2⁻¹ ^ (n + 1)
                                                                                                          Used by
                                                                                                            theorem

                                                                                                            InformationTheory.Kolmogorov.searchPred_computablePred

                                                                                                            source
                                                                                                            {a : } (ha : Computable a) :
                                                                                                            ComputablePred fun (p : × ) => searchPred a p.1 p.2
                                                                                                            Used by
                                                                                                              theorem

                                                                                                              InformationTheory.Kolmogorov.exists_searchPred

                                                                                                              source
                                                                                                              {a : } (ha : ∀ (n : ), (a n) * 2⁻¹ ^ n chaitinOmega + 2⁻¹ ^ n) (n : ) :
                                                                                                              ∃ (t : ), searchPred a n t
                                                                                                              Used by
                                                                                                                theorem

                                                                                                                InformationTheory.Kolmogorov.searchTime_computable

                                                                                                                source
                                                                                                                {a : } (ha : Computable a) (hex : ∀ (n : ), ∃ (t : ), searchPred a n t) :
                                                                                                                Computable fun (n : ) => Nat.find

                                                                                                                The least stage satisfying searchPred is a computable function of the requested precision, because the predicate compares natural numbers over a common power of two.

                                                                                                                Used by
                                                                                                                  theorem

                                                                                                                  InformationTheory.Kolmogorov.chaitinOmega_lt_omegaApprox_find

                                                                                                                  source
                                                                                                                  {a : } (ha : ∀ (n : ), chaitinOmega (a n) * 2⁻¹ ^ n + 2⁻¹ ^ n) (hex : ∀ (n : ), ∃ (t : ), searchPred a n t) (n : ) :
                                                                                                                  Used by

                                                                                                                    The halting probability is not computably approximable #

                                                                                                                    theorem

                                                                                                                    InformationTheory.Kolmogorov.isComputableENNReal_one

                                                                                                                    source
                                                                                                                    Used by
                                                                                                                      theorem

                                                                                                                      InformationTheory.Kolmogorov.omegaApprox_add_le_chaitinOmega

                                                                                                                      source
                                                                                                                      Used by
                                                                                                                        theorem

                                                                                                                        InformationTheory.Kolmogorov.dom_iff_prefixEvaln_find_isSome

                                                                                                                        source
                                                                                                                        {a : } (ha : ∀ (n : ), chaitinOmega (a n) * 2⁻¹ ^ n + 2⁻¹ ^ n) (hex : ∀ (n : ), ∃ (t : ), searchPred a n t) (p : List Bool) :
                                                                                                                        Used by
                                                                                                                          theorem

                                                                                                                          InformationTheory.Kolmogorov.chaitinOmega_not_computable

                                                                                                                          source

                                                                                                                          Chaitin's constant is not computably approximable: no computable sequence of dyadic numerators a satisfies |Ω - a n * 2 ^ (-n)| ≤ 2 ^ (-n) for every n.

                                                                                                                          A computable approximation from above, combined with the computable approximation omegaApprox from below, locates a stage past which no program of length at most n can still halt, and hence decides the halting set of the self-delimiting machine, contradicting prefixUniversalEval_dom_not_computablePred.

                                                                                                                          The textbook statement speaks of a computable real in the sense of a computable sequence of rationals. That notion can be phrased in Lean, being Primcodable, but deriving IsComputableENNReal from it takes a rounding step and hence computable division on , for which Mathlib has no Primrec arithmetic; the two are therefore not linked here, and the negation is proved for IsComputableENNReal itself. @audit:ok

                                                                                                                          Used by