InformationTheory

InformationTheory.Shannon.Kolmogorov.OmegaNoncomputable

source

Chaitin's constant is not computably approximable #

Cover–Thomas (2nd ed.). Running the self-delimiting machine prefixUniversalEval with a finite budget yields a total function prefixEvaln, primitive recursive in the budget and the program, which is sound and complete for the unbounded machine. Summing 2 ^ (-|p|) over the programs that have already halted gives the stage approximations omegaApprox, which increase to the halting probability Ω: each is a subsum of the halting weight, and every finite subsum is already reached at some stage.

A computable approximation of Ω from above therefore lets one search for a stage whose weight is within 2 ^ (-n) of Ω, and that search is itself computable because every comparison involved is between natural numbers over a common power of two. Past that stage no program of length at most n can halt any more, since its weight would push the halting weight above Ω. So the halting set of the machine would be decidable, which it is not.

Main definitions #

  • IsComputableENNReal — computability of an extended nonnegative real by a computable sequence of dyadic rationals with error 2 ^ (-n).
  • IsFloorComputableENNReal — the dyadic floor variant of the same notion, which implies it.
  • prefixEvaln — the step-bounded evaluator of the self-delimiting machine.
  • omegaApprox — the stage-t approximation to Ω from below, with numerator omegaApproxNum over the denominator 2 ^ t.
  • searchPred — the stage-lookup predicate, phrased over natural numbers.

Main statements #

Implementation notes #

IsComputableENNReal states its two bounds additively, so truncated subtraction of extended reals never occurs. The standard notion of a computable real — a computable sequence of rationals converging to x with error 2 ^ (-n) — can be phrased in Lean, since ℚ is Primcodable through Denumerable ℚ. What is out of reach is a proof that it implies IsComputableENNReal: rounding a rational approximation to a multiple of 2 ^ (-n) needs computable division on ℚ, and Mathlib.Computability carries no Primrec arithmetic on ℚ. The comparison that is formalized is isComputableENNReal_of_floor, the implication from the dyadic floor form; since the floor form implies the additive one, negating the additive form is the stronger statement.

def

InformationTheory.Kolmogorov.IsComputableENNReal

source
(x : ENNReal) :

A computable nonnegative extended real: a computable sequence of dyadic numerators a n, whose value a n / 2 ^ n is within 2 ^ (-n) of x. The two bounds are stated additively, so no truncated subtraction occurs.

The standard notion — a computable sequence of rationals converging to x with error 2 ^ (-n) — yields such a numerator sequence by rounding, but that rounding step is not formalized here, for want of computable arithmetic on ℚ. The additive form is preferred over the dyadic floor form IsFloorComputableENNReal because no uniform transformation turns an approximation sequence into a floor sequence: that would mean deciding, from the approximations alone, which side of a dyadic grid point x falls on. @audit:ok

Equations
Instances For
    Used by
      def

      InformationTheory.Kolmogorov.IsFloorComputableENNReal

      source
      (x : ENNReal) :

      The dyadic floor form of computability: a n * 2 ^ (-n) sits in the cell of width 2 ^ (-n) below x. It implies IsComputableENNReal, as isComputableENNReal_of_floor records; there is no uniform converse, since producing a floor sequence from an approximation sequence would mean deciding which side of a dyadic grid point x lies on. @audit:ok

      Equations
      Instances For
        Used by
          theorem

          InformationTheory.Kolmogorov.isComputableENNReal_of_floor

          source
          Used by
            noncomputable def

            InformationTheory.Kolmogorov.prefixEvaln

            source
            (k : ℕ) (p : List Bool) :

            The step-bounded evaluator: the self-delimiting machine run with budget k, following decodePayload_eq_dispatch with eval replaced by evaln k.

            Equations
            • One or more equations did not get rendered due to their size.
            Instances For
              Used by
                theorem

                InformationTheory.Kolmogorov.prefixEvaln_mono

                source
                {k₁ k₂ : ℕ} {p : List Bool} {x : ℕ} (h : k₁ ≤ k₂) (hx : x ∈ prefixEvaln k₁ p) :
                x ∈ prefixEvaln k₂ p
                Used by
                  theorem

                  InformationTheory.Kolmogorov.prefixEvaln_sound

                  source
                  {k : ℕ} {p : List Bool} {x : ℕ} (hx : x ∈ prefixEvaln k p) :
                  Used by
                    theorem

                    InformationTheory.Kolmogorov.prefixEvaln_complete

                    source
                    {p : List Bool} {x : ℕ} :

                    The bounded evaluator is sound and complete for the unbounded machine: x is an output of prefixUniversalEval p exactly when some finite budget k already produces it.

                    Used by
                      theorem

                      InformationTheory.Kolmogorov.prefixEvaln_dom_iff

                      source
                      Used by
                        theorem

                        InformationTheory.Kolmogorov.prefixEvaln_primrec

                        source
                        :
                        Primrec fun (a : ℕ × List Bool) => prefixEvaln a.1 a.2

                        Running the self-delimiting machine with a finite budget is primitive recursive in the budget and the program jointly.

                        Used by

                          Enumerating bit strings #

                          theorem

                          InformationTheory.Kolmogorov.primrec_list_sum

                          source
                          Used by
                            theorem

                            InformationTheory.Kolmogorov.primrec_list_filter

                            source
                            {α : Type u_1} {β : Type u_2} [Primcodable α] [Primcodable β] {f : α → List β} {p : α → β → Bool} (hf : Primrec f) (hp : Primrec₂ p) :
                            Primrec fun (a : α) => List.filter (p a) (f a)
                            Used by
                              theorem

                              InformationTheory.Kolmogorov.primrec_two_pow

                              source
                              :
                              Primrec fun (n : ℕ) => 2 ^ n
                              Used by
                                def

                                InformationTheory.Kolmogorov.allBitStrings

                                source
                                :

                                All bit strings of length exactly n.

                                Equations
                                Instances For
                                  Used by
                                    theorem

                                    InformationTheory.Kolmogorov.mem_allBitStrings

                                    source
                                    {p : List Bool} {n : ℕ} :
                                    Used by
                                      theorem

                                      InformationTheory.Kolmogorov.allBitStrings_nodup

                                      source
                                      Used by
                                        theorem

                                        InformationTheory.Kolmogorov.allBitStrings_primrec

                                        source
                                        Used by
                                          def

                                          InformationTheory.Kolmogorov.allBitStringsLE

                                          source
                                          (t : ℕ) :

                                          All bit strings of length at most t.

                                          Equations
                                          Instances For
                                            Used by
                                              theorem

                                              InformationTheory.Kolmogorov.mem_allBitStringsLE

                                              source
                                              {p : List Bool} {t : ℕ} :
                                              Used by
                                                theorem

                                                InformationTheory.Kolmogorov.allBitStringsLE_nodup

                                                source
                                                Used by
                                                  theorem

                                                  InformationTheory.Kolmogorov.allBitStringsLE_primrec

                                                  source
                                                  Used by

                                                    Approximating the halting probability from below #

                                                    noncomputable def

                                                    InformationTheory.Kolmogorov.haltingList

                                                    source
                                                    (t : ℕ) :

                                                    The programs of length at most t on which the machine already halts within budget t.

                                                    Equations
                                                    Instances For
                                                      Used by
                                                        theorem

                                                        InformationTheory.Kolmogorov.mem_haltingList

                                                        source
                                                        Used by
                                                          theorem

                                                          InformationTheory.Kolmogorov.haltingList_nodup

                                                          source
                                                          (t : ℕ) :
                                                          Used by
                                                            theorem

                                                            InformationTheory.Kolmogorov.haltingList_primrec

                                                            source
                                                            Used by
                                                              noncomputable def

                                                              InformationTheory.Kolmogorov.omegaApprox

                                                              source
                                                              (t : ℕ) :

                                                              The stage-t approximation to Ω from below: the mass of the programs of length at most t on which the machine halts within budget t.

                                                              Equations
                                                              Instances For
                                                                Used by
                                                                  noncomputable def

                                                                  InformationTheory.Kolmogorov.omegaApproxNum

                                                                  source
                                                                  (t : ℕ) :

                                                                  The numerator of omegaApprox t written over the denominator 2 ^ t.

                                                                  Equations
                                                                  Instances For
                                                                    Used by
                                                                      theorem

                                                                      InformationTheory.Kolmogorov.natCast_two_pow_sub_mul_inv_pow

                                                                      source
                                                                      {ℓ t : ℕ} (h : ℓ ≤ t) :
                                                                      ↑(2 ^ (t - ℓ)) * 2⁻¹ ^ t = 2⁻¹ ^ ℓ
                                                                      Used by
                                                                        theorem

                                                                        InformationTheory.Kolmogorov.omegaApprox_eq_num

                                                                        source
                                                                        (t : ℕ) :
                                                                        Used by
                                                                          theorem

                                                                          InformationTheory.Kolmogorov.omegaApprox_ne_top

                                                                          source
                                                                          Used by
                                                                            theorem

                                                                            InformationTheory.Kolmogorov.omegaApproxNum_computable

                                                                            source

                                                                            The numerators of the lower approximation form a computable sequence, so the stage approximations to Ω are computable over the denominators 2 ^ t.

                                                                            Used by

                                                                              Monotonicity and cofinality of the lower approximation #

                                                                              theorem

                                                                              InformationTheory.Kolmogorov.prefixEvaln_isSome_mono

                                                                              source
                                                                              {k₁ k₂ : ℕ} {p : List Bool} (h : k₁ ≤ k₂) (hp : (prefixEvaln k₁ p).isSome = true) :
                                                                              Used by
                                                                                theorem

                                                                                InformationTheory.Kolmogorov.haltingList_toFinset_subset

                                                                                source
                                                                                {t₁ t₂ : ℕ} (h : t₁ ≤ t₂) :
                                                                                Used by
                                                                                  theorem

                                                                                  InformationTheory.Kolmogorov.omegaApprox_mono

                                                                                  source
                                                                                  Used by
                                                                                    theorem

                                                                                    InformationTheory.Kolmogorov.sum_le_chaitinOmega

                                                                                    source
                                                                                    {s : Finset (List Bool)} (hs : ∀ p ∈ s, (prefixUniversalEval p).Dom) :
                                                                                    ∑ p ∈ s, 2⁻¹ ^ p.length ≤ chaitinOmega
                                                                                    Used by
                                                                                      theorem

                                                                                      InformationTheory.Kolmogorov.omegaApprox_le_chaitinOmega

                                                                                      source
                                                                                      Used by
                                                                                        theorem

                                                                                        InformationTheory.Kolmogorov.exists_sum_le_omegaApprox

                                                                                        source
                                                                                        (s : Finset { p : List Bool // (prefixUniversalEval p).Dom }) :
                                                                                        ∃ (t : ℕ), ∑ q ∈ s, 2⁻¹ ^ (↑q).length ≤ omegaApprox t
                                                                                        Used by
                                                                                          theorem

                                                                                          InformationTheory.Kolmogorov.chaitinOmega_eq_iSup_sum

                                                                                          source
                                                                                          :
                                                                                          chaitinOmega = ⨆ (s : Finset { p : List Bool // (prefixUniversalEval p).Dom }), ∑ q ∈ s, 2⁻¹ ^ (↑q).length
                                                                                          Used by
                                                                                            theorem

                                                                                            InformationTheory.Kolmogorov.exists_omegaApprox_gt

                                                                                            source
                                                                                            {ε : ENNReal} (hε : ε ≠ 0) :
                                                                                            ∃ (t : ℕ), chaitinOmega < omegaApprox t + ε
                                                                                            Used by
                                                                                              theorem

                                                                                              InformationTheory.Kolmogorov.iSup_omegaApprox

                                                                                              source
                                                                                              :

                                                                                              The stage approximations increase to the halting probability: each is a subsum of the halting weight, and every finite subsum is reached at some stage, so their supremum is Ω.

                                                                                              Used by

                                                                                                Locating a stage from a computable approximation #

                                                                                                theorem

                                                                                                InformationTheory.Kolmogorov.inv_two_pow_add_self

                                                                                                source
                                                                                                (k : ℕ) :
                                                                                                2⁻¹ ^ (k + 1) + 2⁻¹ ^ (k + 1) = 2⁻¹ ^ k
                                                                                                Used by
                                                                                                  theorem

                                                                                                  InformationTheory.Kolmogorov.natCast_mul_inv_pow_eq

                                                                                                  source
                                                                                                  (c : ℕ) {d ℓ m : ℕ} (h : ℓ + d = m) :
                                                                                                  ↑c * 2⁻¹ ^ ℓ = ↑(c * 2 ^ d) * 2⁻¹ ^ m
                                                                                                  Used by
                                                                                                    def

                                                                                                    InformationTheory.Kolmogorov.searchPred

                                                                                                    source
                                                                                                    (a : ℕ → ℕ) (n t : ℕ) :

                                                                                                    The search predicate behind the stage lookup, written over the common denominator 2 ^ (n + 2 + t) so that every comparison is between natural numbers: a (n + 2) * 2 ^ (-(n+2)) ≤ omegaApprox t + 2 ^ (-(n+1)).

                                                                                                    Equations
                                                                                                    Instances For
                                                                                                      Used by
                                                                                                        noncomputable instance

                                                                                                        InformationTheory.Kolmogorov.instDecidableSearchPred

                                                                                                        source
                                                                                                        @[implicit_reducible]
                                                                                                        (a : ℕ → ℕ) (n t : ℕ) :
                                                                                                        Equations
                                                                                                        Used by
                                                                                                          theorem

                                                                                                          InformationTheory.Kolmogorov.searchPred_iff

                                                                                                          source
                                                                                                          (a : ℕ → ℕ) (n t : ℕ) :
                                                                                                          searchPred a n t ↔ ↑(a (n + 2)) * 2⁻¹ ^ (n + 2) ≤ omegaApprox t + 2⁻¹ ^ (n + 1)
                                                                                                          Used by
                                                                                                            theorem

                                                                                                            InformationTheory.Kolmogorov.searchPred_computablePred

                                                                                                            source
                                                                                                            {a : ℕ → ℕ} (ha : Computable a) :
                                                                                                            ComputablePred fun (p : ℕ × ℕ) => searchPred a p.1 p.2
                                                                                                            Used by
                                                                                                              theorem

                                                                                                              InformationTheory.Kolmogorov.exists_searchPred

                                                                                                              source
                                                                                                              {a : ℕ → ℕ} (ha : ∀ (n : ℕ), ↑(a n) * 2⁻¹ ^ n ≤ chaitinOmega + 2⁻¹ ^ n) (n : ℕ) :
                                                                                                              ∃ (t : ℕ), searchPred a n t
                                                                                                              Used by
                                                                                                                theorem

                                                                                                                InformationTheory.Kolmogorov.searchTime_computable

                                                                                                                source
                                                                                                                {a : ℕ → ℕ} (ha : Computable a) (hex : ∀ (n : ℕ), ∃ (t : ℕ), searchPred a n t) :
                                                                                                                Computable fun (n : ℕ) => Nat.find ⋯

                                                                                                                The least stage satisfying searchPred is a computable function of the requested precision, because the predicate compares natural numbers over a common power of two.

                                                                                                                Used by
                                                                                                                  theorem

                                                                                                                  InformationTheory.Kolmogorov.chaitinOmega_lt_omegaApprox_find

                                                                                                                  source
                                                                                                                  {a : ℕ → ℕ} (ha : ∀ (n : ℕ), chaitinOmega ≤ ↑(a n) * 2⁻¹ ^ n + 2⁻¹ ^ n) (hex : ∀ (n : ℕ), ∃ (t : ℕ), searchPred a n t) (n : ℕ) :
                                                                                                                  Used by

                                                                                                                    The halting probability is not computably approximable #

                                                                                                                    theorem

                                                                                                                    InformationTheory.Kolmogorov.isComputableENNReal_one

                                                                                                                    source
                                                                                                                    Used by
                                                                                                                      theorem

                                                                                                                      InformationTheory.Kolmogorov.omegaApprox_add_le_chaitinOmega

                                                                                                                      source
                                                                                                                      Used by
                                                                                                                        theorem

                                                                                                                        InformationTheory.Kolmogorov.dom_iff_prefixEvaln_find_isSome

                                                                                                                        source
                                                                                                                        {a : ℕ → ℕ} (ha : ∀ (n : ℕ), chaitinOmega ≤ ↑(a n) * 2⁻¹ ^ n + 2⁻¹ ^ n) (hex : ∀ (n : ℕ), ∃ (t : ℕ), searchPred a n t) (p : List Bool) :
                                                                                                                        Used by
                                                                                                                          theorem

                                                                                                                          InformationTheory.Kolmogorov.chaitinOmega_not_computable

                                                                                                                          source

                                                                                                                          Chaitin's constant is not computably approximable: no computable sequence of dyadic numerators a satisfies |Ω - a n * 2 ^ (-n)| ≤ 2 ^ (-n) for every n.

                                                                                                                          A computable approximation from above, combined with the computable approximation omegaApprox from below, locates a stage past which no program of length at most n can still halt, and hence decides the halting set of the self-delimiting machine, contradicting prefixUniversalEval_dom_not_computablePred.

                                                                                                                          The textbook statement speaks of a computable real in the sense of a computable sequence of rationals. That notion can be phrased in Lean, ℚ being Primcodable, but deriving IsComputableENNReal from it takes a rounding step and hence computable division on ℚ, for which Mathlib has no Primrec arithmetic; the two are therefore not linked here, and the negation is proved for IsComputableENNReal itself. @audit:ok

                                                                                                                          Used by