InformationTheory

InformationTheory.Shannon.LZ78.AsymptoticOptimality.EncodingLength

source

LZ78 greedy encoding length + Cover-Thomas bit-length bounds (part 1/3) #

§1. Encoding length + parent-theorem bridge #

def

InformationTheory.Shannon.lz78GreedyEncodingLength

source
{α : Type u_1} [Fintype α] [DecidableEq α] (n : ) (x : Fin nα) :

The greedy encoding length of a finite tuple: parse List.ofFn x with the genuine longest-prefix-match greedy parse lz78PhraseStrings, count its c distinct emitted phrases, and charge c · bitLength c |α| bits (each of the c phrases costs at most bitLength c |α| bits, the uniform Cover–Thomas per-phrase cost at the final dictionary size). This plugs into the parent lz78EncodingLength : ∀ n, (Fin n → α) → ℕ parameter of lz78_asymptotic_optimality.

The phrase count c = (lz78PhraseStrings (List.ofFn x)).length is the genuine distinct-phrase count (c ≤ n always, c = O(n / log n) asymptotically via lz78PhraseStrings_count_isBigO), so the per-symbol rate is data-dependent and asymptotically bounded — unlike a one-symbol-per-phrase parse.

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    Used by
      theorem

      InformationTheory.Shannon.lz78GreedyEncodingLength_zero

      source
      @[simp]
      {α : Type u_1} [Fintype α] [DecidableEq α] (x : Fin 0α) :
      Used by
        theorem

        InformationTheory.Shannon.lz78GreedyPhraseCount_ofFn_le

        source
        {α : Type u_1} [Fintype α] [DecidableEq α] (n : ) (x : Fin nα) :

        The longest-prefix greedy parse lz78PhraseStrings of List.ofFn x emits at most n distinct phrases (lz78PhraseStrings_count_le plus List.length_ofFn).

        Used by
          theorem

          InformationTheory.Shannon.lz78_encoding_length_le_n_log_n_plus_const

          source
          {α : Type u_1} [Fintype α] [DecidableEq α] (n : ) (x : Fin nα) :

          The greedy encoding length lz78GreedyEncodingLength n x of x : Fin n → α is bounded by n · (log(n+1) + log|α| + 2), since the parse has c ≤ n distinct phrases, each costing at most bitLength n |α| bits. Combines the distinct-phrase count bound c ≤ n with bitLength-monotonicity in the dictionary size. This is the bit-length upper bound of Cover–Thomas.

          Used by
            theorem

            InformationTheory.Shannon.lz78_encoding_length_per_symbol_le

            source
            {α : Type u_1} [Fintype α] [DecidableEq α] (n : ) (hn : 0 < n) (x : Fin nα) :
            (lz78GreedyEncodingLength n x) / n (Nat.log 2 (n + 1)) + (Nat.log 2 (Fintype.card α)) + 2

            The per-symbol asymptotic bit-rate bound on for the genuine greedy parse: dividing by n gives ≤ log(n+1) + log|α| + 2.

            Used by
              theorem

              InformationTheory.Shannon.lz78_encoding_length_per_symbol_nonneg

              source
              {α : Type u_1} [Fintype α] [DecidableEq α] (n : ) (x : Fin nα) :

              The per-symbol bit-rate is nonnegative: the greedy encoding length divided by n is ≥ 0 for every n (including n = 0, where the division is 0/0 = 0). The numerator is a cast and the denominator a cast, so the quotient is a nonnegative real.

              Used by
                theorem

                InformationTheory.Shannon.natLog_mul_log_two_le

                source
                (m : ) :
                (Nat.log 2 m) * Real.log 2 Real.log m

                (Nat.log 2 m : ℝ) * Real.log 2 ≤ Real.log m: the integer base-2 logarithm bounded by Real.log m / Real.log 2 with the denominator cleared. A real-valued restatement of Real.natLog_le_logb.

                Used by
                  theorem

                  InformationTheory.Shannon.mul_log_succ_le

                  source
                  (c : ) :
                  c * Real.log (c + 1) c * Real.log 2 + c * Real.log c

                  (c : ℝ) * Real.log (c + 1) ≤ (c : ℝ) * Real.log 2 + (c : ℝ) * Real.log c: the per-phrase +1-shift slack, from Real.log (c + 1) ≤ Real.log 2 + Real.log c (via Real.log (c + 1) ≤ Real.log (2 * c) for c ≥ 1; trivial for c = 0).

                  Used by
                    theorem

                    InformationTheory.Shannon.lz78_rate_le_const

                    source
                    {α : Type u_1} [Fintype α] [DecidableEq α] [Nonempty α] (n : ) (x : Fin nα) :
                    (lz78GreedyEncodingLength n x) / n 1 + 8 * Real.log ((Fintype.card α) + 1) / Real.log 2 + ((Nat.log 2 (Fintype.card α)) + 2)

                    The per-symbol greedy bit rate lz78GreedyEncodingLength n x / n is bounded by a deterministic constant depending only on |α|, for every n (including the degenerate n = 0, where the rate is 0). The constant (1 + 8 * Real.log (|α| + 1) / Real.log 2) + (Nat.log 2 |α| + 2) comes from the Ziv product bound c * Real.log c ≤ 8 * Real.log (|α| + 1) * n (lz78PhraseStrings_mul_log_le), c ≤ n, and natLog_mul_log_two_le.

                    Used by
                      theorem

                      InformationTheory.Shannon.lz78_bitrate_le_clogc_plus_overhead

                      source
                      {α : Type u_1} [Fintype α] [DecidableEq α] [Nonempty α] (n : ) (hn : 0 < n) (x : Fin nα) :

                      The per-symbol greedy bit-rate decomposes, deterministically and for each n, into a c·log c term and an o(1) overhead.

                      For 0 < n, writing c = (lz78PhraseStrings (List.ofFn x)).length for the distinct phrase count of the greedy parse, the greedy bit-rate splits as

                      lz/n ≤ (c · log c) / (log 2 · n) + overhead(n, x)
                      

                      where the overhead overhead(n, x) = (c · log 2 + c · (log₂|α| + 2)) / (log 2 · n) collects the +1-shift slack (log(c+1) ≤ log 2 + log c) and the alphabet / parent-index constant cost. The first term is the combinatorial c·log₂c/n that the Ziv comparison connects to blockLogAvg₂ = -log₂Pₙ/n; the overhead is o(1) since c = O(n/log n) (lz78PhraseStrings_count_isBigO).

                      This is the unit-coherent (Nat.log 2 → Real.log / log 2) restatement of the encoding-length expansion inside lz78_rate_le_const; the bit-rate is left exactly as c·log c/(log 2 · n) + overhead, so the dominant term is available for the a.s.-eventual limsup comparison.

                      Used by
                        def

                        InformationTheory.Shannon.IsLZ78EncodingLengthBoundPassthrough

                        source
                        (α : Type u_1) [Fintype α] [DecidableEq α] (B : ) :

                        IsLZ78EncodingLengthBoundPassthrough B — hypothesis pass-through for an upper bound B : ℕ → ℕ on the genuine greedy encoding length lz78GreedyEncodingLength.

                        Equations
                        Instances For
                          Used by
                            theorem

                            InformationTheory.Shannon.isLZ78EncodingLengthBoundPassthrough_def

                            source
                            @[simp]
                            (α : Type u_1) [Fintype α] [DecidableEq α] (B : ) :
                            Used by
                              theorem

                              InformationTheory.Shannon.IsLZ78EncodingLengthBoundPassthrough.canonical

                              source
                              (α : Type u_1) [Fintype α] [DecidableEq α] :

                              The canonical bound n · (log(n+1) + log|α| + 2) satisfies the encoding-length pass-through predicate, supplied by the Cover–Thomas bound lz78_encoding_length_le_n_log_n_plus_const.

                              Used by
                                theorem

                                InformationTheory.Shannon.IsLZ78EncodingLengthBoundPassthrough.mono

                                source
                                (α : Type u_1) [Fintype α] [DecidableEq α] {B₁ B₂ : } (h : IsLZ78EncodingLengthBoundPassthrough α B₁) (hB : ∀ (n : ), B₁ n B₂ n) :

                                Monotonicity of the bound pass-through.

                                Used by