InformationTheory

InformationTheory.Shannon.AWGN.AchievabilityTypicalDecoder

source

Joint-typicality decoder and the random-coding union bound #

The decoder side of AWGN achievability (Cover–Thomas): the joint-typicality decoder, the measurability plumbing for the per-message error event, and the random-coding union bound culminating in awgn_avg_error_union_bound.

Main definitions #

  • jointTypicalDecoder A codebook — decodes a received vector to the smallest codeword index whose pair lies in the typical set A.

Main statements #

Continuous AEP for the n-dimensional Gaussian #

The continuous AEP is the lemma continuousAepGaussian_holds in InformationTheory/Shannon/AWGN/KLCapacityAndAEP.lean. Consumers in this file call that lemma directly instead of taking a predicate hypothesis.

Joint typical decoder and union bound #

noncomputable def

InformationTheory.Shannon.AWGN.jointTypicalDecoder

source
{n M : } [NeZero M] (A : Set ((Fin n) × (Fin n))) (codebook : Fin MFin n) :
(Fin n)Fin M

The joint-typicality decoder (Cover–Thomas). Given a typical set A ⊆ (Fin n → ℝ) × (Fin n → ℝ) and a candidate codebook, it maps each received vector y to the smallest codeword index m with (codebook m, y) ∈ A; if no such m exists it returns the default ⟨0, …⟩ : Fin M (well-defined under [NeZero M]).

The set A is a parameter so that callers can plug in the AEP-supplied typical set directly.

Equations
Instances For
    Used by
      theorem

      InformationTheory.Shannon.AWGN.jointTypicalDecoder_measurable

      source
      {n M : } [NeZero M] (A : Set ((Fin n) × (Fin n))) (hA : MeasurableSet A) (codebook : Fin MFin n) :

      The joint-typicality decoder is measurable for any measurable typical set A.

      Used by

        Measurability plumbing for the per-message error #

        Private helpers that discharge the AE-measurability of c ↦ (Measure.pi (W ∘ c m)) (errorEvent c m) inside isAwgnTypicalityHypothesis: the joint measurability of the decoder, the codebook kernel, and the kernel-section measurability.

        theorem

        InformationTheory.Shannon.AWGN.jointTypicalDecoder_joint_measurable

        source
        {n M : } [NeZero M] (A : Set ((Fin n) × (Fin n))) (hA : MeasurableSet A) :
        Measurable fun (p : (Fin MFin n) × (Fin n)) => jointTypicalDecoder A p.1 p.2

        Joint measurability in (codebook, y) of jointTypicalDecoder.

        Used by
          noncomputable def

          InformationTheory.Shannon.AWGN.awgnCodebookKernel

          source
          {n M : } (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (m : Fin M) :
          ProbabilityTheory.Kernel (Fin MFin n) (Fin n)

          Bundle c ↦ Measure.pi (fun i => awgnChannel N h_meas (c m i)) as a genuine kernel. Each fibre is a probability measure (so the kernel is Markov, hence s-finite), which lets us feed it to Kernel.measurable_kernel_prodMk_left.

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            Used by
              instance

              InformationTheory.Shannon.AWGN.awgnCodebookKernel.instIsMarkovKernel

              source
              Used by
                theorem

                InformationTheory.Shannon.AWGN.measurable_measurePi_awgnChannel

                source
                {n : } (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) :
                Measurable fun (x : Fin n) => MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (x i)
                Used by
                  theorem

                  InformationTheory.Shannon.AWGN.map_add_prod_pi_gaussianReal_eq_pi_gaussianReal

                  source
                  {n : } (v₁ v₂ : NNReal) :
                  MeasureTheory.Measure.map (fun (p : (Fin n) × (Fin n)) (i : Fin n) => p.1 i + p.2 i) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 v₁).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 v₂)) = MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (v₁ + v₂)
                  Used by
                    theorem

                    InformationTheory.Shannon.AWGN.measurePi_awgnChannel_eq_pi_gaussianReal_map_add

                    source
                    {n : } (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (x : Fin n) :
                    (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (x i)) = MeasureTheory.Measure.map (fun (z : Fin n) (i : Fin n) => x i + z i) (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N)
                    Used by
                      theorem

                      InformationTheory.Shannon.AWGN.lintegral_measurePi_awgnChannel_eq_pi_gaussianReal

                      source
                      {n : } (N v : NNReal) (h_meas : IsAwgnChannelMeasurable N) (B : Set (Fin n)) (hB : MeasurableSet B) :
                      Used by
                        theorem

                        InformationTheory.Shannon.AWGN.awgn_unionBound_trueCodeword_term_le

                        source
                        {n M : } (P : ) (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (m : Fin M) {ε : } ( : 0 < ε) (A : Set ((Fin n) × (Fin n))) (hA_meas : MeasurableSet A) (hA_mass : (MeasureTheory.Measure.map (fun (p : (Fin n) × (Fin n)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) A ENNReal.ofReal (1 - ε)) :
                        ∫⁻ (codebook : Fin MFin n), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) {y : Fin n | (codebook m, y)A} gaussianCodebook M n P.toNNReal ENNReal.ofReal ε
                        Used by
                          theorem

                          InformationTheory.Shannon.AWGN.awgn_unionBound_aliasCodeword_sum_eq

                          source
                          {n M : } [NeZero M] (P : ) (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (m : Fin M) (A : Set ((Fin n) × (Fin n))) (hA_meas : MeasurableSet A) :
                          m'Finset.univ.erase m, ∫⁻ (codebook : Fin MFin n), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) {y : Fin n | (codebook m', y) A} gaussianCodebook M n P.toNNReal = (M - 1) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N))) A
                          Used by
                            theorem

                            InformationTheory.Shannon.AWGN.awgn_unionBound_aliasMass_decay

                            source
                            {n M : } (P : ) (N : NNReal) (hP : 0 < P) (hN : N 0) {ε δ R : } ( : 0 < ε) ( : 0 < δ) (hR_pos : 0 < R) (hslack : R + 3 * δ < 1 / 2 * Real.log (1 + P / N)) (A : Set ((Fin n) × (Fin n))) (hM_le : M Real.exp (n * R)⌉₊) (hg_pos : 0 < 1 / 2 * Real.log (1 + P / N) - R - 3 * δ) (hn : Real.log (2 / ε) / (1 / 2 * Real.log (1 + P / N) - R - 3 * δ)⌉₊ n) (hA_indep : ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N))) A ENNReal.ofReal (Real.exp (-((klDiv (MeasureTheory.Measure.map (fun (p : (Fin n) × (Fin n)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N)))).toReal - n * (3 * δ))))) :
                            Used by

                              Random-coding union bound #

                              theorem

                              InformationTheory.Shannon.AWGN.awgn_random_coding_union_bound

                              source
                              (P : ) (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (hP : 0 < P) (hN : N 0) {ε δ R : } ( : 0 < ε) ( : 0 < δ) (hR_pos : 0 < R) (hslack : R + 3 * δ < 1 / 2 * Real.log (1 + P / N)) :
                              ∃ (N₀ : ), ∀ ⦃n : ⦄, N₀ n∀ ⦃M : ⦄ (hM_pos : 0 < M), M Real.exp (n * R)⌉₊∀ (A : Set ((Fin n) × (Fin n))), MeasurableSet A(MeasureTheory.Measure.map (fun (p : (Fin n) × (Fin n)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) A ENNReal.ofReal (1 - ε)((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N))) A ENNReal.ofReal (Real.exp (-((klDiv (MeasureTheory.Measure.map (fun (p : (Fin n) × (Fin n)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N)))).toReal - n * (3 * δ))))∀ (m : Fin M), ∫⁻ (codebook : Fin MFin n), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) ({ encoder := codebook, decoder := jointTypicalDecoder A codebook }.errorEvent m) gaussianCodebook M n P.toNNReal ENNReal.ofReal (2 * ε)

                              The random-coding union bound (Cover–Thomas, with the typicality slack δ separated from the error target ε). With the codebook drawn from the two-stage Gaussian product law and the decoder fixed to the joint-typicality decoder against A, there is a threshold N₀ such that for every n ≥ N₀, every codebook size M ≤ ⌈exp(nR)⌉, and every measurable typical set A satisfying the two AEP bounds at slack δ, the average per-message error probability is ≤ 2ε:

                              • hA_mass — the joint codebook+noise law puts mass ≥ 1−ε on A.
                              • hA_indep — the independent-pair product law puts mass ≤ exp(−(klDiv_n − 3nδ)) on A.

                              The slack assumption hslack : R + 3δ < (1/2) log(1 + P/N) is what makes the alias term decay: with the typicality margin g = I − R − 3δ > 0 and klDiv_n = n·I, the alias mass is bounded by exp(−ng)·(…) → 0, hence ≤ ε past N₀. The preconditions hP : 0 < P and hN : (N:ℝ) ≠ 0 exclude the degenerate corner 1 + P/N < 0, where P.toNNReal = 0 collapses klDiv to 0 and the alias term no longer decays; under 0 < P and 0 < N we have 1 + P/N > 1 > 0. They are regularity preconditions, not a bundled proof core. @audit:ok

                              Used by
                                theorem

                                InformationTheory.Shannon.AWGN.awgn_avg_error_union_bound

                                source
                                (P : ) (hP : 0 < P) (N : NNReal) (hN : N 0) (h_meas : IsAwgnChannelMeasurable N) {R ε δ : } (hR_pos : 0 < R) ( : 0 < δ) (hslack : R + 3 * δ < 1 / 2 * Real.log (1 + P / N)) ( : 0 < ε) :
                                ∃ (N₀ : ), ∀ (n : ), N₀ n∀ (M : ) (hM_pos : 0 < M), M Real.exp (n * R)⌉₊∃ (A : Set ((Fin n) × (Fin n))), MeasurableSet A ∀ (m : Fin M), ∫⁻ (codebook : Fin MFin n), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) ({ encoder := codebook, decoder := jointTypicalDecoder A codebook }.errorEvent m) gaussianCodebook M n P.toNNReal ENNReal.ofReal (2 * ε)

                                The random-coding union bound closed against the AEP-supplied typical set. Under the random Gaussian codebook and AWGN channel, the average per-message error probability (using jointTypicalDecoder against the AEP-supplied typical set) is ≤ 2ε for all M ≤ ⌈exp(n R)⌉ once n is large enough, given the typicality margin R + 3δ < (1/2) log(1 + P/N) with δ separate from ε.

                                A modular composition of continuousAepGaussian_holds + awgn_random_coding_union_bound; hP/hN are passed through to the union bound at the call site. @audit:ok

                                Used by