InformationTheory

InformationTheory.Shannon.AWGN.AchievabilityTypicalDecoder

source

Joint-typicality decoder and the random-coding union bound #

The decoder side of AWGN achievability (Cover–Thomas): the joint-typicality decoder, the measurability plumbing for the per-message error event, and the random-coding union bound culminating in awgn_avg_error_union_bound.

Main definitions #

  • jointTypicalDecoder A codebook — decodes a received vector to the smallest codeword index whose pair lies in the typical set A.

Main statements #

Continuous AEP for the n-dimensional Gaussian #

The continuous AEP is the lemma continuousAepGaussian_holds in InformationTheory/Shannon/AWGN/KLCapacityAndAEP.lean. Consumers in this file call that lemma directly instead of taking a predicate hypothesis.

Joint typical decoder and union bound #

noncomputable def

InformationTheory.Shannon.AWGN.jointTypicalDecoder

source
{n M : ℕ} [NeZero M] (A : Set ((Fin n → ℝ) × (Fin n → ℝ))) (codebook : Fin M → Fin n → ℝ) :
(Fin n → ℝ) → Fin M

The joint-typicality decoder (Cover–Thomas). Given a typical set A ⊆ (Fin n → ℝ) × (Fin n → ℝ) and a candidate codebook, it maps each received vector y to the smallest codeword index m with (codebook m, y) ∈ A; if no such m exists it returns the default ⟨0, …⟩ : Fin M (well-defined under [NeZero M]).

The set A is a parameter so that callers can plug in the AEP-supplied typical set directly.

Equations
Instances For
    Used by
      theorem

      InformationTheory.Shannon.AWGN.jointTypicalDecoder_measurable

      source
      {n M : ℕ} [NeZero M] (A : Set ((Fin n → ℝ) × (Fin n → ℝ))) (hA : MeasurableSet A) (codebook : Fin M → Fin n → ℝ) :

      The joint-typicality decoder is measurable for any measurable typical set A.

      Used by

        Measurability plumbing for the per-message error #

        Private helpers that discharge the AE-measurability of c ↦ (Measure.pi (W ∘ c m)) (errorEvent c m) inside isAwgnTypicalityHypothesis: the joint measurability of the decoder, the codebook kernel, and the kernel-section measurability.

        theorem

        InformationTheory.Shannon.AWGN.jointTypicalDecoder_joint_measurable

        source
        {n M : ℕ} [NeZero M] (A : Set ((Fin n → ℝ) × (Fin n → ℝ))) (hA : MeasurableSet A) :
        Measurable fun (p : (Fin M → Fin n → ℝ) × (Fin n → ℝ)) => jointTypicalDecoder A p.1 p.2

        Joint measurability in (codebook, y) of jointTypicalDecoder.

        Used by
          noncomputable def

          InformationTheory.Shannon.AWGN.awgnCodebookKernel

          source
          {n M : ℕ} (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (m : Fin M) :
          ProbabilityTheory.Kernel (Fin M → Fin n → ℝ) (Fin n → ℝ)

          Bundle c ↦ Measure.pi (fun i => awgnChannel N h_meas (c m i)) as a genuine kernel. Each fibre is a probability measure (so the kernel is Markov, hence s-finite), which lets us feed it to Kernel.measurable_kernel_prodMk_left.

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            Used by
              instance

              InformationTheory.Shannon.AWGN.awgnCodebookKernel.instIsMarkovKernel

              source
              Used by
                theorem

                InformationTheory.Shannon.AWGN.measurable_measurePi_awgnChannel

                source
                {n : ℕ} (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) :
                Measurable fun (x : Fin n → ℝ) => MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (x i)
                Used by
                  theorem

                  InformationTheory.Shannon.AWGN.map_add_prod_pi_gaussianReal_eq_pi_gaussianReal

                  source
                  {n : ℕ} (v₁ v₂ : NNReal) :
                  MeasureTheory.Measure.map (fun (p : (Fin n → ℝ) × (Fin n → ℝ)) (i : Fin n) => p.1 i + p.2 i) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 v₁).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 v₂)) = MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (v₁ + v₂)
                  Used by
                    theorem

                    InformationTheory.Shannon.AWGN.measurePi_awgnChannel_eq_pi_gaussianReal_map_add

                    source
                    {n : ℕ} (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (x : Fin n → ℝ) :
                    (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (x i)) = MeasureTheory.Measure.map (fun (z : Fin n → ℝ) (i : Fin n) => x i + z i) (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N)
                    Used by
                      theorem

                      InformationTheory.Shannon.AWGN.lintegral_measurePi_awgnChannel_eq_pi_gaussianReal

                      source
                      {n : ℕ} (N v : NNReal) (h_meas : IsAwgnChannelMeasurable N) (B : Set (Fin n → ℝ)) (hB : MeasurableSet B) :
                      Used by
                        theorem

                        InformationTheory.Shannon.AWGN.awgn_unionBound_trueCodeword_term_le

                        source
                        {n M : ℕ} (P : ℝ) (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (m : Fin M) {ε : ℝ} (hε : 0 < ε) (A : Set ((Fin n → ℝ) × (Fin n → ℝ))) (hA_meas : MeasurableSet A) (hA_mass : (MeasureTheory.Measure.map (fun (p : (Fin n → ℝ) × (Fin n → ℝ)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) A ≥ ENNReal.ofReal (1 - ε)) :
                        ∫⁻ (codebook : Fin M → Fin n → ℝ), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) {y : Fin n → ℝ | (codebook m, y) ∉ A} ∂gaussianCodebook M n P.toNNReal ≤ ENNReal.ofReal ε
                        Used by
                          theorem

                          InformationTheory.Shannon.AWGN.awgn_unionBound_aliasCodeword_sum_eq

                          source
                          {n M : ℕ} [NeZero M] (P : ℝ) (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (m : Fin M) (A : Set ((Fin n → ℝ) × (Fin n → ℝ))) (hA_meas : MeasurableSet A) :
                          ∑ m' ∈ Finset.univ.erase m, ∫⁻ (codebook : Fin M → Fin n → ℝ), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) {y : Fin n → ℝ | (codebook m', y) ∈ A} ∂gaussianCodebook M n P.toNNReal = (M - 1) • ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N))) A
                          Used by
                            theorem

                            InformationTheory.Shannon.AWGN.awgn_unionBound_aliasMass_decay

                            source
                            {n M : ℕ} (P : ℝ) (N : NNReal) (hP : 0 < P) (hN : ↑N ≠ 0) {ε δ R : ℝ} (hε : 0 < ε) (hδ : 0 < δ) (hR_pos : 0 < R) (hslack : R + 3 * δ < 1 / 2 * Real.log (1 + P / ↑N)) (A : Set ((Fin n → ℝ) × (Fin n → ℝ))) (hM_le : M ≤ ⌈Real.exp (↑n * R)⌉₊) (hg_pos : 0 < 1 / 2 * Real.log (1 + P / ↑N) - R - 3 * δ) (hn : ⌈Real.log (2 / ε) / (1 / 2 * Real.log (1 + P / ↑N) - R - 3 * δ)⌉₊ ≤ n) (hA_indep : ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N))) A ≤ ENNReal.ofReal (Real.exp (-((klDiv (MeasureTheory.Measure.map (fun (p : (Fin n → ℝ) × (Fin n → ℝ)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N)))).toReal - ↑n * (3 * δ))))) :
                            Used by

                              Random-coding union bound #

                              theorem

                              InformationTheory.Shannon.AWGN.awgn_random_coding_union_bound

                              source
                              (P : ℝ) (N : NNReal) (h_meas : IsAwgnChannelMeasurable N) (hP : 0 < P) (hN : ↑N ≠ 0) {ε δ R : ℝ} (hε : 0 < ε) (hδ : 0 < δ) (hR_pos : 0 < R) (hslack : R + 3 * δ < 1 / 2 * Real.log (1 + P / ↑N)) :
                              ∃ (N₀ : ℕ), ∀ ⦃n : ℕ⦄, N₀ ≤ n → ∀ ⦃M : ℕ⦄ (hM_pos : 0 < M), M ≤ ⌈Real.exp (↑n * R)⌉₊ → ∀ (A : Set ((Fin n → ℝ) × (Fin n → ℝ))), MeasurableSet A → (MeasureTheory.Measure.map (fun (p : (Fin n → ℝ) × (Fin n → ℝ)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) A ≥ ENNReal.ofReal (1 - ε) → ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N))) A ≤ ENNReal.ofReal (Real.exp (-((klDiv (MeasureTheory.Measure.map (fun (p : (Fin n → ℝ) × (Fin n → ℝ)) => (p.1, fun (i : Fin n) => p.1 i + p.2 i)) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 N))) ((MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 P.toNNReal).prod (MeasureTheory.Measure.pi fun (x : Fin n) => ProbabilityTheory.gaussianReal 0 (P.toNNReal + N)))).toReal - ↑n * (3 * δ)))) → ∀ (m : Fin M), ∫⁻ (codebook : Fin M → Fin n → ℝ), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) ({ encoder := codebook, decoder := jointTypicalDecoder A codebook }.errorEvent m) ∂gaussianCodebook M n P.toNNReal ≤ ENNReal.ofReal (2 * ε)

                              The random-coding union bound (Cover–Thomas, with the typicality slack δ separated from the error target ε). With the codebook drawn from the two-stage Gaussian product law and the decoder fixed to the joint-typicality decoder against A, there is a threshold N₀ such that for every n ≥ N₀, every codebook size M ≤ ⌈exp(nR)⌉, and every measurable typical set A satisfying the two AEP bounds at slack δ, the average per-message error probability is ≤ 2ε:

                              • hA_mass — the joint codebook+noise law puts mass ≥ 1−ε on A.
                              • hA_indep — the independent-pair product law puts mass ≤ exp(−(klDiv_n − 3nδ)) on A.

                              The slack assumption hslack : R + 3δ < (1/2) log(1 + P/N) is what makes the alias term decay: with the typicality margin g = I − R − 3δ > 0 and klDiv_n = n·I, the alias mass is bounded by exp(−ng)·(…) → 0, hence ≤ ε past N₀. The preconditions hP : 0 < P and hN : (N:ℝ) ≠ 0 exclude the degenerate corner 1 + P/N < 0, where P.toNNReal = 0 collapses klDiv to 0 and the alias term no longer decays; under 0 < P and 0 < N we have 1 + P/N > 1 > 0. They are regularity preconditions, not a bundled proof core. @audit:ok

                              Used by
                                theorem

                                InformationTheory.Shannon.AWGN.awgn_avg_error_union_bound

                                source
                                (P : ℝ) (hP : 0 < P) (N : NNReal) (hN : ↑N ≠ 0) (h_meas : IsAwgnChannelMeasurable N) {R ε δ : ℝ} (hR_pos : 0 < R) (hδ : 0 < δ) (hslack : R + 3 * δ < 1 / 2 * Real.log (1 + P / ↑N)) (hε : 0 < ε) :
                                ∃ (N₀ : ℕ), ∀ (n : ℕ), N₀ ≤ n → ∀ (M : ℕ) (hM_pos : 0 < M), M ≤ ⌈Real.exp (↑n * R)⌉₊ → ∃ (A : Set ((Fin n → ℝ) × (Fin n → ℝ))), MeasurableSet A ∧ ∀ (m : Fin M), ∫⁻ (codebook : Fin M → Fin n → ℝ), (MeasureTheory.Measure.pi fun (i : Fin n) => (awgnChannel N h_meas) (codebook m i)) ({ encoder := codebook, decoder := jointTypicalDecoder A codebook }.errorEvent m) ∂gaussianCodebook M n P.toNNReal ≤ ENNReal.ofReal (2 * ε)

                                The random-coding union bound closed against the AEP-supplied typical set. Under the random Gaussian codebook and AWGN channel, the average per-message error probability (using jointTypicalDecoder against the AEP-supplied typical set) is ≤ 2ε for all M ≤ ⌈exp(n R)⌉ once n is large enough, given the typicality margin R + 3δ < (1/2) log(1 + P/N) with δ separate from ε.

                                A modular composition of continuousAepGaussian_holds + awgn_random_coding_union_bound; hP/hN are passed through to the union bound at the call site. @audit:ok

                                Used by