InformationTheory

InformationTheory.Shannon.WynerZiv.Achievability.Covering

source

Wyner–Ziv achievability — covering + binning construction #

Side-information decoder confusion bound #

Instantiation of the Slepian–Wolf alias bound swError_EX_expectation_le with the covering codeword U in the source (α) role and the side information Y in the β role. The bound is exp(n · (H(U,Y) − H(Y) + 2ε)) / M = exp(n · (H(U|Y) + 2ε)) / M, the confusable-codeword count divided by the bin count.

theorem

InformationTheory.Shannon.wz_sideInfo_decoder_confusion_expectation_le

source
{β : Type u_2} {U : Type u_4} [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [Fintype U] [DecidableEq U] [Nonempty U] [MeasurableSpace U] [MeasurableSingletonClass U] {Ω : Type u_5} [MeasurableSpace Ω] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (Us : ΩU) (Ys : Ωβ) (hUs : ∀ (i : ), Measurable (Us i)) (hYs : ∀ (i : ), Measurable (Ys i)) (hindepY_full : ProbabilityTheory.iIndepFun (fun (i : ) => Ys i) μ) (hidentY : ∀ (i : ), ProbabilityTheory.IdentDistrib (Ys i) (Ys 0) μ μ) (hindepZ_full : ProbabilityTheory.iIndepFun (fun (i : ) => ChannelCoding.jointSequence Us Ys i) μ) (hidentZ : ∀ (i : ), ProbabilityTheory.IdentDistrib (ChannelCoding.jointSequence Us Ys i) (ChannelCoding.jointSequence Us Ys 0) μ μ) (hposY : ∀ (y : β), 0 < (MeasureTheory.Measure.map (Ys 0) μ).real {y}) (hposZ : ∀ (p : U × β), 0 < (MeasureTheory.Measure.map (ChannelCoding.jointSequence Us Ys 0) μ).real {p}) {n M : } [NeZero M] {ε : } ( : 0 < ε) :
(f : (Fin nU)Fin M), μ.real (ChannelCoding.swError_EX μ Us Ys n ε f) binningMeasure U n M Real.exp (n * (entropy μ (ChannelCoding.jointSequence Us Ys 0) - entropy μ (Ys 0) + 2 * ε)) * (↑M)⁻¹

Wyner–Ziv side-information decoder confusion bound. For a random binning f of the covering-codeword space Fin n → U into M bins, the expected μ-probability (over the binning f ∼ binningMeasure U n M) that some codeword u' ≠ U^n that is jointly typical with the received side information Y^n hashes to the same bin as the true codeword U^n is at most exp(n · (H(U|Y) + 2ε)) / M.

This is the decoder-confusion half of Wyner–Ziv achievability. It is the side-information analogue of the Slepian–Wolf alias bound, with the covering codeword U in the source role and the side information Y as the conditioning variable; the proof is a direct instantiation of swError_EX_expectation_le, witnessing that the binning ∘ conditional-typicality composition closes as plumbing over an existing atom. @audit:ok

Used by

    Covering acceptance mass bound #

    Instantiation of the strong conditional-slice mass bound conditionalStronglyTypicalSlice_mass_ge with the same alphabet assignment. For a strongly-typical covering codeword u, the product Y-mass of the fiber of side words jointly (strongly) typical with u is at least exp(−n · (I(U;Y) + slack)). This ensures the true covering codeword is not rejected by the side-information decoder.

    theorem

    InformationTheory.Shannon.wz_covering_sideInfo_mass_ge

    source
    {β : Type u_2} {U : Type u_4} [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [Fintype U] [DecidableEq U] [Nonempty U] [MeasurableSpace U] [MeasurableSingletonClass U] {Ω : Type u_5} [MeasurableSpace Ω] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (Us : ΩU) (Ys : Ωβ) (hUs : ∀ (i : ), Measurable (Us i)) (hYs : ∀ (i : ), Measurable (Ys i)) (hindep_Z_pair : Pairwise fun (i j : ) => ProbabilityTheory.IndepFun (ChannelCoding.jointSequence Us Ys i) (ChannelCoding.jointSequence Us Ys j) μ) (hident_Z : ∀ (i : ), ProbabilityTheory.IdentDistrib (ChannelCoding.jointSequence Us Ys i) (ChannelCoding.jointSequence Us Ys 0) μ μ) (hposZ : ∀ (p : U × β), 0 < (MeasureTheory.Measure.map (ChannelCoding.jointSequence Us Ys 0) μ).real {p}) (hposX : ∀ (a : U), 0 < (MeasureTheory.Measure.map (Us 0) μ).real {a}) (hposY : ∀ (b : β), 0 < (MeasureTheory.Measure.map (Ys 0) μ).real {b}) (hmarg_X : MeasureTheory.Measure.map Prod.fst (MeasureTheory.Measure.map (ChannelCoding.jointSequence Us Ys 0) μ) = MeasureTheory.Measure.map (Us 0) μ) (hmarg_Y : MeasureTheory.Measure.map Prod.snd (MeasureTheory.Measure.map (ChannelCoding.jointSequence Us Ys 0) μ) = MeasureTheory.Measure.map (Ys 0) μ) {ε ε_X δ : } ( : 0 < ε) (hε_X : 0 ε_X) (hε_X_lt_ε : ε_X < ε) ( : 0 < δ) (qZ_min : ) (hqZ_min_pos : 0 < qZ_min) (hqZ_min_le : ∀ (p : U × β), qZ_min (MeasureTheory.Measure.map (ChannelCoding.jointSequence Us Ys 0) μ).real {p}) (hδ_dominates_kl : 8 * (Fintype.card U) * (Fintype.card β) * ε_X ^ 2 δ * qZ_min) :
    ∃ (N : ), ∀ (n : ), N nustronglyTypicalSet μ Us n ε_X, Real.exp (-n * (entropy μ (Us 0) + entropy μ (Ys 0) - entropy μ (ChannelCoding.jointSequence Us Ys 0) + ((Fintype.card U) * ε_X * logSumAbs μ Ys + ε_X * logSumAbs μ Us + ε_X * logSumAbs μ (ChannelCoding.jointSequence Us Ys) + δ))) (MeasureTheory.Measure.pi fun (x : Fin n) => MeasureTheory.Measure.map (Ys 0) μ).real (conditionalStronglyTypicalSlice μ Us Ys n ε u)

    Wyner–Ziv covering acceptance mass bound. For a strongly-typical covering codeword u : Fin n → U, the product Y-mass of the fiber of side words jointly strongly typical with u is bounded below by exp(−n · (H(U) + H(Y) − H(U,Y) + slack)), i.e. exp(−n · (I(U;Y) + slack)). This is the covering-acceptance half of Wyner–Ziv achievability: the correct covering codeword is conditionally typical with the side information with high probability. Direct instantiation of conditionalStronglyTypicalSlice_mass_ge. @audit:ok

    Used by

      Rate non-negativity leaf (data-processing) #

      The reshaped Wyner–Ziv rate is non-negative: every factorizable feasible objective I(X;U) − I(Y;U) is ≥ 0 by the data-processing inequality for the Markov chain U − X − Y (wzObjective_nonneg_of_factorizable), so its infimum over the non-degenerate value set is ≥ 0. Combined with h_rate, this pins 0 < R, which is exactly what the codebook-rate tendsto codebookSize_log_div_tendsto needs.

      theorem

      InformationTheory.Shannon.wynerZivRate_nonneg

      source
      {α : Type u_1} {β : Type u_2} {γ : Type u_3} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [Fintype γ] [DecidableEq γ] [Nonempty γ] [MeasurableSpace γ] [MeasurableSingletonClass γ] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] (d : DistortionFn α γ) (D : ) :
      0 wynerZivRate (fun (p : α × β) => P_XY.real {p}) (fun (a : α) (b : γ) => (d a b)) D

      The reshaped Wyner–Ziv rate for a probability-measure source is ≥ 0. Via Real.sInf_nonneg, every value of the rate set is the objective of a feasible factorizable point, which is ≥ 0 by the data-processing inequality wzObjective_nonneg_of_factorizable; a feasible factorizable point forces k > 0 (a Fin 0 kernel has row-sum ∑_{u:Fin 0} κ x u = 0 ≠ 1). The bound holds even in the empty-feasible-set regime (0 ≤ sInf ∅ = 0), where Real.sInf_nonneg's premise is vacuously satisfied. @audit:ok

      Used by

        Covering + binning construction #

        The centerpiece of Wyner–Ziv achievability: from a feasible test channel below the rate R, build a sequence of Wyner–Ziv block codes with codebookSize R n = ⌈exp(n R)⌉ messages whose expected block distortion is eventually within D + ε.

        The construction is the two-layer hybrid (rate-distortion covering on the X → U side, Slepian–Wolf binning on the side-information Y side) whose two error mechanisms are the atoms wz_sideInfo_decoder_confusion_expectation_le and wz_covering_sideInfo_mass_ge, with a good codebook extracted by the pigeonhole averaging exists_codebook_low_avg.

        theorem

        InformationTheory.Shannon.wz_testChannel_of_rate_lt

        source
        {α : Type u_1} {β : Type u_2} {γ : Type u_3} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [Fintype γ] [DecidableEq γ] [Nonempty γ] [MeasurableSpace γ] [MeasurableSingletonClass γ] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] (d : DistortionFn α γ) (R D : ) (h_ne : (wzRateValueSet (fun (p : α × β) => P_XY.real {p}) (fun (a : α) (b : γ) => (d a b)) D).Nonempty) (h_rate : wynerZivRate (fun (p : α × β) => P_XY.real {p}) (fun (a : α) (b : γ) => (d a b)) D < R) :
        ∃ (k : ), qfWynerZivFactorizableConstraint (Fin k) (fun (p : α × β) => P_XY.real {p}) (fun (a : α) (b : γ) => (d a b)) D, wzMutualInfoXU (Fin k) qf.1 - wzMutualInfoYU (Fin k) qf.1 < R

        Witness extraction. From the feasibility guard h_ne and the rate strict inequality h_rate, extract a concrete finite auxiliary alphabet Fin k, a factorizable test channel qf feasible at distortion D, whose Wyner–Ziv objective I(X;U) − I(Y;U) is strictly below R.

        This is exists_lt_of_csInf_lt on the infimum-of-values definition of wynerZivRate (= sInf (wzRateValueSet …)), with the resulting value unpacked by mem_wzRateValueSet_iff into a feasible factorizable point. exists_lt_of_csInf_lt requires h_ne (value set nonempty), so sInf < R is realized by an actual value; no vacuous sInf ∅ shortcut. @audit:ok

        Used by

          Leaf atoms for the covering + binning construction #

          The following helper lemmas are the small, fully-proved atoms that the heavy covering+binning core (wz_goodCode_exists_of_testChannel) consumes: a Nonempty (Fin k) extractor from feasibility, a full-support kernel perturbation, and a public exp(n c)/codebookSize R n → 0 decay adapter (re-proved locally because the Slepian–Wolf original is private).

          theorem

          InformationTheory.Shannon.wz_nonempty_of_factorizable

          source
          {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] {P : α × β} {k : } {q : α × β × Fin k} (hfact : IsWynerZivFactorizable (Fin k) P q) :

          Nonempty auxiliary alphabet. A Wyner–Ziv factorizable joint over a source pmf on α × β forces a nonempty covering alphabet Fin k: the row-stochastic kernel condition ∑_{u : Fin k} κ x u = 1 is impossible for k = 0 (the empty sum is 0 ≠ 1), using Nonempty α to pick a row x.

          Used by
            theorem

            InformationTheory.Shannon.wz_fullKernelSupport_perturbation

            source
            {α : Type u_1} {β : Type u_2} {γ : Type u_3} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [Fintype γ] [DecidableEq γ] [Nonempty γ] [MeasurableSpace γ] [MeasurableSingletonClass γ] (P : α × β) (d : αγ) (D : ) {k : } {qf : (α × β × Fin k) × (Fin k × βγ)} (hfact : IsWynerZivFactorizable (Fin k) P qf.1) (hdist : wzExpectedDistortion (Fin k) d qf.1 qf.2 D) {R : } (hobj : wzMutualInfoXU (Fin k) qf.1 - wzMutualInfoYU (Fin k) qf.1 < R) {δ : } ( : 0 < δ) :
            ∃ (q' : α × β × Fin k) (κ' : αFin k), (∀ (x : α) (y : β) (u : Fin k), q' (x, y, u) = κ' x u * P (x, y)) (∀ (x : α) (u : Fin k), 0 < κ' x u) (∀ (x : α), u : Fin k, κ' x u = 1) IsWynerZivFactorizable (Fin k) P q' wzMutualInfoXU (Fin k) q' - wzMutualInfoYU (Fin k) q' < R wzExpectedDistortion (Fin k) d q' qf.2 D + δ

            Full-support kernel perturbation. From a feasible factorizable test channel qf (row-stochastic kernel, distortion ≤ D) whose Wyner–Ziv objective is strictly below R, and any slack δ > 0, produce a perturbed factorizable channel q' with a strictly positive kernel κ' (full support), whose objective is still < R and whose distortion is ≤ D + δ.

            The perturbation is q' := (1 - τ) • qf.1 + τ • q_unif with q_unif the uniform-kernel factorizable joint and τ ∈ (0, 1] small: convex combination preserves factorizability (IsWynerZivFactorizable_convex_combination) and distortion feasibility (WynerZivFactorizableConstraint_convex_combination), the kernel κ' = (1 - τ) κ + τ/k ≥ τ/k > 0 gains full support, and continuity of the objective (continuous_wzObjective) keeps it < R for small τ.

            Note this yields full support of the kernel, hence full support of the (X, U) joint marginal wzMarginalXU q' only on {x | 0 < P_X x} (the construction lemma restricts the source alphabet to this support): wzMarginalXU q' (x,u) = κ'(x,u)·P_X(x).

            Used by
              theorem

              InformationTheory.Shannon.wz_tendsto_exp_mul_codebookSize_inv

              source
              {c R : } (hcR : c < R) :
              Filter.Tendsto (fun (n : ) => Real.exp (n * c) * (↑(codebookSize R n))⁻¹) Filter.atTop (nhds 0)

              Message-count decay adapter. For c < R, the ratio exp(n c) / codebookSize R n → 0 as n → ∞. This is the E2 decoder-confusion decay term (collision mass over the bin count). Re-proved locally here because the Slepian–Wolf original tendsto_exp_mul_codebookSize_inv is private to PairBound.lean; the proof is a squeeze_zero against exp(n (c − R)) using (codebookSize R n)⁻¹ ≤ exp(−n R) from Nat.le_ceil.

              Used by

                Covering + binning construction skeleton #

                The monolithic covering+binning body of wz_goodCode_exists_of_testChannel is decomposed into an ordered chain of sub-lemmas. The pure-regularity leaf wz_restrictedCoveringJoint_pos is proved here; the covering / source-support / diagonalization steps (wz_covering_lossyCode_exists, wz_expectedBlockDistortion_source_agree, wz_diagonalize_slack) and the per-n binning+covering assembly wz_perN_covering_binning_code follow. Full support of the covering source stays proof-internal (restricted to the subtype {x // 0 < P_X x}), never a signature hypothesis.

                theorem

                InformationTheory.Shannon.wz_restrictedCoveringJoint_pos

                source
                {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] {k : } (κ' : αFin k) (hκ'pos : ∀ (x : α) (u : Fin k), 0 < κ' x u) (hκ'sum : ∀ (x : α), u : Fin k, κ' x u = 1) :
                Nonempty { x : α // 0 < y : β, P_XY.real {(x, y)} } (∀ (p : { x : α // 0 < y : β, P_XY.real {(x, y)} } × Fin k), 0 < κ' (↑p.1) p.2 * y : β, P_XY.real {(p.1, y)}) (fun (p : { x : α // 0 < y : β, P_XY.real {(x, y)} } × Fin k) => κ' (↑p.1) p.2 * y : β, P_XY.real {(p.1, y)}) stdSimplex ({ x : α // 0 < y : β, P_XY.real {(x, y)} } × Fin k)

                Restricted covering joint, full support. From a strictly positive row-stochastic kernel κ' and the source marginal P_X x = ∑_y P_XY(x,y), the (X, U) joint κ'(x, u) · P_X(x) restricted to the support subtype α' := {x // 0 < P_X x} is a strictly positive pmf on α' × Fin k:

                • α' is nonempty (a probability measure cannot have every row of P_X vanish);
                • the joint is strictly positive on α' × Fin k (both factors are positive there);
                • it lies in the standard simplex (row-sums collapse to ∑_{x' : α'} P_X(x'.1) = 1, the zero atoms of P_X contributing nothing).

                This is the global-full-support source the rate-distortion covering theorem rate_distortion_achievability hard-requires (hqStar_pos), obtained on the restricted alphabet because factorizability forces P_X's zero atoms into the joint regardless of κ'. @audit:ok

                Used by
                  theorem

                  InformationTheory.Shannon.wz_expectedBlockDistortion_source_agree

                  source
                  {α : Type u_1} {β : Type u_2} {γ : Type u_3} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [Fintype γ] [DecidableEq γ] [Nonempty γ] [MeasurableSpace γ] [MeasurableSingletonClass γ] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] (d : DistortionFn α γ) {M n : } (c₁ c₂ : WynerZivCode M n α β γ) (hagree : ∀ (x : Fin nα) (y : Fin nβ), (∀ (i : Fin n), 0 < y' : β, P_XY.real {(x i, y')})c₁.decoder (c₁.encoder x, y) = c₂.decoder (c₂.encoder x, y)) :

                  Source-support block-distortion reconciliation. Two Wyner–Ziv codes that decode identically on every source sequence hitting only support atoms of P_X have equal expected block distortion, because Measure.pi P_XY assigns zero mass to sequences reaching a zero atom of P_X. This is the null-set transport that lets a code built on the support subtype α' := {x // 0 < P_X x} extend to a code on the full alphabet α without changing its distortion.

                  hagree is a genuine agreement precondition (not a bundled covering bound); the conclusion is the measure-level distortion equality only.

                  Used by
                    theorem

                    InformationTheory.Shannon.wz_jointStronglyTypical_mem_distortionTypical

                    source
                    {k : } [Nonempty (Fin k)] {α' : Type u_5} [Fintype α'] [DecidableEq α'] [Nonempty α'] [MeasurableSpace α'] [MeasurableSingletonClass α'] (qStar : α' × Fin k) (hmem : qStar stdSimplex (α' × Fin k)) (d' : DistortionFn α' (Fin k)) {ε_join ε_dist δ_typ : } (hej_nn : 0 ε_join) (hbX : (Fintype.card (Fin k)) * ε_join * logSumAbs (rdAmbient qStar) ChannelCoding.iidXs < ε_dist) (hbY : (Fintype.card α') * ε_join * logSumAbs (rdAmbient qStar) ChannelCoding.iidYs < ε_dist) (hbJ : ε_join * logSumAbs (rdAmbient qStar) (ChannelCoding.jointSequence ChannelCoding.iidXs ChannelCoding.iidYs) < ε_dist) (hdist : ε_join * p : α' × Fin k, (d' p.1 p.2) δ_typ) {n : } (hn : 0 < n) (x : Fin nα') (y : Fin nFin k) (hxy : (x, y) jointStronglyTypicalSet (rdAmbient qStar) ChannelCoding.iidXs ChannelCoding.iidYs n ε_join) :

                    Strong-typicality ⟹ distortion-typicality bridge for the rdAmbient source. A joint strongly typical pair (x, y) (within ε_join) is entropy-typical on all three axes and its empirical block distortion is within δ_typ of the expected distortion, provided the three axis slacks fit under ε_dist and the aggregate distortion drift fits under δ_typ. Used to discharge the covering theorem's h_jts_subset_dts premise.

                    Used by
                      theorem

                      InformationTheory.Shannon.wz_covering_lossyCode_exists

                      source
                      {k : } [Nonempty (Fin k)] {α' : Type u_5} [Fintype α'] [DecidableEq α'] [Nonempty α'] [MeasurableSpace α'] [MeasurableSingletonClass α'] (qStar : α' × Fin k) (hpos : ∀ (p : α' × Fin k), 0 < qStar p) (hmem : qStar stdSimplex (α' × Fin k)) (d' : DistortionFn α' (Fin k)) {R₁ D : } (hI : mutualInfoPmf qStar < R₁) (hfeas : expectedDistortionPmf d' qStar D) {ε' : } (hε' : 0 < ε') :
                      ∃ (N : ), ∀ (n : ), N n∃ (M : ), Real.exp (n * R₁)⌉₊ M M Real.exp (n * R₁) + 1 ∃ (c : LossyCode M n α' (Fin k)), c.expectedBlockDistortion (MeasureTheory.Measure.map (ChannelCoding.iidXs 0) (rdAmbient qStar)) d' D + ε'

                      Rate-distortion covering layer. For a strictly positive joint pmf qStar on α' × Fin k with mutualInfoPmf qStar < R₁ and a proxy distortion d' feasible at D, the rate-distortion achievability theorem yields, for all large block lengths n, a lossy code with ≥ ⌈exp(n R₁)⌉ codewords whose expected block distortion (under the rdAmbient-pushed source) is within D + ε'.

                      The full support hpos is a regularity precondition (the covering theorem's hqStar_pos); the rate-distortion slack quintet (ε_X … δ_typ, qZ_min) is constructed in the body, not exposed. The reconciliation between the covering proxy d' (X↔U) and the Wyner–Ziv distortion (X↔γ) stays load-bearing in the body / (BD), never bundled into a predicate.

                      Used by
                        theorem

                        InformationTheory.Shannon.wz_coveringDistortion_reconcile

                        source
                        {α : Type u_1} {β : Type u_2} {γ : Type u_3} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [Fintype γ] [DecidableEq γ] [Nonempty γ] [MeasurableSpace γ] [MeasurableSingletonClass γ] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] (d : DistortionFn α γ) {k : } (κ' : αFin k) (f : Fin k × βγ) :
                        (expectedDistortionPmf (fun (x' : { x : α // 0 < y : β, P_XY.real {(x, y)} }) (u : Fin k) => (∑ y : β, (P_XY.real {(x', y)} / y' : β, P_XY.real {(x', y')}) * (d (↑x') (f (u, y)))).toNNReal) fun (p : { x : α // 0 < y : β, P_XY.real {(x, y)} } × Fin k) => κ' (↑p.1) p.2 * y : β, P_XY.real {(p.1, y)}) = wzExpectedDistortion (Fin k) (fun (a : α) (b : γ) => (d a b)) (fun (p : α × β × Fin k) => κ' p.1 p.2.2 * P_XY.real {(p.1, p.2.1)}) f

                        Covering-distortion reconciliation identity. The covering proxy distortion d' on the source-support subtype α' := {x // 0 < P_X x}, defined as the Y-conditional expectation d'(⟨x, _⟩, u) := ∑_y (P_XY(x,y) / P_X x) · d(x, f(u, y)), reconciles with the Wyner–Ziv distortion functional: for the restricted (X, U)-joint qStar(⟨x, _⟩, u) := κ'(x, u) · P_X x, the pmf-form expected distortion of d' equals the Wyner–Ziv expected distortion of the factorizable joint q'(x, y, u) := κ'(x, u) · P_XY(x, y) under the reconstruction f.

                        The identity is the load-bearing bridge that lets the rate-distortion covering theorem (which measures distortion X ↔ U via d') discharge the Wyner–Ziv feasibility (X ↔ γ via f). It holds because P_X x · (P_XY(x,y) / P_X x) = P_XY(x,y) on the support (where P_X x > 0), and the zero atoms of P_X contribute nothing on either side (q' vanishes there since every P_XY(x,y) = 0 when P_X x = 0). @audit:ok

                        Used by
                          noncomputable def

                          InformationTheory.Shannon.wzSideInfoMarginal

                          source
                          {α : Type u_1} {β : Type u_2} [Fintype α] [MeasurableSpace α] [MeasurableSpace β] (P_XY : MeasureTheory.Measure (α × β)) {k : } (κ' : αFin k) :
                          Fin k × { y : β // 0 < x : α, P_XY.real {(x, y)} }

                          The (U, Y)-marginal joint pmf feeding the side-information ambient, restricted to the positive-Y-marginal subtype. For a full-support covering kernel κ' and the source law P_XY, the value at (u, y) is ∑ₓ κ'(x, u) · P_XY{(x, y)}, the Y-side analogue of the covering pmf qStar (which lives on the positive-X-marginal subtype).

                          Equations
                          Instances For
                            Used by
                              theorem

                              InformationTheory.Shannon.wzSideInfoMarginal_pos

                              source
                              {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) {k : } (κ' : αFin k) (hκ'pos : ∀ (x : α) (u : Fin k), 0 < κ' x u) (p : Fin k × { y : β // 0 < x : α, P_XY.real {(x, y)} }) :
                              0 < wzSideInfoMarginal P_XY κ' p
                              Used by
                                theorem

                                InformationTheory.Shannon.wzSideInfoMarginal_sum_eq_one

                                source
                                {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] {k : } (κ' : αFin k) (hκ'sum : ∀ (x : α), u : Fin k, κ' x u = 1) :
                                p : Fin k × { y : β // 0 < x : α, P_XY.real {(x, y)} }, wzSideInfoMarginal P_XY κ' p = 1
                                Used by
                                  theorem

                                  InformationTheory.Shannon.wzSideInfoMarginal_mem_stdSimplex

                                  source
                                  {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] {k : } (κ' : αFin k) (hκ'pos : ∀ (x : α) (u : Fin k), 0 < κ' x u) (hκ'sum : ∀ (x : α), u : Fin k, κ' x u = 1) :
                                  wzSideInfoMarginal P_XY κ' stdSimplex (Fin k × { y : β // 0 < x : α, P_XY.real {(x, y)} })
                                  Used by
                                    theorem

                                    InformationTheory.Shannon.wzSideInfoMarginal_subtype_nonempty

                                    source
                                    {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] :
                                    Nonempty { y : β // 0 < x : α, P_XY.real {(x, y)} }
                                    Used by
                                      def

                                      InformationTheory.Shannon.wzCoveringAcceptFailSet

                                      source
                                      {α : Type u_1} {β : Type u_2} [Fintype α] [MeasurableSpace α] [Fintype β] [MeasurableSpace β] (P_XY : MeasureTheory.Measure (α × β)) {k : } (κ' : αFin k) {M n : } (c : LossyCode M n { x : α // 0 < y : β, P_XY.real {(x, y)} } (Fin k)) (ε : ) :
                                      Set (Fin n{ x : α // 0 < y : β, P_XY.real {(x, y)} } × β)

                                      Covering-acceptance failure event. For a covering LossyCode c on the source-support subtype α' := {x // 0 < P_X x}, the set of block source–side pairs p : Fin n → α' × β whose true covering codeword c.decoder (c.encoder x) is not jointly (strongly) typical, at radius ε, with the side information y in the side-information ambient rdAmbient (wzSideInfoMarginal P_XY κ'). This is the covering half of the Wyner–Ziv error event E2: acceptance failure of the correct covering word (wzBinTypicalDecoder_eq_of_unique requires this joint typicality to recover it), so C2 ⊆ E2. Pure event set (data), used to state the covering-acceptance-failure mass bound threaded from the covering construction to wz_exists_binning_E2_bound (A3).

                                      Equations
                                      • One or more equations did not get rendered due to their size.
                                      Instances For
                                        Used by
                                          theorem

                                          InformationTheory.Shannon.wz_QXY_mem_stdSimplex

                                          source
                                          {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] :
                                          (fun (p : { x : α // 0 < y : β, P_XY.real {(x, y)} } × β) => P_XY.real {(p.1, p.2)}) stdSimplex ({ x : α // 0 < y : β, P_XY.real {(x, y)} } × β)

                                          The source–side covering pmf (x', y) ↦ P_XY{(x'.1, y)} (on the source-support subtype) is a probability vector: its values are nonnegative measures and they total 1 (the zero-P_X atoms carry no mass, so the subtype sum equals the full joint mass). Used to supply the IsProbabilityMeasure instance for the correlated-joint source measure Measure.pi (pmfToMeasure …).

                                          Used by

                                            Markov-core chain and its regularity helpers #

                                            These lemmas are placed before the covering atom wz_coveringFamily_of_testChannel (below) so it can consume the leaf wz_covering_chosenWord_sideInfo_typical.

                                            theorem

                                            InformationTheory.Shannon.rdAmbient_iIndepFun_iidXs

                                            source
                                            Used by
                                              theorem

                                              InformationTheory.Shannon.rdAmbient_iIndepFun_iidYs

                                              source
                                              Used by
                                                theorem

                                                InformationTheory.Shannon.rdAmbient_iIndepFun_jointSequence

                                                source
                                                Used by
                                                  theorem

                                                  InformationTheory.Shannon.rdAmbient_pairwise_indep_jointSequence

                                                  source
                                                  Used by
                                                    theorem

                                                    InformationTheory.Shannon.rdAmbient_identDistrib_iidXs

                                                    source
                                                    Used by
                                                      theorem

                                                      InformationTheory.Shannon.rdAmbient_identDistrib_iidYs

                                                      source
                                                      Used by
                                                        theorem

                                                        InformationTheory.Shannon.rdAmbient_identDistrib_jointSequence

                                                        source
                                                        Used by
                                                          theorem

                                                          InformationTheory.Shannon.rdAmbient_iidXs_real_singleton_pos

                                                          source
                                                          {A : Type u_5} {B : Type u_6} [Fintype A] [DecidableEq A] [Nonempty A] [MeasurableSpace A] [MeasurableSingletonClass A] [Fintype B] [DecidableEq B] [Nonempty B] [MeasurableSpace B] [MeasurableSingletonClass B] (q : A × B) (hq : q stdSimplex (A × B)) (hpos : ∀ (p : A × B), 0 < q p) (x : A) :
                                                          Used by
                                                            theorem

                                                            InformationTheory.Shannon.rdAmbient_iidYs_real_singleton_pos

                                                            source
                                                            {A : Type u_5} {B : Type u_6} [Fintype A] [DecidableEq A] [Nonempty A] [MeasurableSpace A] [MeasurableSingletonClass A] [Fintype B] [DecidableEq B] [Nonempty B] [MeasurableSpace B] [MeasurableSingletonClass B] (q : A × B) (hq : q stdSimplex (A × B)) (hpos : ∀ (p : A × B), 0 < q p) (y : B) :
                                                            Used by
                                                              theorem

                                                              InformationTheory.Shannon.rdAmbient_jointSequence_real_singleton_pos

                                                              source
                                                              Used by
                                                                theorem

                                                                InformationTheory.Shannon.rdAmbient_map_fst_jointSequence

                                                                source
                                                                Used by
                                                                  theorem

                                                                  InformationTheory.Shannon.rdAmbient_map_snd_jointSequence

                                                                  source
                                                                  Used by
                                                                    theorem

                                                                    InformationTheory.Shannon.rdAmbient_map_jointRV_jointSequence_eq_pi

                                                                    source

                                                                    The n-fold pair-sequence law of rdAmbient q is the product of the pmf q: the joint (X, Y)-sequence jointRV (jointSequence iidXs iidYs) n pushes rdAmbient q to Measure.pi (pmfToMeasure q). The iid-to-product identity for the pair sequence (the jointSequence analogue of wz_ambient_jointRV_iidYs_eq_pi).

                                                                    Used by
                                                                      theorem

                                                                      InformationTheory.Shannon.wz_map_injective_real_singleton

                                                                      source
                                                                      {Ω : Type u_5} {γ₀ : Type u_6} {δ₀ : Type u_7} [MeasurableSpace Ω] [MeasurableSpace γ₀] [MeasurableSingletonClass γ₀] [MeasurableSpace δ₀] [MeasurableSingletonClass δ₀] (μ : MeasureTheory.Measure Ω) (X : Ωγ₀) (hX : Measurable X) (g : γ₀δ₀) (hg : Function.Injective g) (hgmeas : Measurable g) (a : γ₀) :
                                                                      (MeasureTheory.Measure.map (fun (ω : Ω) => g (X ω)) μ).real {g a} = (MeasureTheory.Measure.map X μ).real {a}

                                                                      Per-atom mass is preserved by pushing forward along an injective (measurable) alphabet map: (μ.map (g ∘ X)).real {g a} = (μ.map X).real {a}.

                                                                      Used by
                                                                        theorem

                                                                        InformationTheory.Shannon.wz_entropy_map_injective

                                                                        source
                                                                        {Ω : Type u_5} {γ₀ : Type u_6} {δ₀ : Type u_7} [MeasurableSpace Ω] [Fintype γ₀] [DecidableEq γ₀] [Nonempty γ₀] [MeasurableSpace γ₀] [MeasurableSingletonClass γ₀] [Fintype δ₀] [DecidableEq δ₀] [Nonempty δ₀] [MeasurableSpace δ₀] [MeasurableSingletonClass δ₀] (μ : MeasureTheory.Measure Ω) (X : Ωγ₀) (hX : Measurable X) (g : γ₀δ₀) (hg : Function.Injective g) (hgmeas : Measurable g) :
                                                                        (entropy μ fun (ω : Ω) => g (X ω)) = entropy μ X

                                                                        Shannon entropy is invariant under an injective (measurable) relabeling of the alphabet.

                                                                        Used by
                                                                          theorem

                                                                          InformationTheory.Shannon.wz_sourcePmf_mem_stdSimplex

                                                                          source
                                                                          {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] [Nonempty { x : α // 0 < y : β, P_XY.real {(x, y)} }] :
                                                                          (fun (p : { x : α // 0 < y : β, P_XY.real {(x, y)} } × β) => P_XY.real {(p.1, p.2)}) stdSimplex ({ x : α // 0 < y : β, P_XY.real {(x, y)} } × β)

                                                                          The Wyner–Ziv source per-coordinate pmf p ↦ P_XY{(p.1.1, p.2)} on α' × β is a pmf.

                                                                          Used by
                                                                            theorem

                                                                            InformationTheory.Shannon.wz_source_snd_eq_ambient_snd_map

                                                                            source
                                                                            {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] [Nonempty { x : α // 0 < y : β, P_XY.real {(x, y)} }] {k : } [Nonempty (Fin k)] (κ' : αFin k) (hκ'pos : ∀ (x : α) (u : Fin k), 0 < κ' x u) (hκ'sum : ∀ (x : α), u : Fin k, κ' x u = 1) :

                                                                            Side-information-law agreement: the source's full-β Y-law equals the β-image (under the subtype coercion) of the ambient's β'-Y-law.

                                                                            Used by
                                                                              theorem

                                                                              InformationTheory.Shannon.wz_ambient_jointRV_iidYs_eq_pi

                                                                              source
                                                                              {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) [MeasureTheory.IsProbabilityMeasure P_XY] {k : } [Nonempty (Fin k)] (κ' : αFin k) (hκ'pos : ∀ (x : α) (u : Fin k), 0 < κ' x u) (hκ'sum : ∀ (x : α), u : Fin k, κ' x u = 1) (n : ) :

                                                                              The n-fold side-information law of the ambient factorizes as the product of its single-letter β'-Y-marginal.

                                                                              Used by