InformationTheory

InformationTheory.Shannon.WynerZiv.Achievability.Decomposition

source

Wyner–Ziv achievability — Steps 3–7 distortion decomposition and pmf-side product bounds #

Steps 3–7 decomposition (binning / decoder / error exponents / squeeze) #

The covering data of Steps 1–2 (wz_coveringFamily_of_testChannel) is consumed by the binning + decoder leg. This leg is decomposed into:

  • wzCodeOfCoveringBinning — the Wyner–Ziv code assembled from a covering codebook, a binning of the covering index, and a bin/side-information decoder (pure def).
  • wzBinTypicalDecoder (+ uniqueness wzBinTypicalDecoder_eq_of_unique) — the bin-restricted conditional-typicality decoder, searching a bin's covering codebook members for the one jointly typical with Y^n (pure def + the decoder equation under a unique witness), mirroring Slepian–Wolf swJointTypicalDecoder / swJointTypicalDecoder_eq_of_unique.
  • wz_covering_failure_prob_le — covering-failure exponent.
  • wz_codebook_confusion_expectation_le — codebook-restricted decoder confusion exponent (the crux).
  • wz_perDelta_covering_binning — the capstone consuming the covering data and producing the per-slack code family (binning + decoder + error exponents + derandomize + squeeze + source extension).
  • wzLiftSupportCode — the source-extension lift α' → α (pure def), used together with wz_expectedBlockDistortion_source_agree.
def

InformationTheory.Shannon.wzCodeOfCoveringBinning

source
{β : Type u_2} {γ : Type u_3} [MeasurableSpace β] [MeasurableSpace γ] {α' : Type u_5} [MeasurableSpace α'] {k M M₁ n : } (c₁ : LossyCode M₁ n α' (Fin k)) (f : Fin M₁Fin M) (rec : Fin k × βγ) (dec : Fin M × (Fin nβ)Fin nFin k) :
WynerZivCode M n α' β γ

Wyner–Ziv code from a covering codebook + binning + bin decoder. The encoder covers the source with the covering codebook (c₁.encoder) and bins the covering index (f). The decoder reconstructs γ^n letterwise via rec (the test-channel decoder qf.2 : Fin k × β → γ) from the bin decoder's word dec (m, y) : Fin n → Fin k and the side information y. Pure assembly; the covering codebook c₁, the binning f, the reconstruction map rec and the bin decoder dec are all supplied.

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    Used by
      noncomputable def

      InformationTheory.Shannon.wzBinTypicalDecoder

      source
      {β : Type u_2} [Fintype β] [MeasurableSpace β] {α' : Type u_5} [MeasurableSpace α'] {Ω : Type u_6} [MeasurableSpace Ω] {k M M₁ n : } [Nonempty (Fin k)] (μ : MeasureTheory.Measure Ω) (Us : ΩFin k) (Ys : Ωβ) (ε : ) (c₁ : LossyCode M₁ n α' (Fin k)) (f : Fin M₁Fin M) :
      Fin M × (Fin nβ)Fin nFin k

      Bin/side-information conditional-typicality decoder. Given a bin m and side information y, search the bin's covering codebook members {c₁.decoder m' | f m' = m} for the unique word jointly typical with y, returning that Fin n → Fin k word (falling back to an arbitrary word if none exists or the witness is not unique). The search ranges over codebook members only (indexed by the covering index m'), not over all Fin n → Fin k words — this restriction is what makes the decoder-confusion event (S5b) achievable at the Wyner–Ziv rate. Mirror of Slepian–Wolf swJointTypicalDecoder.

      Equations
      • One or more equations did not get rendered due to their size.
      Instances For
        Used by
          theorem

          InformationTheory.Shannon.wzBinTypicalDecoder_eq_of_unique

          source
          {β : Type u_2} [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] {α' : Type u_5} [MeasurableSpace α'] {Ω : Type u_6} [MeasurableSpace Ω] {k M M₁ n : } [Nonempty (Fin k)] (μ : MeasureTheory.Measure Ω) (Us : ΩFin k) (Ys : Ωβ) (ε : ) (c₁ : LossyCode M₁ n α' (Fin k)) (f : Fin M₁Fin M) {m₁ : Fin M₁} {y : Fin nβ} (htrue : (c₁.decoder m₁, y) ChannelCoding.jointlyTypicalSet μ Us Ys n ε) (hunique : ∀ (u : Fin nFin k), (∃ (m' : Fin M₁), f m' = f m₁ c₁.decoder m' = u)(u, y) ChannelCoding.jointlyTypicalSet μ Us Ys n εu = c₁.decoder m₁) :
          wzBinTypicalDecoder μ Us Ys ε c₁ f (f m₁, y) = c₁.decoder m₁

          If the covering codeword c₁.decoder m₁ is jointly typical with y and is the unique bin-f m₁ codebook member so typical, then wzBinTypicalDecoder recovers it. Mirror of swJointTypicalDecoder_eq_of_unique.

          Used by
            theorem

            InformationTheory.Shannon.wz_covering_failure_prob_le

            source
            {α' : Type u_5} [Fintype α'] [DecidableEq α'] [Nonempty α'] [MeasurableSpace α'] [MeasurableSingletonClass α'] {Ω : Type u_6} [MeasurableSpace Ω] {k n M₁ : } [Nonempty (Fin k)] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (Xs : Ωα') (Us : ΩFin k) (ε : ) (P_X : MeasureTheory.Measure (Fin nα')) [MeasureTheory.IsProbabilityMeasure P_X] (I δ : ) (hmass : ∀ (x : Fin nα'), Real.exp (-n * (I + δ)) (MeasureTheory.Measure.pi fun (x : Fin n) => MeasureTheory.Measure.map (Us 0) μ).real {u : Fin nFin k | (x, u) ChannelCoding.jointlyTypicalSet μ Xs Us n ε}) :
            (x : Fin nα'), (1 - (MeasureTheory.Measure.pi fun (x : Fin n) => MeasureTheory.Measure.map (Us 0) μ).real {u : Fin nFin k | (x, u) ChannelCoding.jointlyTypicalSet μ Xs Us n ε}) ^ M₁ P_X Real.exp (-M₁ * Real.exp (-n * (I + δ)))

            Covering-failure exponent. The codebook-averaged probability that a strongly-typical source x finds no covering codeword jointly typical with it decays doubly-exponentially: ∫ x, (1 − p_typ x)^{M₁} ≤ exp(−M₁ · exp(−n(I + δ))), where p_typ x is the per-codeword conditional-typicality mass (bounded below by exp(−n(I + δ)) via wz_covering_sideInfo_mass_ge), passed here as hmass.

            hmass is the per-source covering-acceptance mass lower bound exp(−n(I+δ)) ≤ p_typ x. With it, (1−p)^M₁ ≤ e^{−M₁ p} ≤ e^{−M₁·exp(−n(I+δ))} pointwise (p_typ x ∈ [0,1], p ≥ exp(−n(I+δ))), then integrate over the probability measure P_X. The pointwise p_typ x ≤ 1 holds even without measurability of Us 0: μ.map (Us 0) is a sub-probability measure (Measure.isFiniteMeasure_map + map mass ≤ 1), so its product Measure.pi is a sub-probability measure (Measure.pi_univ), and the mass of any set is ≤ 1. The (1−t)^M ≤ e^{−Mt} step reuses one_sub_pow_le_exp_neg_mul. @audit:ok

            Used by
              theorem

              InformationTheory.Shannon.wz_codebook_confusion_expectation_le

              source
              {β : Type u_2} [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] {α' : Type u_5} [MeasurableSpace α'] {Ω : Type u_6} [MeasurableSpace Ω] {k n M M₁ : } [Nonempty (Fin k)] [NeZero M] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (Ys : Ωβ) (c₁ : LossyCode M₁ n α' (Fin k)) (trueIdx : ΩFin M₁) (hYs : ∀ (i : ), Measurable (Ys i)) (htrueIdx : Measurable trueIdx) (binMeas : MeasureTheory.Measure (Fin M₁Fin M)) [MeasureTheory.IsProbabilityMeasure binMeas] (jts : Set ((Fin nFin k) × (Fin nβ))) (hjts_meas : MeasurableSet jts) (I_YU : ) (hmass : ∀ (m' : Fin M₁), μ.real {ω : Ω | (c₁.decoder m', jointRV Ys n ω) jts} Real.exp (-n * I_YU)) (hcollision : ∀ (m' m : Fin M₁), m' mbinMeas.real {f : Fin M₁Fin M | f m' = f m} = (↑M)⁻¹) :
              (f : Fin M₁Fin M), μ.real {ω : Ω | ∃ (m' : Fin M₁), m' trueIdx ω f m' = f (trueIdx ω) (c₁.decoder m', jointRV Ys n ω) jts} binMeas M₁ * Real.exp (-n * I_YU) * (↑M)⁻¹

              Codebook-restricted decoder confusion exponent. The binning-averaged probability that some codebook member c₁.decoder m' other than the true covering codeword shares the true bin and is jointly typical with Y^n is at most M₁ · exp(−n · I(U;Y)) · M⁻¹.

              Restricting the confusable set to the covering codebook is what achieves the Wyner–Ziv rate. Binning all u-sequences would give the count exp(n·H(U|Y)), forcing the rate down to H(U|Y) — too weak; this bound instead restricts to the covering codebook (M₁ = ⌈exp(n·I(X;U))⌉ members), so the alias count is M₁ rather than exp(n·H(U|Y)). With M = ⌈exp(n·R)⌉ bins, the bound is M₁ · exp(−n·I(U;Y)) / M ≈ exp(n·(I(X;U) − I(U;Y) − R)), which vanishes precisely when R > I(X;U) − I(Y;U) — the Wyner–Ziv rate.

              hmass is the per-codeword joint-typicality mass upper bound μ{codeword m' typical with Y^n} ≤ exp(−n·I_YU) (the AEP bound for a covering codeword independent of Y^n); hcollision is the binning-collision property binMeas{f | f m' = f m} = M⁻¹ for distinct indices, mirroring binning_collision_prob. The codebook-restricted union over m' : Fin M₁ stays in the body (not a hypothesis): swap the order of integration, bound the per-ω binMeas-slice by union bound + hcollision as M⁻¹ · #{m' typical}, integrate over μ, then apply hmass to each of the M₁ codewords to get M⁻¹ · M₁ · exp(−n·I_YU). hYs/htrueIdx (measurability of the side-information block RV and of the covering index) are regularity preconditions for the Tonelli swap, supplied by the call site.

              Implementation note: the typical set is an abstract measurable set jts (parameter hjts_meas : MeasurableSet jts) rather than a concrete jointlyTypicalSet, since the body uses no property of it beyond measurability. This lets the call site instantiate the confusion integral under the source product measure with the typical set defined on the side-information ambient — two different measures a concrete typical set could never match; the per-codeword mass hmass is then supplied via a side-information-marginal transfer to wz_covering_codeword_sideInfo_mass_le. @audit:ok

              Used by
                noncomputable def

                InformationTheory.Shannon.wzLiftSupportCode

                source
                {α : Type u_1} {β : Type u_2} {γ : Type u_3} [MeasurableSpace α] [Fintype β] [MeasurableSpace β] [MeasurableSpace γ] (P_XY : MeasureTheory.Measure (α × β)) {M n : } (x₀ : { x : α // 0 < y : β, P_XY.real {(x, y)} }) (cSupp : WynerZivCode M n { x : α // 0 < y : β, P_XY.real {(x, y)} } β γ) :
                WynerZivCode M n α β γ

                Source-extension lift α' → α. Lift a Wyner–Ziv code over the source support subtype α' := {x // 0 < P_X x} to a code over the full alphabet α, using the default support element x₀ for out-of-support coordinates (which have zero Measure.pi P_XY-mass, so the lift preserves expected block distortion via wz_expectedBlockDistortion_source_agree). The decoder is unchanged (it does not touch α). Pure def.

                Equations
                • One or more equations did not get rendered due to their size.
                Instances For
                  Used by
                    noncomputable def

                    InformationTheory.Shannon.wzIndexBinningMeasure

                    source
                    (M₁ M : ) [NeZero M] :

                    Index-binning measure. Hash each of the M₁ covering-codebook indices Fin M₁ independently to a uniformly random bin in Fin M. This is the Fin M₁-index analogue of binningMeasure (which hashes whole sequences (Fin n → α) → Fin M); it is the concrete binMeas : Measure (Fin M₁ → Fin M) that the codebook-restricted decoder-confusion exponent wz_codebook_confusion_expectation_le consumes.

                    Equations
                    Instances For
                      Used by
                        instance

                        InformationTheory.Shannon.wzIndexBinningMeasure.instIsProbabilityMeasure

                        source

                        The index-binning measure is a probability measure.

                        Used by
                          theorem

                          InformationTheory.Shannon.wzIndexBinningMeasure_singleton_real

                          source
                          (M₁ M : ) [NeZero M] (f : Fin M₁Fin M) :

                          Singleton mass for the index-binning measure. For any hash function f : Fin M₁ → Fin M, its wzIndexBinningMeasure-mass is (1/M)^{M₁} (each of the M₁ covering indices independently picks one of M bins). The Fin M₁-index mirror of binningMeasure_singleton_real.

                          Used by
                            theorem

                            InformationTheory.Shannon.wzIndexBinningMeasure_collision

                            source
                            {M₁ M : } [NeZero M] {m' m : Fin M₁} (h : m' m) :
                            (wzIndexBinningMeasure M₁ M).real {f : Fin M₁Fin M | f m' = f m} = (↑M)⁻¹

                            Index-binning collision probability. Two distinct covering indices m' ≠ m hash to the same bin with probability exactly 1/M. Supplies hcollision to wz_codebook_confusion_expectation_le; the Fin M₁-index mirror of binning_collision_prob.

                            Used by
                              theorem

                              InformationTheory.Shannon.wz_mutualInfo_restriction_eq

                              source
                              {α : Type u_1} {β : Type u_2} [Fintype α] [DecidableEq α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] (P_XY : MeasureTheory.Measure (α × β)) (k : ) (q' : α × β × Fin k) (κ' : αFin k) (qStar : { x : α // 0 < y : β, P_XY.real {(x, y)} } × Fin k) (hfact_eq : ∀ (x : α) (y : β) (u : Fin k), q' (x, y, u) = κ' x u * P_XY.real {(x, y)}) (hκ'sum : ∀ (x : α), u : Fin k, κ' x u = 1) (hqStar_eq : ∀ (p : { x : α // 0 < y : β, P_XY.real {(x, y)} } × Fin k), qStar p = κ' (↑p.1) p.2 * y : β, P_XY.real {(p.1, y)}) :

                              Mutual-information restriction identity. The covering mutual information computed on the support-restricted joint qStar (over the source support subtype α' := {x // 0 < P_X x}) equals the Wyner–Ziv covering objective wzMutualInfoXU computed on the full-alphabet factorizable joint q'. The support restriction drops only zero atoms of the source marginal P_X, which contribute Real.negMulLog 0 = 0 to every marginal and joint entropy sum, so the two mutual informations coincide. This algebraic leaf lets the covering family hcov — whose premise is mutualInfoPmf qStar < R₁ — be fed at a covering rate R₁ chosen strictly above wzMutualInfoXU q' = I(X;U).

                              The support-restriction principle (key) sums the vanishing off-support terms away (Real.negMulLog 0 = 0), matching the three marginal/joint entropy sums of qStar (over the support subtype) against those of wzMarginalXU q' (over the full alphabet). The factorization hypotheses hfact_eq/hκ'sum/hqStar_eq are genuine definitional constraints (without them the two mutual informations differ, since qStar lives over the support subtype and q' over the full alphabet); none is the conclusion. @audit:ok

                              Used by

                                pmf-side product bounds for D2 #

                                The per-codeword AEP mass bound D2 is assembled purely from single-symbol pmf products (no joint-sequence independence is available in D2's hypotheses). The following three leaves convert the typical-set membership predicate into product bounds on the alphabet-side laws μ.map (Xs 0).

                                theorem

                                InformationTheory.Shannon.wz_covering_codeword_sideInfo_mass_le

                                source
                                {β : Type u_2} [Fintype β] [DecidableEq β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] {Ω : Type u_5} [MeasurableSpace Ω] {k n : } [Nonempty (Fin k)] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (Us : ΩFin k) (Ys : Ωβ) (ε : ) ( : 0 < ε) (hUs : ∀ (i : ), Measurable (Us i)) (hYs : ∀ (i : ), Measurable (Ys i)) (hindepU : ProbabilityTheory.iIndepFun (fun (i : ) => Us i) μ) (hidentU : ∀ (i : ), ProbabilityTheory.IdentDistrib (Us i) (Us 0) μ μ) (hindepY : ProbabilityTheory.iIndepFun (fun (i : ) => Ys i) μ) (hidentY : ∀ (i : ), ProbabilityTheory.IdentDistrib (Ys i) (Ys 0) μ μ) (hposU : ∀ (u : Fin k), 0 < (MeasureTheory.Measure.map (Us 0) μ).real {u}) (hposY : ∀ (y : β), 0 < (MeasureTheory.Measure.map (Ys 0) μ).real {y}) (hposZ : ∀ (p : Fin k × β), 0 < (MeasureTheory.Measure.map (ChannelCoding.jointSequence Us Ys 0) μ).real {p}) (I_YU : ) (hI_YU : I_YU entropy μ (Us 0) + entropy μ (Ys 0) - entropy μ (ChannelCoding.jointSequence Us Ys 0) - 3 * ε) (u : Fin nFin k) :
                                μ.real {ω : Ω | (u, jointRV Ys n ω) ChannelCoding.jointlyTypicalSet μ Us Ys n ε} Real.exp (-n * I_YU)

                                Covering-codeword side-information mass upper bound. For any fixed covering codeword u : Fin n → Fin k, the probability (over the noise generating Y^n = jointRV Ys n) that u is jointly typical with Y^n is at most exp(−n · I_YU), where I_YU ≲ I(U;Y). This is the per-codeword AEP mass bound that wz_codebook_confusion_expectation_le consumes as its hmass hypothesis: because the covering codewords are drawn independently of the side information Y, a fixed covering codeword lands in a Y^n-conditional typical slice with the packing exponent exp(−n · I(U;Y)).

                                The per-codeword form is assembled directly from single-symbol pmf products (no joint-sequence independence is needed and none is available in the hypotheses). Reframing the ω-event as the Y-law mass of the fixed-u slice {y | (u, y) ∈ jointlyTypicalSet} (via map_measureReal_apply on jointRV Ys n), the slice mass is bounded by ∑_{y} exp(−n(H(Y)−ε)) · [1 ≤ exp(n(H(Z)+ε))·∏ P_Z(u,y)]; folding in the joint-typical product lower bound (prod_map_singleton_ge_of_mem_typicalSet) and marginalizing ∑_y ∏_i P_Z(u_i,y_i) = ∏_i P_U(u_i) (Finset.prod_univ_sum + sum_real_prod_singleton_of_map_fst_eq), the U-typical product bound (prod_map_singleton_le_of_mem_typicalSet) gives mass ≤ exp(−n(H(U)+H(Y)−H(U,Y)−3ε)) = exp(−n(I(U;Y)−3ε)) ≤ exp(−n·I_YU) since hI_YU : I_YU ≤ I(U;Y) − 3ε. For an atypical u the slice is empty and the mass is 0.

                                The exponent slack is exactly the sum of the joint-product slack (ε) and the Y/U typicality slacks (ε each); hI_YU is a precondition supplying the standard typicality slack, not load-bearing (the upper bound on I_YU only weakens the RHS exp(−n·I_YU)). hindepU/hidentU/ are inherited regularity preconditions that the pmf-side assembly does not consume. @audit:ok

                                Used by