InformationTheory.Shannon.ChannelCoding.CodeToAmbient
From a block code to its ambient law #
Model-independent infrastructure for the operational converses: a block code together with a
message law and a per-letter product channel determines an ambient probability measure on
message × output block, and the structural hypotheses of the single-letter converses (Markov
factorization, memorylessness, per-letter joint laws) are read off that measure. None of the
statements below mention a particular channel model — the encoder enters only through the
factorization hypotheses κ m = Wcode (g m) and κ m = ∏ⱼ W (x m j) — so a multi-user converse
instantiates them by supplying its own encoder and channel.
Main statements #
isMarkovChain_of_compProd_encoder: an ambientν ⊗ₘ κwhose kernel factors through a deterministic encodergcarries the Markov chainM → g M → Y.isMarkovChain_of_compProd_pi: for a per-letter product kernel∏ⱼ W (x m j), the output letterYᵢis conditionally independent of any variable that does not read thei-th output coordinate, given the input letterXᵢ. The conditioner is the input letter itself; the variable decoupled fromYᵢis an arbitrary measurable function of the whole ambient point, constrained only by invariance under re-randomizing coordinatei, so it may read the message as well as the other letters.isMemorylessChannel_of_compProd_pi: if the kernel is the per-letter product∏ⱼ W (x m j)of a channel applied to a deterministic codeword, the ambient is a memoryless channel.compProd_pi_map_pair_eq_of_update_invariant: pair thei-th output letter with any mapGthat is invariant under updating thei-th output coordinate and that retracts onto the input letter; the joint law of that pair is((ν ⊗ₘ κ).map G) ⊗ₘ W.comap g, the composition product ofG's own law with the channel read along the retraction. The invariance is what lets a padded auxiliary variable sit in the first component.compProd_pi_map_pair_eq: its special case whereGis the input letterx · iitself, so the joint law of thei-th input-output pair is the channel joint(ν.map fun m ↦ x m i) ⊗ₘ W.compProd_comap_map_prodMap: a composition product with a comapped kernel is the composition product of the pushed-forward measure with the kernel itself.compProd_map_prodMap: the two-sided form of the previous item — a composition product transported by a pair of maps, one on the base and one on the fiber, provided the kernel's dependence on the base factors through the base map.pi_map_comp_of_injective: a finite product measure reindexed along an injection is the product measure of the reindexed family.piBlockKernel,pi_map_unzip_eq_compProd: a product of laws each of which is its own first marginal followed by a kernelQ, read as a pair of blocks, is the composition product of the product of the first marginals with the blockwise product ofQ.mutualInfo_map_comp,condDistrib_map_comp,condMutualInfo_map_comp: information quantities and conditional distributions are invariant under a shared pushforward of all their arguments.le_log_of_ceil_exp_le:⌈exp x⌉₊ ≤ Mimpliesx ≤ log M, turning a message count into a rate bound.le_toReal_of_inv_mul_le: an averaged boundm⁻¹ · S ≤ Jinℝ≥0∞together withm · r ≤ S.toRealgivesr ≤ J.toReal, which is how a per-letter bound becomes a rate bound.
The uniform message law #
InformationTheory.Shannon.uniformCount_isProbabilityMeasure
sourceThe uniform probability law (card X)⁻¹ • count on a nonempty finite type.
Used by
Markov factorization of the ambient #
InformationTheory.Shannon.isMarkovChain_of_compProd_encoder
sourceAbstract Markov-chain factorization M → g M → Y for an ambient ν ⊗ₘ κ in which the
message-to-output kernel κ factors through a deterministic encoder g : M → Z and a
codeword kernel Wcode : Z → Y (i.e. κ m = Wcode (g m)). This is the general shape behind
the message-to-codeword-to-output chain of a multi-user converse; it needs no product/pi
structure, only the factorization hκ.
@audit:ok
Used by
Marginalizing and reindexing a product measure #
InformationTheory.Shannon.lintegral_pi_reRandomize
sourceRe-randomizing a single coordinate of a product of probability measures leaves the
Measure.pi-integral unchanged. Used to peel the i-th output letter off the block channel
∏ⱼ W (xⱼ) in the memoryless-channel derivation.
@audit:ok
Used by
InformationTheory.Shannon.lintegral_pi_eval
sourceMarginalization of a product of probability measures at a single coordinate. @audit:ok
Used by
InformationTheory.Shannon.pi_map_comp_of_injective
sourceUsed by
Per-letter conditional independence, memorylessness and the joint law #
InformationTheory.Shannon.isMarkovChain_of_compProd_pi
sourcePer-letter conditional independence for a product-channel ambient: if the message-to-output
kernel factors as the per-letter product κ m = ∏ⱼ W (x m j), then at every letter i the
output ω.2 i is conditionally independent of F given the input letter x ω.1 i, for any
measurable F that does not read output coordinate i. Not reading that coordinate is what
hFupd says — re-randomizing it leaves F unchanged — so F may read the message, hence every
input letter, together with all the other output letters.
@audit:ok
Used by
InformationTheory.Shannon.isMemorylessChannel_of_compProd_pi
sourceA product-channel ambient is a memoryless channel: if the message-to-output kernel
factors as the per-letter product κ m = ∏ⱼ W (x m j) of a channel W applied to a
deterministic codeword x m, then ν ⊗ₘ κ is a memoryless channel with per-letter
inputs x ω.1 i and per-letter outputs ω.2 i. This is isMarkovChain_of_compProd_pi read
at the conditioner that collects every input and output letter other than the i-th.
@audit:ok
Used by
InformationTheory.Shannon.compProd_comap_map_prodMap
source@audit:ok
Used by
InformationTheory.Shannon.compProd_map_prodMap
sourceUsed by
InformationTheory.Shannon.compProd_pi_map_pair_eq_of_update_invariant
source@audit:ok
Used by
InformationTheory.Shannon.compProd_pi_map_pair_eq
sourcePer-letter joint pushforward of a product-channel compProd: for an ambient ν ⊗ₘ κ
whose message-to-output kernel factors as the per-letter product κ m = ∏ⱼ W (x m j), the
joint law of the i-th input-output pair (x ω.1 i, ω.2 i) is the channel joint
(ν.map fun m ↦ x m i) ⊗ₘ W. Stated separately from isMarkovChain_of_compProd_pi, which
establishes the same identity internally, because it is what identifies a per-letter
information quantity of the ambient with the corresponding channel quantity.
Used by
Unzipping a product of two-stage laws #
InformationTheory.Shannon.measure_singleton_eq_mul_of_append
sourceUsed by
InformationTheory.Shannon.piBlockKernel
sourceThe blockwise product u ↦ ∏ⱼ Q (u j) of a kernel Q out of a finite alphabet.
Mathlib has no Kernel.pi; over a countable alphabet with measurable singletons the family of
product measures is a kernel for free, which is what Kernel.ofFunOfCountable supplies here.
Equations
- InformationTheory.Shannon.piBlockKernel Q = ProbabilityTheory.Kernel.ofFunOfCountable fun (u : Fin k → A) => MeasureTheory.Measure.pi fun (j : Fin k) => Q (u j)
Instances For
Used by
InformationTheory.Shannon.piBlockKernel_isMarkovKernel
sourceUsed by
InformationTheory.Shannon.pi_map_unzip_eq_compProd
sourceUsed by
Information transport under a shared pushforward #
InformationTheory.Shannon.mutualInfo_map_comp
sourceMutual information is invariant under a shared pushforward of both random variables:
I(f; g) = I(f ∘ T; g ∘ T) when the pair law on μ.map T matches the pair law of the composed
variables on μ.
Used by
InformationTheory.Shannon.condDistrib_map_comp
sourcecondDistrib is stable under a shared pushforward of the conditioning and conditioned
variables: condDistrib f h (μ.map T) =ᵐ condDistrib (f ∘ T) (h ∘ T) μ on the conditioning
marginal (μ.map T).map h.
Used by
InformationTheory.Shannon.condMutualInfo_map_comp
sourceConditional mutual information is invariant under a shared pushforward of all three random
variables: I(f; g | h) = I(f ∘ T; g ∘ T | h ∘ T).
Used by
InformationTheory.Shannon.isMarkovChain_map_comp
sourceUsed by
InformationTheory.Shannon.condMutualInfo_map_comp'
sourcecondMutualInfo_map_comp phrased against any measure ρ propositionally equal to μ.map T.
The equation hypothesis is substituted (transporting its IsFiniteMeasure instance), which
sidesteps the ill-typed motive of rewriting the measure argument of condMutualInfo directly.
Used by
From a message count to a rate #
InformationTheory.Shannon.le_log_of_ceil_exp_le
sourceIf ⌈exp x⌉₊ ≤ M then x ≤ log M, converting a message count into a rate bound. From
exp x ≤ ⌈exp x⌉₊ ≤ M, taking logs (both sides positive) gives x = log (exp x) ≤ log M.