InformationTheory.Shannon.AWGN.KLCapacityAndAEP
Continuous Gaussian AEP and per-letter / n-fold KL identities #
The KL-divergence identities for the AWGN channel together with the continuous Gaussian asymptotic equipartition property. These feed the achievability side of the AWGN channel coding theorem.
Main statements #
klDiv_perLetter_eq_capacity— per-letter KL divergence equals(1/2) log(1+P/N).klDiv_nFold_eq_nsmul—klDiv(Jₙ,Qₙ).toReal = n · klDiv(J₁,Q₁).toReal.continuousAepGaussian_holds— the continuous Gaussian AEP exponent bound.
Continuous Gaussian AEP #
InformationTheory.Shannon.AWGN.pi_withDensity
sourceTensorization of withDensity through Measure.pi (G-2, absent from Mathlib): the
finite product of per-coordinate weighted measures (μ i).withDensity (f i) equals the product
measure Measure.pi μ weighted by the coordinatewise product density x ↦ ∏ i, f i (x i).
Proved via Measure.pi_eq (agreement on measurable boxes) + the box factorization
lintegral_pi_prod_eq_prod.
Used by
Per-letter AWGN KL closed form and n-fold identity #
Two shared lemmas consumed by the achievability development (continuousAepGaussian_holds
below and awgn_random_coding_union_bound in AchievabilityTypicalDecoder.lean).
klDiv_perLetter_eq_capacity: the per-letter jointJ₁/productQ₁KL equals the AWGN capacity(1/2) log(1 + P/N). Routed through the conditional-KL integral (klDiv_compProd_const_toReal_integral,CondKLIntegral.lean) + the 1-D Gaussian KL closed form (klDiv_gaussianReal_gaussianReal_eq,DifferentialEntropy.lean), avoidingmutualInfoOfChannel/MIClosedForm.leanwhich would create the import cycleKLCapacityAndAEP → MIClosedForm → ContChannelMIDecomp → KLCapacityAndAEP.klDiv_nFold_eq_nsmul:klDiv(J_n,Q_n).toReal = n · klDiv(J₁,Q₁).toReal, via thearrowProdEquivProdArrowreshape (klDiv_map_measurableEquiv) +klDiv_pi_eq_sum- i.i.d.
Finset.sum_constcollapse.
- i.i.d.
InformationTheory.Shannon.AWGN.awgn_perLetter_changeOfMeasure_facts
sourcePer-letter change-of-measure facts for the AWGN joint J₁ = law(X, X+Z) /
product Q₁ = μX ⊗ μY (nondegenerate P', N ≠ 0): the mutual absolute continuity
J₁ ≪ Q₁, Q₁ ≪ J₁, and the pointwise density relation
(Q₁.rnDeriv J₁ p).toReal = exp(−φ p) a.e. [J₁] (where φ = log dJ₁/dQ₁). The f_X
factor in the density ratio cancels, so the ratio is a ratio of two strictly-positive-and-
finite Gaussian densities, giving the exponential relation; mutual AC follows from the
everywhere-positive-and-finite densities w.r.t. volume.prod volume.
Used by
InformationTheory.Shannon.AWGN.awgn_perLetter_klDiv_degenerate
sourceDegenerate per-letter KL vanishes (P' = 0 ∨ N = 0): in either degenerate case
the per-letter KL (klDiv J₁ Q₁).toReal = 0. When P' = 0 the input collapses to Dirac 0,
the shear is the identity on {0} × ℝ, so J₁ = Q₁ (klDiv_self). When N = 0
(with P' ≠ 0) the joint J₁ concentrates on the diagonal {(x, x)}, which is Q₁-null
(positive-variance Gaussian product, atomless), so ¬ J₁ ≪ Q₁ ⇒ klDiv = ⊤ ⇒ toReal = 0.
Used by
InformationTheory.Shannon.AWGN.klDiv_perLetter_eq_capacity
sourcebridge ① per-letter closed form (genuine, sorryAx-free): per-letter joint
J₁ = law(X, X+Z) and product of marginals Q₁ = μX ⊗ μY have KL equal to the AWGN
per-letter capacity (1/2) log(1 + P/N) (nondegenerate P > 0, N ≠ 0). Routed through the
conditional-KL integral (klDiv_compProd_const_toReal_integral) + the 1-D Gaussian KL closed
form (klDiv_gaussianReal_gaussianReal_eq), integrating the per-fibre quadratic against the
mean-0 variance-P' input — deliberately avoiding mutualInfoOfChannel / MIClosedForm
(import cycle KLCapacityAndAEP → MIClosedForm → ContChannelMIDecomp → KLCapacityAndAEP).
The signature carries the genuine preconditions
0 < P / (N:ℝ) ≠ 0 (the union-bound consumer derives both before invoking); no
circularity / bundling / degenerate-def.
@audit:ok
Used by
InformationTheory.Shannon.AWGN.klDiv_nFold_eq_nsmul
sourcebridge ① n-fold identity (genuine, sorryAx-free):
klDiv(J_n, Q_n).toReal = n · klDiv(J₁, Q₁).toReal, where J_n/Q_n are the verbatim
n-letter joint/product measures from the continuousAepGaussian_holds signature. Via
arrowProdEquivProdArrow reshape (klDiv_map_measurableEquiv) + klDiv_pi_eq_sum
- i.i.d.
Finset.sum_const.
Unconditional measure identity (no P/N
precondition — holds even in the degenerate cases, both sides equal); no circularity /
bundling / degenerate-def.
@audit:ok
Used by
InformationTheory.Shannon.AWGN.gaussian_shear_logRnDeriv_memLp_two
sourceUsed by
InformationTheory.Shannon.AWGN.continuousAepGaussian_degenerate_witness
sourceUsed by
InformationTheory.Shannon.AWGN.awgn_joint_law_reshape_eq
sourceUsed by
InformationTheory.Shannon.AWGN.awgn_changeOfMeasure_pi_mass_le
sourceUsed by
InformationTheory.Shannon.AWGN.continuousAepGaussian_holds
sourceContinuous AEP for the n-dimensional Gaussian.
Given P : ℝ, N : ℝ≥0, a typicality slack δ > 0, and an error tolerance ε > 0
(independent parameters), there exists a threshold N₀ such that for every n ≥ N₀
there is a measurable typical set A ⊆ (Fin n → ℝ) × (Fin n → ℝ) satisfying:
- (i) joint codebook+noise mass
≥ 1 - ε, under the joint law of(X, Y)withX ∼ N(0,P)i.i.d. andY = X + Z,Z ∼ N(0,N)i.i.d.; - (ii) independent-pair upper bound (
X'independent ofY): under the product of marginals,Ahas mass≤ exp(−(klDivₙ − n·3δ)) = exp(−n(I − 3δ)), whereklDivₙ = klDiv(joint, product) = n·Iis then-letter KL (per-letter MII).
The slack δ controls the typical set's width; the error target ε controls the
mass-failure level of (i). Decoupling them lets the consumer pick R + 3δ < I
independently of ε, which makes the union bound's second term decay.
Implementation:
- (i) uses only the finite-
nChebyshev weak law (pi_empirical_mean_concentration/pi_empirical_mean_typical_mass). The typical setAis built from the per-letter joint info-densityφ(x,y) = log dJ₁/dQ₁. - (ii) is discharged by the tensorize
pi_withDensity(built on the lintegral box-Fubinilintegral_pi_prod_eq_prod): withQ₁ ≪ J₁,pi Q₁ = (pi J₁).withDensity (∏ᵢ Q₁.rnDeriv J₁ (wᵢ)), and onBthe product density(∏ᵢ Q₁.rnDeriv J₁ (wᵢ)).toReal = exp(−∑φ) ≤ exp(−n(I − δ))(awgn_perLetter_changeOfMeasure_facts,J₁[φ] = (klDiv J₁ Q₁).toRealviatoReal_klDiv_of_measure_eq);klDivₙ = n·IisklDiv_nFold_eq_nsmul. - The degenerate case
(klDiv Jn Qn).toReal = 0forP.toNNReal = 0 ∨ N = 0is handled viaklDiv_nFold_eq_nsmul+awgn_perLetter_klDiv_degenerate:P'=0givesJ₁ = Q₁(shear identity on{0}×ℝ,klDiv_self);N=0 ∧ P'≠0putsJ₁on the diagonal{(x,x)}(Q₁-null sinceμXis atomless), so¬J₁≪Q₁andklDiv = ⊤,toReal = 0. The exponentn·3δ ≥ 0makes the boundexp(n·3δ) ≥ 1 ≥ Qn univ. The degenerateA := Set.univbranch is an honest witness: bound (ii) is genuinely loose (RHS≥ 1) whenklDiv = 0, which is machine-proved rather than asserted. @audit:ok