InformationTheory.Shannon.EPI.G2.ConvEntropyMonotone
EPI G2 — (β) Convolution does not decrease differential entropy #
This file supplies the lower bound h(pX) ≤ h(pX ∗ g_t) of the EPI G2 general
sandwich: the differential entropy of a Gaussian-smoothed density convDensityAdd pX g_t
(with g_t = gaussianPDFReal 0 ⟨t,_⟩) is at least the differential entropy of pX.
The mathematical route is the continuous conditioning-reduces-entropy
inequality, applied to W := X + √t·Z with Z ⊥ X a Gaussian:
h(X + √t·Z) ≥ h(X + √t·Z | Z) = h(X).
- The first
≥is conditioning-reduces-entropy,h(W|Z) ≤ h(W)(condDifferentialEntropy_le), which is the differential analogue ofI(W;Z) = h(W) − h(W|Z) = KL(joint ‖ product) ≥ 0. - The equality
h(X + √t·Z | Z) = h(X)is the independent-sum fiber identification (condDifferentialEntropy_indep_add_eq): conditioned onZ = z, the variableX + √t·Zis the constant shiftX + √t·z, whose entropy equalsh(X)by translation invariance (differentialEntropy_map_add_const).
condDistrib is the regular conditional probability distribution from Mathlib
(ProbabilityTheory.condDistrib X Z μ = conditional law of X given Z), so the
conditional differential entropy is defined Mathlib-shape:
condDifferentialEntropy X Z μ := ∫ z, differentialEntropy ((condDistrib X Z μ) z) ∂(μ.map Z).
This is a reusable, EPI-line-wide asset. Continuous conditional differential
entropy + conditioning-reduces-entropy are absent from Mathlib (genuine gap, not a
wall), but the condDistrib machinery exists, so a construction is possible.
Assembly of the bridge #
The bridge differentialEntropy_sub_condDifferentialEntropy_eq_toReal_klDiv is assembled
from three components:
- (a)
InformationTheory.klDiv_compProd_const_toReal_integral(CondKLIntegral.lean) turns the joint KLtoRealinto theμ_Z-average of fiberwise KL∫ z, (klDiv (κ z) μ_X).toReal ∂μ_Z; - (b)
klDiv_toReal_eq_neg_differentialEntropy_sub_cross(BridgeDensityHelpers.lean) expands each fiber into−h(κ z) − ∫ p_z · log q_X; - (c)
integral_condDistrib_density_marginal_eq(BridgeDensityHelpers.lean) identifies theμ_Z-average of the cross term with∫ q_X · log q_X = −h(μ_X)(Fubini marginal).
Assembling (a)+(b)+(c) gives RHS = −h(X|Z) − (−h(X)) = h(X) − h(X|Z), the bridge.
condDifferentialEntropy_le then follows by ENNReal.toReal_nonneg. The regularity /
integrability preconditions (joint ≪, per-fiber ≪, the μ_Z-integrability of the
fiber entropy and the cross term, the marginal log-density integrability) are threaded
through as honest preconditions (absolute continuity / KL finiteness), not load-bearing
bundles; the downstream device form differentialEntropy_indep_gaussian_add_ge and
density form negMulLog_convDensity_entropy_ge thread them at the heat-flow path
W := X + √s·Z with conclusions unchanged.
condDifferentialEntropy_indep_add_eq (independent-sum fiber identification) is
obtained via the affine-shift kernel affineShiftKernel.
InformationTheory.Shannon.condDifferentialEntropy
sourceThe continuous conditional differential entropy h(X | Z). Defined directly on
the Mathlib regular conditional distribution condDistrib X Z μ (the conditional law
of X given Z, a Kernel α ℝ): the fiber differential entropy
differentialEntropy ((condDistrib X Z μ) z) averaged over the law μ.map Z of Z.
Mathlib-shape: the textbook ∫_z h(X | Z = z) dμ_Z(z) is realized through the
condDistrib disintegration so that compProd_map_condDistrib and
differentialEntropy_map_add_const are usable verbatim.
Equations
Instances For
Used by
InformationTheory.Shannon.differentialEntropy_sub_condDifferentialEntropy_eq_toReal_klDiv
sourceDifferential mutual information expressed as a Kullback-Leibler divergence:
h(X) − h(X | Z) = KL(joint ‖ product).toReal,
where joint := (μ.map Z) ⊗ₘ condDistrib X Z μ (the law of (Z, X), by
compProd_map_condDistrib) and product := (μ.map Z) ⊗ₘ Kernel.const _ (μ.map X)
(= (μ.map Z).prod (μ.map X), by Measure.compProd_const). This is the
differential-entropy-level statement of I(X;Z) = h(X) − h(X|Z).
Assembled from three components, with the regularity / integrability hypotheses threaded as honest preconditions:
- (a)
InformationTheory.klDiv_compProd_const_toReal_integral(CondKLIntegral.lean) turns the joint KLtoRealinto theμ_Z-average of the fiberwise KL∫ z, (klDiv (κ z) μ_X).toReal ∂μ_Z; - (b)
klDiv_toReal_eq_neg_differentialEntropy_sub_cross(BridgeDensityHelpers.lean) expands each fiber into−h(κ z) − ∫ p_z · log q_X; - (c)
integral_condDistrib_density_marginal_eq(BridgeDensityHelpers.lean) identifies∫_z ∫ p_z log q_X ∂μ_Z = ∫ q_X log q_X = −h(μ_X)(Fubini marginal).
Assembling: RHS = −h(X|Z) − (−h(X)) = h(X) − h(X|Z). The classical continuous
I(X;Z) = h(X) − h(X|Z) = D(P_{Z,X} ‖ P_Z ⊗ P_X) identity.
All added hypotheses are regularity preconditions (absolute continuity, per-fiber
absolute continuity, equal mass via Markov, integrability = KL finiteness), not
load-bearing bundles. The equal-mass condition κ z univ = μ_X univ is discharged
internally (both probability measures). @audit:ok
Used by
InformationTheory.Shannon.condDifferentialEntropy_le
sourceConditioning reduces the differential entropy: h(X | Z) ≤ h(X).
The differential analogue of I(X;Z) = h(X) − h(X|Z) = KL(joint ‖ product) ≥ 0.
The bridge
differentialEntropy_sub_condDifferentialEntropy_eq_toReal_klDiv identifies the entropy
difference with (klDiv joint product).toReal, whose non-negativity is type-trivial
(ENNReal.toReal_nonneg, since klDiv is ℝ≥0∞-valued), so the
conditioning-reduces-entropy inequality follows by sub_nonneg.
The hypotheses are all preconditions (regularity / absolute continuity / integrability =
KL finiteness), not load-bearing: hX_ac : μ.map X ≪ volume ensures h(X) reflects the
density, measurability is structural, and the bridge's regularity / integrability
preconditions are threaded through unchanged. @audit:ok
Used by
InformationTheory.Shannon.affineShiftKernel
sourceThe z-dependent affine-shift kernel κ z := νX.map (· + c·z), built as a
Kernel ℝ ℝ. Construction: push the parametrized pairing z ↦ νX.map (Prod.mk z)
(measurable by Measurable.map_prodMk_left) through the measurable affine map
(z, x) ↦ x + c·z.
@audit:ok
Equations
- InformationTheory.Shannon.affineShiftKernel νX c = { toFun := fun (z : ℝ) => MeasureTheory.Measure.map (fun (x : ℝ) => x + c * z) νX, measurable' := ⋯ }
Instances For
Used by
InformationTheory.Shannon.affineShiftKernel_apply
sourceUsed by
InformationTheory.Shannon.affineShiftKernel.instIsMarkov
sourceUsed by
InformationTheory.Shannon.prod_map_affine_eq_compProd
sourcePlumbing core (buildable, not a Mathlib wall): the pushforward of the product
measure νZ ⊗ νX through the affine map g (z, x) = (z, x + c·z) equals the composition
product of νZ with the z-dependent affine-shift kernel affineShiftKernel νX c.
@audit:ok
Used by
InformationTheory.Shannon.condDifferentialEntropy_indep_add_eq
sourceFiber identification for an independent sum: for X ⊥ Z,
h(X + c·Z | Z) = h(X).
Conditioned on Z = z, the variable fun ω ↦ X ω + c · Z ω is the constant shift
X + c·z, whose differential entropy equals h(X) by translation invariance
(differentialEntropy_map_add_const). Averaging the constant h(X) over the
probability law μ.map Z reproduces h(X).
The fiber identification
condDistrib (X + c·Z) Z μ =ᵐ[μ.map Z] affineShiftKernel (μ.map X) c is assembled
via:
indepFun_iff_map_prod_eq_prod_map_mapgivesμ.map (fun ω ↦ (Z ω, X ω)) = (μ.map Z).prod (μ.map X)(independence).- Push the product through the affine map
g (z, x) = (z, x + c·z)and identify it with(μ.map Z) ⊗ₘ (affineShiftKernel (μ.map X) c)(prod_map_affine_eq_compProd, the z-dependent affine-shift kernel built above). condDistrib_ae_eq_of_measure_eq_compProdthen gives the fiber identification, anddifferentialEntropy_map_add_constdischarges each fiber toh(μ.map X).
The hypotheses are all preconditions: IndepFun X Z μ is a genuine independence
precondition (not a load-bearing bundle), hX_ac is absolute continuity, measurability
is structural.
@audit:ok
Used by
InformationTheory.Shannon.differentialEntropy_indep_gaussian_add_ge
sourceThe (β) device form: convolution does not decrease differential entropy,
stated through an underlying independent pair X ⊥ Z with Z Gaussian.
h(X) ≤ h(X + √s·Z) via the chain h(X) = h(X+√s·Z | Z) ≤ h(X+√s·Z)
(condDifferentialEntropy_indep_add_eq + condDifferentialEntropy_le).
All hypotheses are regularity preconditions (the fields of
IsHeatFlowEndpointRegular): measurability, independence, the noise law, and the
absolute continuity of μ.map X. The bridge regularity / integrability preconditions
(stated at the heat-flow path W := X + √s·Z) are threaded through unchanged.
@audit:ok
Used by
InformationTheory.Shannon.negMulLog_convDensity_entropy_ge
sourceThe (β) density form: the target consumed by the EPI G2 sandwich layer-2.
Convolution with a Gaussian does not decrease the negMulLog entropy integral:
∫ negMulLog pX ≤ ∫ negMulLog (pX ∗ g_{u n}). Equivalently h(pX) ≤ h(pX ∗ g_t).
The underlying independent pair X ⊥ Z (with Z ∼ 𝒩(0, v_Z), s·v_Z = u n) is
supplied as regularity preconditions, matching the fields of
IsHeatFlowEndpointRegular. pX is identified with the density of μ.map X.
The bridge regularity / integrability preconditions (stated at the heat-flow path
W := X + √s·Z with s := u n / v_Z) are threaded through unchanged.
@audit:ok