InformationTheory

InformationTheory.Shannon.HypercubeEdge.BoundarySharp

source

Hypercube edge-boundary entropy-sharp inequality #

The entropy-sharp isoperimetric inequality for the edge boundary on the Boolean cube Fin n → Bool:

|A| · (n − log₂ |A|) ≤ |∂_e A|,

which is sharper than the AM-GM form edgeBoundary_ge_AMGM.

Main statements #

Implementation notes #

The conditional entropy condEntropy_coord_eq is computed via the chain rule H(X_{≠i}, Xᵢ) = H(X_{≠i}) + H(Xᵢ | X_{≠i}), evaluating H(X_{≠i}) directly from the projection fibre masses, which avoids a pointwise condDistrib Bernoulli computation.

Uniform measure on A #

The conditional entropy condEntropy_coord_eq #

The direction-i conditional entropy is computed via the chain rule H(X_{≠i}, Xᵢ) = H(X_{≠i}) + H(Xᵢ | X_{≠i}): the joint entropy is log |A|, and H(X_{≠i}) is computed directly from the projection fibre masses c_i(y) / |A|.

theorem

InformationTheory.Shannon.condEntropy_coord_eq

source
{n : } {A : Finset (Fin nBool)} (hA : A.Nonempty) (i : Fin n) :
(MeasureFano.condEntropy (ProbabilityTheory.uniformOn A) (fun (ω : Fin nBool) => ω i) fun (ω : Fin nBool) (j : { j : Fin n // j i }) => ω j) = 2 * (A.card - (projectionExcept i A).card) / A.card * Real.log 2

The direction-i conditional entropy under μ_A: condEntropy μ_A (Xs i) X_{≠i} = (2 (|A| − |π_{≠i}(A)|) / |A|) · log 2.

Used by

    Chain rule and conditioning monotonicity #

    theorem

    InformationTheory.Shannon.sum_condEntropy_le_log_card

    source
    {n : } {A : Finset (Fin nBool)} (hA : A.Nonempty) :
    (∑ i : Fin n, MeasureFano.condEntropy (ProbabilityTheory.uniformOn A) (fun (ω : Fin nBool) => ω i) fun (ω : Fin nBool) (j : { j : Fin n // j i }) => ω j) Real.log A.card

    The sum of the direction-wise conditional entropies is bounded by log |A|: ∑ i, H(Xᵢ | X_{≠i}) ≤ log |A|.

    Used by

      Main theorem #

      theorem

      InformationTheory.Shannon.edgeBoundary_entropy_sharp

      source
      {n : } {A : Finset (Fin nBool)} (hA : A.Nonempty) :
      A.card * (n - Real.logb 2 A.card) (edgeBoundaryCount A)

      The entropy-sharp edge-isoperimetric inequality: for nonempty A ⊆ Fin n → Bool, |A| · (n − log₂ |A|) ≤ |∂_e A|.

      Used by