InformationTheory

InformationTheory.Shannon.FisherInfo.DeBruijnStandalone

source

de Bruijn identity — standalone headlines (Cover–Thomas) #

This file assembles the sorry-free parts already present in the project into clean, self-contained statements of the de Bruijn identity along the Gaussian heat flow:

  • the per-time identity (d/dt) h(X + √t·Z) = (1/2)·J(X + √t·Z), and
  • its integrated form h(X + √T·Z) − h(X) = ∫₀ᵀ (1/2)·J(X + √t·Z) dt.

The per-time analytic core is the existing assembly debruijnIdentityV2_holds_assembled (routed through deBruijn_identity_v2); the integrated form is the existing FTC assembly debruijnIntegrationIdentity_holds. The content of this file is the non-vacuity witness for IsDeBruijnPathRegular: a Gaussian inhabitant, and a general absolutely-continuous producer.

Main statements #

References #

[CoverThomas2006].

Density-supplied de Bruijn regularity #

noncomputable def

InformationTheory.Shannon.FisherInfo.isRegularDeBruijnHypV2_of_density

source
{Ω : Type u_1} {_mΩ : MeasurableSpace Ω} {P : MeasureTheory.Measure Ω} [MeasureTheory.IsProbabilityMeasure P] {X Z : Ω} (hZ_law : MeasureTheory.Measure.map Z P = ProbabilityTheory.gaussianReal 0 1) (pX : ) (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_law : MeasureTheory.Measure.map X P = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (pX x)) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) {t : } (ht : 0 < t) :

The V2 de Bruijn regularity bundle from an explicit Lebesgue density pX of X: a probability density (nonnegative, measurable) carrying the withDensity law pX_law and a finite second moment hpX_mom, together with the standard-normal law of Z. The density witness density_t is pinned to the smooth convolution convDensityAdd pX g_t, so density_t_eq is rfl.

@audit:ok

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    Used by

      The per-time de Bruijn identity #

      theorem

      InformationTheory.Shannon.FisherInfo.debruijn_identity_per_time

      source
      {Ω : Type u_1} {_mΩ : MeasurableSpace Ω} {P : MeasureTheory.Measure Ω} [MeasureTheory.IsProbabilityMeasure P] (X Z : Ω) (hX : Measurable X) (hZ : Measurable Z) (hXZ : ProbabilityTheory.IndepFun X Z P) (hZ_law : MeasureTheory.Measure.map Z P = ProbabilityTheory.gaussianReal 0 1) (pX : ) (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_law : MeasureTheory.Measure.map X P = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (pX x)) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) {t : } (ht : 0 < t) :

      de Bruijn identity (per-time, density form, Cover–Thomas).

      For X ⊥ Z with Z ∼ 𝒩(0, 1) and an explicit Lebesgue density pX of X (a probability density with finite second moment), the heat-flow entropy h(X + √s·Z) has, at every time t > 0, derivative (1/2)·J(X + √t·Z) with the V2 Fisher information of the smooth convolution density pX ∗ g_t.

      The class is non-vacuous: Gaussian X ∼ 𝒩(m, v) instantiates it with pX = gaussianPDFReal m v (see isDeBruijnPathRegular_gaussian), and every absolutely-continuous X with finite second moment supplies pX = (P.map X).rnDeriv volume (see isDeBruijnPathRegular_of_heat_flow).

      References: [CoverThomas2006].

      @audit:ok

      Used by

        Gaussian path-regularity #

        noncomputable def

        InformationTheory.Shannon.FisherInfo.isDeBruijnPathRegular_gaussian

        source

        The Gaussian inhabitant of IsDeBruijnPathRegular: for X ∼ 𝒩(m, v) (v ≠ 0), Z ∼ 𝒩(0, 1), X ⊥ Z, the heat-flow path is regular on [0, T]. The density witness path is fPath t = gaussianPDFReal m (v + t), whose Fisher information is the bounded continuous closed form 1 / (v + t), so the path integrand is interval-integrable; the heat-flow entropy is the closed form (1/2)·log(2π e (v + s)), continuous on [0, T].

        This witness shows IsDeBruijnPathRegular (and hence debruijnIntegrationIdentity_holds) is non-vacuous.

        @audit:ok

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          Used by
            theorem

            InformationTheory.Shannon.FisherInfo.debruijn_identity_integrated_gaussian

            source
            {Ω : Type u_1} {_mΩ : MeasurableSpace Ω} {P : MeasureTheory.Measure Ω} [MeasureTheory.IsProbabilityMeasure P] (X Z : Ω) (hX : Measurable X) (hZ : Measurable Z) (hXZ : ProbabilityTheory.IndepFun X Z P) {m : } {v : NNReal} (hv : v 0) (hX_law : MeasureTheory.Measure.map X P = ProbabilityTheory.gaussianReal m v) (hZ_law : MeasureTheory.Measure.map Z P = ProbabilityTheory.gaussianReal 0 1) (T : ) (hT : 0 T) :
            ∃ (fPath : ), ∀ (h_X h_target : ), h_X = differentialEntropy (MeasureTheory.Measure.map X P)h_target = differentialEntropy (MeasureTheory.Measure.map (gaussianConvolution X Z T) P)h_target - h_X = (t : ) in Set.Ioo 0 T, 1 / 2 * (fisherInfoOfMeasureV2 (MeasureTheory.Measure.map (gaussianConvolution X Z t) P) (fPath t)).toReal

            Integrated de Bruijn identity (gaussian, Cover–Thomas). The integrated form h(X + √T·Z) − h(X) = ∫₀ᵀ (1/2)·J(X + √t·Z) dt for a Gaussian X ∼ 𝒩(m, v), obtained by applying debruijnIntegrationIdentity_holds to the Gaussian path-regularity witness.

            References: [CoverThomas2006].

            @audit:ok

            Used by

              General absolutely-continuous producer #

              noncomputable def

              InformationTheory.Shannon.FisherInfo.isDeBruijnPathRegular_of_heat_flow

              source
              {Ω : Type u_1} {_mΩ : MeasurableSpace Ω} {P : MeasureTheory.Measure Ω} [MeasureTheory.IsProbabilityMeasure P] (X Z : Ω) (hX : Measurable X) (hZ : Measurable Z) (hXZ : ProbabilityTheory.IndepFun X Z P) (hZ_law : MeasureTheory.Measure.map Z P = ProbabilityTheory.gaussianReal 0 1) (pX : ) (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_law : MeasureTheory.Measure.map X P = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (pX x)) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mass : (y : ), pX y = 1) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) (hpX_ent : MeasureTheory.Integrable (fun (x : ) => (pX x).negMulLog) MeasureTheory.volume) (T : ) :

              The general absolutely-continuous producer of IsDeBruijnPathRegular. For X ⊥ Z with Z ∼ 𝒩(0, 1) and an explicit probability density pX of X with finite second moment and finite differential entropy (hpX_ent), the heat-flow path is regular on [0, T]. The per-time regularity (reg_t) is the density bundle isRegularDeBruijnHypV2_of_density; the heat-flow entropy continuity (cont) combines the endpoint continuity heatFlowDifferentialEntropy_continuousWithinAt_zero (at t = 0⁺) with the interior continuity from the per-time de Bruijn HasDerivAt.

              The interval-integrability of the path integrand (1/2)·J(X + √t·Z) (integrable) is closed by debruijnHeatPath_intervalIntegrable: the integrand is nonnegative and, by the convolution bound J(pX ∗ g_t) ≤ 1/t (gaussianConv_fisher_le_inv_var), bounded on each [ε, T] (ε > 0); there it equals the derivative of the heat-flow entropy, so the FTC gives ∫_ε^T = h(X + √T·Z) − h(X + √ε·Z), and the G2 endpoint continuity (ε ↓ 0) bounds these uniformly, upgrading to integrability on (0, T) via integrableOn_Ioc_of_intervalIntegral_norm_bounded_left. This routes only through the per-time de Bruijn identity, never its integrated form, so it is non-circular.

              @audit:ok

              Equations
              • One or more equations did not get rendered due to their size.
              Instances For
                Used by
                  theorem

                  InformationTheory.Shannon.FisherInfo.debruijn_identity_integrated

                  source
                  {Ω : Type u_1} {_mΩ : MeasurableSpace Ω} {P : MeasureTheory.Measure Ω} [MeasureTheory.IsProbabilityMeasure P] (X Z : Ω) (hX : Measurable X) (hZ : Measurable Z) (hXZ : ProbabilityTheory.IndepFun X Z P) (hZ_law : MeasureTheory.Measure.map Z P = ProbabilityTheory.gaussianReal 0 1) (pX : ) (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_law : MeasureTheory.Measure.map X P = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (pX x)) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mass : (y : ), pX y = 1) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) (hpX_ent : MeasureTheory.Integrable (fun (x : ) => (pX x).negMulLog) MeasureTheory.volume) (T : ) (hT : 0 T) :
                  ∃ (fPath : ), ∀ (h_X h_target : ), h_X = differentialEntropy (MeasureTheory.Measure.map X P)h_target = differentialEntropy (MeasureTheory.Measure.map (gaussianConvolution X Z T) P)h_target - h_X = (t : ) in Set.Ioo 0 T, 1 / 2 * (fisherInfoOfMeasureV2 (MeasureTheory.Measure.map (gaussianConvolution X Z t) P) (fPath t)).toReal

                  Integrated de Bruijn identity (general a.c. Cover–Thomas). The integrated form h(X + √T·Z) − h(X) = ∫₀ᵀ (1/2)·J(X + √t·Z) dt for a general absolutely-continuous X, obtained by applying debruijnIntegrationIdentity_holds to the general path-regularity producer isDeBruijnPathRegular_of_heat_flow.

                  References: [CoverThomas2006].

                  @audit:ok

                  Used by