InformationTheory

InformationTheory.Shannon.FisherInfo.DeBruijnPerTime

source

Per-time de Bruijn identity — analytic-core atoms #

The analytic-core atoms of the per-time de Bruijn identity for a general X, decomposed for the assembly debruijnIdentityV2_holds_assembled in FisherInfo/DeBruijnAssembly/Assembly.lean.

Main statements #

Law factorization #

theorem

InformationTheory.Shannon.FisherInfo.gaussianConvolution_law_conv

source

The law of the heat-flow path X + √s · Z factors as the convolution of the law of X with 𝒩(0, s · v_Z) when Z ∼ 𝒩(0, v_Z): P.map (gaussianConvolution X Z s) = (P.map X) ∗ gaussianReal 0 ⟨s · v_Z, _⟩. Holds for arbitrary X (no density witness needed). Via √s · Z ∼ 𝒩(0, s · v_Z) (gaussianReal_map_const_mul) and IndepFun.map_add_eq_map_conv_map.

@audit:ok

Used by

    Density identification #

    theorem

    InformationTheory.Shannon.FisherInfo.pPath_eq_convDensityAdd

    source
    {Ω : Type u_1} {_mΩ : MeasurableSpace Ω} {P : MeasureTheory.Measure Ω} [MeasureTheory.IsProbabilityMeasure P] (X Z : Ω) (hX : Measurable X) (hZ : Measurable Z) (hXZ : ProbabilityTheory.IndepFun X Z P) (v_Z : NNReal) (hv_Z_pos : 0 < v_Z) (hZ_law : MeasureTheory.Measure.map Z P = ProbabilityTheory.gaussianReal 0 v_Z) (pX : ) (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_law : MeasureTheory.Measure.map X P = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (pX x)) {s : } (hs : 0 < s) :

    When P.map X has a real density witness pX and Z ∼ 𝒩(0, v_Z) with v_Z > 0, the density of the heat-flow path X + √s · Z is a.e. equal to convDensityAdd pX (gaussianPDFReal 0 ⟨s · v_Z, _⟩). The chain composes gaussianConvolution_law_conv (law factorization), gaussianReal_of_var_ne_zero, conv_withDensity_eq_lconvolution, and the bridge pPath_eq_convDensityAdd_lconvolution_bridge. The general v_Z is needed for the sum instance (X+Y, Z_X+Z_Y), whose noise has variance 2.

    @audit:ok

    Used by

      Heat equation per density #

      noncomputable def

      InformationTheory.Shannon.FisherInfo.heatFlow_density_heat_equation_kernel

      source
      (σ u : ) :

      The heat kernel g(σ, u) = (√(2πσ))⁻¹ · exp(-u²/(2σ)) with σ ranging over . Used so the σ-derivative can be taken over a real neighborhood (the coercion ⟨σ, _⟩ : ℝ≥0 cannot be formed for σ < 0); it agrees with gaussianPDFReal 0 ⟨σ, _⟩ for σ > 0 (heatFlow_density_heat_equation_kernel_eq).

      Equations
      Instances For
        Used by
          theorem

          InformationTheory.Shannon.FisherInfo.heatFlow_density_heat_equation_kernel_eq

          source

          The kernel agrees with gaussianPDFReal 0 ⟨σ, _⟩ for σ > 0.

          @audit:ok

          Used by
            theorem

            InformationTheory.Shannon.FisherInfo.heatFlow_density_heat_equation_kernel_x_deriv1

            source
            {σ : } ( : 0 < σ) (u : ) :

            The spatial first derivative of the Gaussian heat kernel: ∂_u g_σ(u) = g_σ(u) · (-(u/σ)) for σ > 0.

            @audit:ok

            Used by
              theorem

              InformationTheory.Shannon.FisherInfo.heatFlow_density_heat_equation_kernel_x_deriv2

              source
              {σ : } ( : 0 < σ) (u : ) :
              HasDerivAt (fun (ξ : ) => heatFlow_density_heat_equation_kernel σ ξ * -(ξ / σ)) (heatFlow_density_heat_equation_kernel σ u * (u ^ 2 / σ ^ 2 - 1 / σ)) u

              The spatial second derivative of the Gaussian heat kernel: ∂²_u g_σ(u) = g_σ(u) · (u²/σ² - 1/σ).

              @audit:ok

              Used by
                theorem

                InformationTheory.Shannon.FisherInfo.heatFlow_density_heat_equation_kernel_sigma_deriv

                source
                {σ : } ( : 0 < σ) (u : ) :
                HasDerivAt (fun (τ : ) => heatFlow_density_heat_equation_kernel τ u) (1 / 2 * (heatFlow_density_heat_equation_kernel σ u * (u ^ 2 / σ ^ 2 - 1 / σ))) σ

                The σ-derivative of the Gaussian heat kernel: ∂_σ g_σ(u) = (1/2) · g_σ(u) · (u²/σ² - 1/σ).

                @audit:ok

                Used by
                  theorem

                  InformationTheory.Shannon.FisherInfo.heatFlow_density_heat_equation_kernel_heat_eq

                  source
                  {σ : } ( : 0 < σ) (u : ) :
                  HasDerivAt (fun (τ : ) => heatFlow_density_heat_equation_kernel τ u) (1 / 2 * (heatFlow_density_heat_equation_kernel σ u * (u ^ 2 / σ ^ 2 - 1 / σ))) σ HasDerivAt (fun (ξ : ) => heatFlow_density_heat_equation_kernel σ ξ * -(ξ / σ)) (heatFlow_density_heat_equation_kernel σ u * (u ^ 2 / σ ^ 2 - 1 / σ)) u

                  The Gaussian heat kernel solves the heat equation ∂_σ g_σ(u) = (1/2) · ∂²_u g_σ(u); both sides equal (1/2) · g_σ(u) · (u²/σ² - 1/σ).

                  @audit:ok

                  Used by
                    theorem

                    InformationTheory.Shannon.FisherInfo.heatFlow_density_heat_equation

                    source
                    (pX : ) (pPath pathDeriv1 pathDeriv2 : ) (hpPath : ∀ (σ : ) ( : 0 < σ), pPath σ = EPIConvDensity.convDensityAdd pX (ProbabilityTheory.gaussianPDFReal 0 σ, )) (hpathDeriv1 : ∀ (σ y : ), HasDerivAt (fun (ξ : ) => pPath σ ξ) (pathDeriv1 σ y) y) (hpathDeriv2 : ∀ (σ y : ), HasDerivAt (fun (ξ : ) => pathDeriv1 σ ξ) (pathDeriv2 σ y) y) {s : } (hs : 0 < s) (x : ) (boundσ : ) (hboundσ_int : MeasureTheory.Integrable boundσ MeasureTheory.volume) (hFσ_meas : ∀ᶠ (σ : ) in nhds s, MeasureTheory.AEStronglyMeasurable (fun (y : ) => pX y * heatFlow_density_heat_equation_kernel σ (x - y)) MeasureTheory.volume) (hFσ_int : MeasureTheory.Integrable (fun (y : ) => pX y * heatFlow_density_heat_equation_kernel s (x - y)) MeasureTheory.volume) (hFσ'_meas : MeasureTheory.AEStronglyMeasurable (fun (y : ) => pX y * (1 / 2 * (heatFlow_density_heat_equation_kernel s (x - y) * ((x - y) ^ 2 / s ^ 2 - 1 / s)))) MeasureTheory.volume) (hbσ : ∀ᵐ (y : ), σSet.Ioo (s / 2) (2 * s), pX y * (1 / 2 * (heatFlow_density_heat_equation_kernel σ (x - y) * ((x - y) ^ 2 / σ ^ 2 - 1 / σ))) boundσ y) (boundξ1 : ) (hboundξ1_int : MeasureTheory.Integrable boundξ1 MeasureTheory.volume) (hFξ1_meas : ∀ (ξ : ), MeasureTheory.AEStronglyMeasurable (fun (y : ) => pX y * heatFlow_density_heat_equation_kernel s (ξ - y)) MeasureTheory.volume) (hFξ1_int : ∀ (ξ : ), MeasureTheory.Integrable (fun (y : ) => pX y * heatFlow_density_heat_equation_kernel s (ξ - y)) MeasureTheory.volume) (hFξ1'_meas : ∀ (ξ : ), MeasureTheory.AEStronglyMeasurable (fun (y : ) => pX y * (heatFlow_density_heat_equation_kernel s (ξ - y) * -((ξ - y) / s))) MeasureTheory.volume) (hbξ1 : ∀ᵐ (y : ), ξSet.univ, pX y * (heatFlow_density_heat_equation_kernel s (ξ - y) * -((ξ - y) / s)) boundξ1 y) (boundξ2 : ) (hboundξ2_int : MeasureTheory.Integrable boundξ2 MeasureTheory.volume) (hFξ2_int : MeasureTheory.Integrable (fun (y : ) => pX y * (heatFlow_density_heat_equation_kernel s (x - y) * -((x - y) / s))) MeasureTheory.volume) (hFξ2'_meas : MeasureTheory.AEStronglyMeasurable (fun (y : ) => pX y * (heatFlow_density_heat_equation_kernel s (x - y) * ((x - y) ^ 2 / s ^ 2 - 1 / s))) MeasureTheory.volume) (hbξ2 : ∀ᵐ (y : ), ξSet.univ, pX y * (heatFlow_density_heat_equation_kernel s (ξ - y) * ((ξ - y) ^ 2 / s ^ 2 - 1 / s)) boundξ2 y) :
                    HasDerivAt (fun (σ : ) => pPath σ x) (1 / 2 * pathDeriv2 s x) s

                    The heat-flow density satisfies the heat equation: ∂_σ pPath σ x = (1/2) ∂²_x pPath σ x at σ = s. Here hpPath pins pPath σ to the heat-flow convolution convDensityAdd pX (gaussianPDFReal 0 ⟨σ, _⟩) on the positive σ-range, and hpathDeriv1 / hpathDeriv2 identify pathDeriv1 / pathDeriv2 as its spatial first and second derivatives. These pins fix which functions the arguments are; the heat-equation equality is the conclusion, derived from the kernel-level heat equation, not a hypothesis. The remaining arguments are per-y integrand domination preconditions in the shape consumed by hasDerivAt_integral_of_dominated_loc_of_deriv_le.

                    @audit:ok

                    Used by

                      Entropy parametric differentiation #

                      theorem

                      InformationTheory.Shannon.FisherInfo.entropy_hasDerivAt_via_parametric

                      source
                      (pPath entDeriv : ) (bound : ) {t : } (ht : 0 < t) (hbound_int : MeasureTheory.Integrable bound MeasureTheory.volume) (hmeas : ∀ᶠ (s : ) in nhds t, MeasureTheory.AEStronglyMeasurable (fun (x : ) => (pPath s x).negMulLog) MeasureTheory.volume) (hint : MeasureTheory.Integrable (fun (x : ) => (pPath t x).negMulLog) MeasureTheory.volume) (hderiv_meas : MeasureTheory.AEStronglyMeasurable (entDeriv t) MeasureTheory.volume) (hb : ∀ᵐ (x : ), sSet.Ioo (t / 2) (2 * t), entDeriv s x bound x) (hdiff : ∀ᵐ (x : ), sSet.Ioo (t / 2) (2 * t), HasDerivAt (fun (s : ) => (pPath s x).negMulLog) (entDeriv s x) s) :
                      HasDerivAt (fun (s : ) => (x : ), (pPath s x).negMulLog) ( (x : ), entDeriv t x) t

                      Differentiation under the integral sign for the entropy along the heat-flow path: (d/ds) ∫ x, negMulLog (pPath s x) ∂volume = ∫ x, entDeriv t x ∂volume at s = t, via the gateway hasDerivAt_integral_of_dominated_loc_of_deriv_le. The domination and per-x derivative hypotheses are quantified over the neighborhood Set.Ioo (t/2) (2*t) (all the gateway needs); a universal form would be un-instantiable since the integrand diverges as s → 0⁺ and s → ∞.

                      @audit:ok

                      Used by

                        Infinite-interval integration by parts #

                        theorem

                        InformationTheory.Shannon.FisherInfo.debruijn_ibp_step

                        source
                        (u v u' v' : ) (hu : xtsupport v, HasDerivAt u (u' x) x) (hv : xtsupport u, HasDerivAt v (v' x) x) (huv' : MeasureTheory.Integrable (u * v') MeasureTheory.volume) (hu'v : MeasureTheory.Integrable (u' * v) MeasureTheory.volume) (huv : MeasureTheory.Integrable (u * v) MeasureTheory.volume) :
                        (x : ), u x * v' x = - (x : ), u' x * v x

                        The de Bruijn integration-by-parts step on the whole line: ∫ x, u x · v' x = - ∫ x, u' x · v x, where the boundary terms vanish by tail decay. A direct application of MeasureTheory.integral_mul_deriv_eq_deriv_mul_of_integrable.

                        @audit:ok

                        Used by

                          logDeriv to Fisher congruence #

                          theorem

                          InformationTheory.Shannon.FisherInfo.fisher_from_logDeriv

                          source
                          (p : ) (hp_nn : ∀ (x : ), 0 p x) (hint : MeasureTheory.Integrable (fun (x : ) => logDeriv p x ^ 2 * p x) MeasureTheory.volume) :

                          Shape congruence connecting the IBP output to the V2 Fisher information: ∫ x, (logDeriv p x)² · p x ∂volume = fisherInfoOfDensityReal p, under nonnegativity and integrability so the ℝ≥0∞ ↔ ℝ round-trip holds.

                          @audit:ok

                          Used by

                            Convolution-density positivity and Gaussian lower bound #

                            The Gaussian convolution density convDensityAdd pX g_s is everywhere strictly positive and bounded below by a shifted Gaussian, so its support is all of .

                            theorem

                            InformationTheory.Shannon.FisherInfo.convDensityAdd_pos

                            source
                            (pX : ) (hpX_nn : ∀ (x : ), 0 pX x) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mass : 0 < (y : ), pX y) {s : } (hs : 0 < s) (x : ) :

                            When pX is a nonnegative integrable density with positive mass (0 < ∫ pX), the Gaussian convolution density is strictly positive at every x. The integrand y ↦ pX y · g_s(x-y) is nonnegative and integrable with support equal to that of pX (the Gaussian factor never vanishes), and 0 < ∫ pX gives positive-measure support, hence a positive integral.

                            @audit:ok

                            Used by
                              theorem

                              InformationTheory.Shannon.FisherInfo.convDensityAdd_lower_bound_gaussian

                              source
                              (pX : ) (hpX_nn : ∀ (x : ), 0 pX x) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mass : (y : ), pX y = 1) {s : } (hs : 0 < s) :

                              The Gaussian convolution density is bounded below by a (1/2)-scaled shifted Gaussian: there is a radius R > 0 with (1/2) · g_s (|x| + R) ≤ convDensityAdd pX g_s x for every x. The proof picks R so that ∫_{[-R,R]} pX ≥ 1/2 (tightness), drops the integral to that box, and uses the monotonicity of g_s in |·|.

                              @audit:ok

                              Used by