InformationTheory

InformationTheory.Shannon.EPI.G2.KLFatouLSC

source

EPI G2 (α) upper bound — KL lower-semicontinuity via klFun-Fatou #

This file supplies the (α) upper bound of the EPI G2 general-sandwich result along a constructive route that avoids the Donsker–Varadhan dual hard direction.

The route:

  • klDiv μ γ = ∫⁻ klFun(rnDeriv μ γ) dγ (klDiv_eq_lintegral_klFun_of_ac, the ℝ≥0∞ integral form, no integrability side-condition),
  • klFun ≥ 0 + lintegral_liminf_le (Fatou) gives klDiv μ γ ≤ liminf klDiv (μ_n) γ from a.e. pointwise convergence of the densities (W1 = klDiv_le_liminf_of_ae_tendsto),
  • the a.e. density convergence is identified through the withDensity-quotient bridge (W2 = rnDeriv_withDensity_quotient_ae),
  • the cross-term ∫ f_n log g → ∫ pX log g (W3) and the density-level a.e. subsequence (W4) plug in,
  • assembly turns the KL liminf bound into the differential-entropy limsup bound (α) through klDiv_toReal_eq_neg_differentialEntropy_sub_cross (EPI/G2/BridgeDensityHelpers.lean, @audit:ok).

The two missing facts (KL-LSC, withDensity rnDeriv quotient) are assemblies of existing Mathlib parts. The final boundedness step of the (α) assembly is supplied by the pX-only (β) lower bound negMulLog_convDensity_entropy_ge_density (EPI/G2/ConvEntropyDensity.lean).

theorem

InformationTheory.EPIG2KLFatou.toReal_le_liminf_toReal_of_le_liminf

source
{a : ENNReal} {b : ENNReal} {C : } (hb : b ) (hC : ∀ᶠ (i : ) in Filter.atTop, a i ENNReal.ofReal C) (hle : b Filter.liminf a Filter.atTop) :
Used by
    theorem

    InformationTheory.EPIG2KLFatou.tendsto_of_le_liminf_of_le_tendsto

    source
    {a b : } {v C : } (hnn : ∀ (i : ), 0 a i) (hub : ∀ (i : ), a i b i) (hbdd : ∀ (i : ), a i C) (hb : Filter.Tendsto b Filter.atTop (nhds v)) (hliminf_ge : v Filter.liminf a Filter.atTop) :
    Used by
      theorem

      InformationTheory.EPIG2KLFatou.rnDeriv_withDensity_quotient_ae

      source
      {f g : } (hf_meas : Measurable f) (hg_meas : Measurable g) (hf_nn : ∀ (x : ), 0 f x) (hg_pos : ∀ (x : ), 0 < g x) (hf_int : MeasureTheory.Integrable f MeasureTheory.volume) (hg_int : MeasureTheory.Integrable g MeasureTheory.volume) :

      W2, the withDensity rnDeriv quotient identification (the largest gap, an assembly of existing parts). For f ≥ 0, g > 0, both measurable and integrable, the Radon–Nikodym derivative of volume.withDensity (ofReal∘f) w.r.t. volume.withDensity (ofReal∘g) is, γ-a.e. (with γ the g-weighted measure), the pointwise quotient ofReal (f x / g x).

      Route: Measure.rnDeriv_withDensity collapses each withDensity rnDeriv to its density on the volume axis, rnDeriv_withDensity_right inverts the right withDensity, and the base is transferred =ᵐ[volume] ⟹ =ᵐ[γ] through volume ≪ γ (g > 0).

      hf_meas/hg_meas/hf_nn/hg_pos/hf_int/hg_int are regularity preconditions. @audit:ok

      Used by
        theorem

        InformationTheory.EPIG2KLFatou.klDiv_le_liminf_of_ae_tendsto

        source
        (γ : MeasureTheory.Measure ) [MeasureTheory.IsFiniteMeasure γ] (μ : MeasureTheory.Measure ) (μ_n : MeasureTheory.Measure ) [MeasureTheory.IsFiniteMeasure μ] [∀ (n : ), MeasureTheory.IsFiniteMeasure (μ_n n)] (hμ_ac : μ.AbsolutelyContinuous γ) (hμn_ac : ∀ (n : ), (μ_n n).AbsolutelyContinuous γ) (h_ae : ∀ᵐ (x : ) γ, Filter.Tendsto (fun (n : ) => ((μ_n n).rnDeriv γ x).toReal) Filter.atTop (nhds (μ.rnDeriv γ x).toReal)) :
        klDiv μ γ Filter.liminf (fun (n : ) => klDiv (μ_n n) γ) Filter.atTop

        W1, KL lower-semicontinuity via klFun-Fatou (the heart of the route). If μ_n.rnDeriv γ → μ.rnDeriv γ γ-a.e. (as reals), then klDiv μ γ ≤ liminf klDiv (μ_n) γ.

        Route: rewrite both sides with the ℝ≥0∞ integral form klDiv_eq_lintegral_klFun_of_ac, apply Fatou lintegral_liminf_le (klFun ≥ 0, lifted by ENNReal.ofReal), and discharge the pointwise liminf bound via continuity of klFun composed with the a.e. convergence.

        hμ_ac/hμn_ac (absolute continuity) and h_ae (a.e. convergence input) are preconditions; the conclusion is the LSC inequality (not bundled). @audit:ok

        Used by
          theorem

          InformationTheory.EPIG2KLFatou.convDensity_tendsto_ae_subseq

          source
          {pX : } (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) (u : ) (hu_pos : ∀ (n : ), 0 < u n) (hu_lim : Filter.Tendsto u Filter.atTop (nhdsWithin 0 (Set.Ioi 0))) :

          W4, density-level a.e. subsequence convergence (the negMulLog-free companion of negMulLog_convDensity_tendsto_ae_subseq in EPI/Vitali/AE.lean). Along any sequence u → 0⁺, the smoothed densities convDensityAdd pX g_{u n} converge to pX a.e. along a subsequence n ↦ u (ns n) (StrictMono ns), before composing with any continuous map.

          Same route as EPI/Vitali/AE.lean but cut before the negMulLog composition: layer-1 L¹ convergence convDensityAdd_tendsto_L1_zero (@audit:ok, sorryAx-free) → tendstoInMeasure_of_tendsto_eLpNorm (Lp → measure) → TendstoInMeasure.exists_seq_tendsto_ae (measure → a.e. subsequence). No own sorry.

          All hpX_* are regularity preconditions; hu_lim is the input filter. @audit:ok

          Used by
            theorem

            InformationTheory.EPIG2KLFatou.log_gaussianPDFReal_zero

            source
            {v : NNReal} (hv : v 0) (x : ) :

            The log of the Gaussian density as a quadratic (verbatim from the gaussianPDFReal definition (√(2πv))⁻¹ · exp(-(x-μ)²/(2v))). For v ≠ 0 and μ = 0: log (gaussianPDFReal 0 v x) = - log (√(2πv)) - x² / (2v). @audit:ok

            Used by
              theorem

              InformationTheory.EPIG2KLFatou.cross_term_closed_form

              source
              {pX : } (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) {σ2 : NNReal} ( : σ2 0) {t : } (ht : 0 < t) :
              (x : ), Shannon.EPIConvDensity.convDensityAdd pX (ProbabilityTheory.gaussianPDFReal 0 t, ) x * Real.log (ProbabilityTheory.gaussianPDFReal 0 σ2 x) = (-Real.log (2 * Real.pi * σ2) * (x : ), pX x) - 1 / (2 * σ2) * (( (x : ), x ^ 2 * pX x) + ( (x : ), pX x) * t)

              The cross-term closed form for a smoothed density against a Gaussian log-density. With g := gaussianPDFReal 0 σ² (σ² ≠ 0) and f_t := convDensityAdd pX g_t (t > 0), the cross integral ∫ f_t · log g is an affine function of t: ∫ x, f_t x · log (g x) = c₀ · 1 − (1/(2σ²)) · (M2(pX) + (∫pX)·t), where c₀ = − log (√(2πσ²)) and M2(pX) = ∫ x²·pX. @audit:ok

              Used by
                theorem

                InformationTheory.EPIG2KLFatou.pX_cross_term_expand

                source
                {pX : } (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) {σ2 : NNReal} ( : σ2 0) :
                (x : ), pX x * Real.log (ProbabilityTheory.gaussianPDFReal 0 σ2 x) = (-Real.log (2 * Real.pi * σ2) * (x : ), pX x) - 1 / (2 * σ2) * (x : ), x ^ 2 * pX x

                The limit of the marginal cross-term ∫ pX · log g in the same expanded form. With g := gaussianPDFReal 0 σ², ∫ x, pX x · log (g x) = c₀·(∫pX) − (1/(2σ²))·M2(pX). @audit:ok

                Used by
                  theorem

                  InformationTheory.EPIG2KLFatou.cross_term_tendsto

                  source
                  {pX : } (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) {σ2 : NNReal} ( : σ2 0) (u : ) (hu_pos : ∀ (n : ), 0 < u n) (hu_lim : Filter.Tendsto u Filter.atTop (nhds 0)) :

                  W3, cross-term convergence ∫ f_n · log g → ∫ pX · log g as u_n → 0⁺. Combines the affine closed form cross_term_closed_form (in t) with the marginal expansion pX_cross_term_expand; the difference is −(1/(2σ²))·(∫pX)·u_n → 0. @audit:ok

                  Used by
                    theorem

                    InformationTheory.EPIG2KLFatou.integrable_mul_log_gaussianPDFReal_of_sq_integrable

                    source
                    Used by
                      theorem

                      InformationTheory.EPIG2KLFatou.integrable_mul_log_self_of_negMulLog_integrable

                      source
                      Used by
                        theorem

                        InformationTheory.EPIG2KLFatou.integrable_rnDeriv_toReal_mul_log_rnDeriv_toReal_of_density

                        source
                        {ν ρ : MeasureTheory.Measure } {p q : } (hp_nn : ∀ (x : ), 0 p x) (hq_nn : ∀ (x : ), 0 q x) (hν_rnDeriv : ν.rnDeriv MeasureTheory.volume =ᵐ[MeasureTheory.volume] fun (x : ) => ENNReal.ofReal (p x)) (hρ_rnDeriv : ρ.rnDeriv MeasureTheory.volume =ᵐ[MeasureTheory.volume] fun (x : ) => ENNReal.ofReal (q x)) (hpq_int : MeasureTheory.Integrable (fun (x : ) => p x * Real.log (q x)) MeasureTheory.volume) :
                        Used by
                          theorem

                          InformationTheory.EPIG2KLFatou.differentialEntropy_withDensity_eq_integral_negMulLog

                          source
                          {p : } (hp_meas : Measurable p) (hp_nn : ∀ (x : ), 0 p x) :
                          Used by
                            theorem

                            InformationTheory.EPIG2KLFatou.integral_rnDeriv_toReal_mul_log_rnDeriv_toReal_eq_of_density

                            source
                            {ν ρ : MeasureTheory.Measure } {p q : } (hp_nn : ∀ (x : ), 0 p x) (hq_nn : ∀ (x : ), 0 q x) (hν_rnDeriv : ν.rnDeriv MeasureTheory.volume =ᵐ[MeasureTheory.volume] fun (x : ) => ENNReal.ofReal (p x)) (hρ_rnDeriv : ρ.rnDeriv MeasureTheory.volume =ᵐ[MeasureTheory.volume] fun (x : ) => ENNReal.ofReal (q x)) :
                            Used by
                              theorem

                              InformationTheory.EPIG2KLFatou.klDiv_ne_top_of_density_log_integrable

                              source
                              {ν ρ : MeasureTheory.Measure } [MeasureTheory.SigmaFinite ν] [MeasureTheory.SigmaFinite ρ] [ν.HaveLebesgueDecomposition MeasureTheory.volume] {p q : } (hp_nn : ∀ (x : ), 0 p x) (hq_nn : ∀ (x : ), 0 q x) (hν_v : ν.AbsolutelyContinuous MeasureTheory.volume) (hρ_v : ρ.AbsolutelyContinuous MeasureTheory.volume) (hνρ : ν.AbsolutelyContinuous ρ) (hν_rnDeriv : ν.rnDeriv MeasureTheory.volume =ᵐ[MeasureTheory.volume] fun (x : ) => ENNReal.ofReal (p x)) (hρ_rnDeriv : ρ.rnDeriv MeasureTheory.volume =ᵐ[MeasureTheory.volume] fun (x : ) => ENNReal.ofReal (q x)) (hlogp_int : MeasureTheory.Integrable (fun (x : ) => p x * Real.log (p x)) MeasureTheory.volume) (hcross_int : MeasureTheory.Integrable (fun (x : ) => p x * Real.log (q x)) MeasureTheory.volume) :
                              klDiv ν ρ
                              Used by
                                theorem

                                InformationTheory.EPIG2KLFatou.rnDeriv_toReal_tendsto_ae_subseq_of_density_quotient

                                source
                                {p g : } (hp_nn : ∀ (x : ), 0 p x) (hg_pos : ∀ (x : ), 0 < g x) {f : } {idx : } (μn : MeasureTheory.Measure ) (μ γ : MeasureTheory.Measure ) (hf_nn : ∀ (n : ) (x : ), 0 f n x) (hμn_quot : ∀ (n : ), (μn n).rnDeriv γ =ᵐ[γ] fun (x : ) => ENNReal.ofReal (f n x / g x)) (hμ_quot : μ.rnDeriv γ =ᵐ[γ] fun (x : ) => ENNReal.ofReal (p x / g x)) (hae_vol_γ : ∀ᵐ (x : ) γ, Filter.Tendsto (fun (i : ) => f (idx i) x) Filter.atTop (nhds (p x))) :
                                ∀ᵐ (x : ) γ, Filter.Tendsto (fun (i : ) => ((μn (idx i)).rnDeriv γ x).toReal) Filter.atTop (nhds (μ.rnDeriv γ x).toReal)
                                Used by
                                  theorem

                                  InformationTheory.EPIG2KLFatou.tendsto_negMulLog_integral_of_klFatou_squeeze

                                  source
                                  {pX : } (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) {f : } (g : ) {u : } (hu_pos : ∀ (n : ), 0 < u n) (hu_lim : Filter.Tendsto u Filter.atTop (nhdsWithin 0 (Set.Ioi 0))) (hf_def : f = fun (n : ) => Shannon.EPIConvDensity.convDensityAdd pX (ProbabilityTheory.gaussianPDFReal 0 u n, )) (μn : MeasureTheory.Measure ) (μ γ : MeasureTheory.Measure ) [MeasureTheory.IsFiniteMeasure γ] [MeasureTheory.IsFiniteMeasure μ] [∀ (n : ), MeasureTheory.IsFiniteMeasure (μn n)] (hμn_wd : ∀ (n : ), μn n = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (f n x)) (hμ_wd : μ = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (pX x)) (hf_nn : ∀ (n : ) (x : ), 0 f n x) (hf_meas : ∀ (n : ), Measurable (f n)) (hf_int : ∀ (n : ), MeasureTheory.Integrable (f n) MeasureTheory.volume) (hg_pos : ∀ (x : ), 0 < g x) (hg_meas : Measurable g) (hg_int : MeasureTheory.Integrable g MeasureTheory.volume) (hγ_wd : γ = MeasureTheory.volume.withDensity fun (x : ) => ENNReal.ofReal (g x)) (hμ_γ : μ.AbsolutelyContinuous γ) (hμn_γ : ∀ (n : ), (μn n).AbsolutelyContinuous γ) (hγ_v : γ.AbsolutelyContinuous MeasureTheory.volume) {h_n KLr cross_n : } {crossμ L : } (hhn_eq : ∀ (n : ), h_n n = -KLr n - cross_n n) (hKLr_def : KLr = fun (n : ) => (klDiv (μn n) γ).toReal) (hKLr_upper : ∀ (n : ), KLr n -cross_n n - L) (hKLr_nn : ∀ (n : ), 0 KLr n) (hupper_lim : Filter.Tendsto (fun (n : ) => -cross_n n - L) Filter.atTop (nhds (klDiv μ γ).toReal)) (hcross_tendsto : Filter.Tendsto cross_n Filter.atTop (nhds crossμ)) (hKL_ne_top : ∀ (n : ), klDiv (μn n) γ ) (hμ_KL_ne_top : klDiv μ γ ) (hL_eq_val : L = -(klDiv μ γ).toReal - crossμ) :
                                  Used by
                                    theorem

                                    InformationTheory.EPIG2KLFatou.negMulLog_convDensity_limsup_le

                                    source
                                    {pX : } (hpX_nn : ∀ (x : ), 0 pX x) (hpX_meas : Measurable pX) (hpX_int : MeasureTheory.Integrable pX MeasureTheory.volume) (hpX_mass : (y : ), pX y = 1) (hpX_mom : MeasureTheory.Integrable (fun (y : ) => y ^ 2 * pX y) MeasureTheory.volume) (hpX_ent : MeasureTheory.Integrable (fun (x : ) => (pX x).negMulLog) MeasureTheory.volume) {σ2 : NNReal} ( : σ2 0) (u : ) (hu_pos : ∀ (n : ), 0 < u n) (hu_lim : Filter.Tendsto u Filter.atTop (nhdsWithin 0 (Set.Ioi 0))) :
                                    Used by