InformationTheory.Shannon.Cramer.Cramer
Cramér's theorem #
The Legendre transform of a real-valued function and the Cramér rate
function (I(a) = Λ^*(a) for Λ = cgf X μ), together with the upper bound of
Cramér's large deviation theorem (Cover–Thomas, upper half) in
per-n Chernoff, log, and limsup forms. The lower bound is developed downstream
in CramerGeneralLower.lean.
Main definitions #
legendre Λ a— the Legendre transformΛ^*(a) := sup_λ (λ·a − Λ(λ)).cramerRate X μ a— the Cramér rate functionI(a) := (cgf X μ)^*(a).
Main statements #
cramerRate_apply_le,cramerRate_nonneg— basic bounds on the rate function.chernoff_bound_n_iid— per-nChernoff bound for the upper tail of an i.i.d. bounded sum.cramer_upper,cramer_upper_legendre— the limsup-form upper bound.integral_tilted_eq_deriv_cgf,klDiv_tilted_eq— tilted change-of-measure identities used by the lower bound.
References #
- T. M. Cover and J. A. Thomas, Elements of Information Theory (2nd ed.), Wiley, 2006.
legendre and cramerRate: definitions and basic properties #
InformationTheory.Shannon.Cramer.legendre
sourceThe Legendre transform of Λ : ℝ → ℝ at a: Λ^*(a) := sup_λ (λ·a − Λ(λ)).
Mathlib has no general Legendre-transform / convex-conjugate API, so this is the
textbook sSup form. When the image set is not BddAbove, Mathlib returns
sSup = 0, so the basic properties below take an explicit BddAbove hypothesis.
Equations
Instances For
Used by
InformationTheory.Shannon.Cramer.cramerRate
sourceCramér rate function I(a) := Λ^*(a) for Λ := cgf X μ.
Equations
Instances For
Used by
InformationTheory.Shannon.Cramer.legendre_apply_le
sourceEach linear-minus-Λ value is ≤ the Legendre transform.
Used by
InformationTheory.Shannon.Cramer.legendre_nonneg
sourceIf Λ 0 = 0 (e.g. Λ = cgf X μ for a probability measure), the Legendre
transform is non-negative whenever the supremum exists.
Used by
InformationTheory.Shannon.Cramer.cramerRate_apply_le
sourceCramér rate-function version of legendre_apply_le.
Used by
InformationTheory.Shannon.Cramer.cramerRate_nonneg
sourceCramér rate function is non-negative whenever the Legendre supremum exists
for a probability measure (then cgf X μ 0 = 0).
Used by
cgf sum and bounded-RV integrability helpers #
InformationTheory.Shannon.Cramer.integrable_exp_mul_of_bounded
sourceFor a bounded real random variable on a finite measure space, the
exponential moment exp (t * Y) is integrable for every t. This is the
hypothesis-eliminator the Cramér chain uses to remove Integrable premises
from the main statements.
Used by
InformationTheory.Shannon.Cramer.cgf_sum_eq_nsmul
sourcei.i.d. CGF sum formula: for an i.i.d. family X : ℕ → Ω → ℝ (with X i
identically distributed to X 0 and the exponential moments integrable for
every i), cgf (∑ i ∈ range n, X i) μ t = n · cgf (X 0) μ t.
Used by
Cramér upper bound: per-n Chernoff bound (i.i.d. strengthening) #
InformationTheory.Shannon.Cramer.chernoff_bound_n_iid
sourcePer-n Chernoff bound for the upper tail of an i.i.d. sum of bounded real
random variables: tilting by any lam ≥ 0, the upper-tail probability decays
exponentially with rate at least lam * a − Λ(lam).
Used by
InformationTheory.Shannon.Cramer.cramer_log_bound_n_iid
sourcePer-n Cramér upper bound, log form: for each n ≥ 1 with positive tail
probability, (1/n) · log P[a·n ≤ Sₙ] ≤ -(lam · a − Λ(lam)) for every
lam ≥ 0.
Used by
Cramér upper bound: limsup form #
InformationTheory.Shannon.Cramer.cramer_upper
sourceCramér's theorem (upper bound, limsup form): for each lam ≥ 0, the
upper-tail probability of the i.i.d. sample sum decays at exponential rate at
least lam * a − Λ(lam):
limsup_n (1/n) log P[a·n ≤ Sₙ] ≤ -(lam · a − Λ(lam)).
Two technical hypotheses make the result clean:
h_pos— the tail probability is eventually positive (e.g. whena ≤ ess sup X), so thatlogis finite.h_cobdd— the resulting log-rate sequence is cobounded below in the limsup sense. This holds whenever the sequence does not blow up to-∞, e.g. when the tail probabilities admit any sub-exponential lower bound.
Used by
InformationTheory.Shannon.Cramer.cramer_upper_legendre
sourceCramér's theorem (upper bound, Legendre form): if the Legendre transform
of Λ = cgf (X 0) μ at a is attained by some non-negative lam (hlam_opt),
then
limsup_n (1/n) log P[a·n ≤ Sₙ] ≤ -cramerRate (X 0) μ a.
hlam_opt is a regularity precondition (it holds for a ≥ 𝔼[X] by convexity of
the Legendre transform), not part of the proof core.
See also cramer_upper.
Used by
Cramér lower bound: tilted change-of-measure identities #
The lower bound uses an exponential change-of-measure to the tilted measure
μ_lam := μ.tilted (lam * X ·), where lam is chosen so that Λ'(lam) = a.
Under the tilted measure the mean of X equals a, a tilted-side law of large
numbers concentrates the sample mean near a, and converting back to μ via the
Radon–Nikodym derivative of tilted picks up an
exp(-n · (lam · a − Λ(lam))) = exp(-n · cramerRate) factor.
This file publishes the change-of-measure building blocks — klDiv_tilted_eq
(the KL-of-tilted identity) and integral_tilted_eq_deriv_cgf — that the
downstream lower bound consumes.
InformationTheory.Shannon.Cramer.mem_interior_integrableExpSet_of_bounded
sourceUniversal integrability for bounded RVs: a bounded random variable has
every t ∈ ℝ in its integrableExpSet, hence the whole real line lies in the
interior. This kills the interior (integrableExpSet X μ) hypothesis of
integral_tilted_mul_self / variance_tilted_mul in the bounded-RV setting.
Used by
InformationTheory.Shannon.Cramer.isProbabilityMeasure_tilted_of_bounded
sourceThe tilted measure of a bounded random variable is a probability measure.
Used by
InformationTheory.Shannon.Cramer.integral_tilted_eq_deriv_cgf
sourceFor a bounded random variable Y, the expectation of Y under
μ.tilted (lam * Y ·) equals the first derivative of cgf Y μ at lam.
Used by
InformationTheory.Shannon.Cramer.klDiv_tilted_eq
sourceKL-of-tilted identity.
For a bounded real random variable X : Ω → ℝ on a probability measure μ,
the (integral form of the) Kullback-Leibler divergence between μ.tilted (lam * X ·)
and μ admits the closed form
∫ ω, log (rnDeriv (μ.tilted (lam * X ·)) μ ω).toReal ∂(μ.tilted (lam * X ·)) = lam * (μ.tilted (lam * X ·))[X] - cgf X μ lam.
We work directly with the ∫ log (rnDeriv) representation rather than the
ℝ≥0∞-valued klDiv to keep the algebraic identity transparent.