Log-partition function ψ(λ) := log (∑ y, exp ⟨λ, f y⟩).

In the Lagrangian for constrained maximum entropy, ψ(λ) is the Legendre dual of H and its gradient ∇ψ(λ) = 𝔼_{p_λ^*}[f] encodes the KKT first-order condition (moment matching).

Exponential family pmf, Lagrangian / KKT-canonical form:

expFamilyDist λ f x := exp (⟨λ, f x⟩ - ψ(λ))

where ⟨λ, f x⟩ = ∑ i, λ i · f i x. This is the saddle-point optimizer of the Lagrangian L(p, λ) = H(p) + ∑ i, λ i (𝔼_p[f i] - c i). By expFamilyDist_eq_gibbsPmf it agrees pointwise with gibbsPmf f λ, so all positivity / pmf properties transfer.

Bridge: the KKT-canonical form expFamilyDist agrees pointwise with the Boltzmann–Gibbs form gibbsPmf:

exp (⟨λ, f x⟩ - ψ(λ)) = exp ⟨λ, f x⟩ / Z(λ).

Proof: Real.exp_sub + Real.exp_log on Z(λ) > 0.

expFamilyDist is pointwise strictly positive.

expFamilyDist λ f ∈ stdSimplex ℝ α.

KKT solution for the constrained maximum-entropy problem with feature maps f : Fin k → α → ℝ and moment targets c : Fin k → ℝ.

A KKTSolution f c packages a Lagrange multiplier vector lam : Fin k → ℝ with a proof that the exponential-family ansatz expFamilyDist f lam satisfies the moment-matching first-order condition 𝔼_{p*}[f i] = c i for all i.

This is precisely the KKT condition for the Lagrangian L(p, λ) = H(p) + ∑ i, λ i (𝔼_p[f i] - c i): stationarity in p picks out the exponential family, and feasibility in λ is the moment match.

KKT-solution moment matching restated in the gibbsPmf language.

Legendre / saddle-point identity for the exponential family. With KKT solution (λ, moment_match) for constraints c, the entropy of the exponential-family optimum has the closed form

H(p*) = ψ(λ) - ⟨λ, c⟩.

This is the KKT duality: at the saddle point of the Lagrangian, the primal value equals the dual value ψ(λ) - ⟨λ, c⟩.

Maximum entropy theorem (upper bound, KKT form): under moment constraints 𝔼_P[f i] = c i for all i, the entropy of P is bounded above by the entropy of the exponential-family solution expFamilyDist f λ, provided the latter also satisfies the same moments (KKT first-order condition).

Maximum entropy theorem (upper bound, KKTSolution form): a constraint-feasible P cannot exceed the entropy of the exponential-family solution attached to a KKT witness.

See also expFamily_maximizes_entropy.

Maximum entropy theorem (uniqueness, KKT form): for constraint-feasible P and a KKT-witnessed exponential-family solution, entropy equality H(P) = H(expFamilyDist f λ) holds if and only if P = expFamilyDist f λ pointwise.

Maximum entropy theorem (uniqueness, KKTSolution form): entropy equality holds if and only if P = expFamilyDist f λ pointwise.

See also expFamily_unique.

Variational upper bound (Legendre / free-energy form) — any constraint-feasible P satisfies the dual bound

H(P) ≤ ψ(λ) - ⟨λ, c⟩

for every λ whose exponential-family solution satisfies the same moments. This is the variational characterization of logPartitionψ as the Legendre transform of -H restricted to the feasibility set.

KKT-packaged variational form.

KKT first-order condition equivalence — the moment-matching hypothesis 𝔼_{p*}[f] = c (the gradient-of-ψ condition ∇ψ(λ) = c) is equivalent to the gibbs ansatz satisfying the same constraint as P. This is the formal content of "KKT stationarity in λ".

KKT zero-multiplier reduction: expFamilyDist f 0 = expFamilyDist g 0 for any features f, g (both equal to the uniform pmf). This is the unconstrained-Lagrangian degenerate case λ = 0.