InformationTheory

InformationTheory.Shannon.RelayCutset

source

Relay channel — cut-set outer bound (structure + single-letterization) #

The relay channel (Cover–Thomas) has a single sender, a single relay, and a single receiver. The cut-set outer bound bounds any achievable rate by the minimum of two mutual-information quantities, evaluated at the worst joint input distribution:

  • the broadcast cut I(X; Y₁, Y | X₁) — information leaving the sender, conditioned on the relay's transmission;
  • the MAC cut I(X, X₁; Y) — information arriving at the receiver from the sender–relay pair.

This file provides the structural definitions (RelayChannel, RelayCode, relayCutsetBound) and the two single-letterization lemmas that turn the block quantities I(Xⁿ, X₁ⁿ; Yⁿ) and I(Xⁿ; Y₁ⁿ, Yⁿ | X₁ⁿ) into per-letter sums under a memoryless relay channel. Both single-letterizations are instances of the multiple-access converse machinery: the MAC cut reuses the unconditional per-letter bound and the broadcast cut reuses the conditional one, with the joint input (Xᵢ, X₁ᵢ) and joint output (Y₁ᵢ, Yᵢ) playing the roles of the MAC's joint input/output.

The memoryless structure is a precondition (regularity); the per-letter inequality (the genuine content) is proven, not assumed. The operational outer bound relay_cutset_outer_bound (Fano + data processing + the min combination) is assembled below, in the CutsetHeadline section, from the two per-cut outer bounds.

Main statements #

abbrev

InformationTheory.Shannon.Relay.RelayChannel

source
@[reducible, inline]
(α : Type u_1) (α₁ : Type u_2) (β : Type u_3) (β₁ : Type u_4) [MeasurableSpace α] [MeasurableSpace α₁] [MeasurableSpace β] [MeasurableSpace β₁] :
Type (max (max u_2 u_1) u_4 u_3)

A relay channel: Markov kernel from joint input (α × α₁) to joint output (β × β₁) (sender input α, relay input α₁, receiver output β, relay observation β₁).

Equations
Instances For
    Used by
      structure

      InformationTheory.Shannon.Relay.RelayCode

      source
      (M n : ) (α : Type u_1) (α₁ : Type u_2) (β : Type u_3) (β₁ : Type u_4) [MeasurableSpace α] [MeasurableSpace α₁] [MeasurableSpace β] [MeasurableSpace β₁] :
      Type (max (max (max u_1 u_2) u_3) u_4)

      A relay block code of length n with M messages: a sender encoder, a causal relay function (reads past relay observations β₁, emits the next relay input α₁), and a decoder.

      • encoder : Fin MFin nα
      • relay(i : Fin n) : (Fin iβ₁)α₁
      • decoder : (Fin nβ)Fin M
      Instances For
        Used by
          noncomputable def

          InformationTheory.Shannon.Relay.relayCutsetBound

          source
          (Ib Im : ) :

          The cut-set outer bound in scalar form: the minimum of the broadcast-cut rate Ib = I(X; Y₁, Y | X₁) and the MAC-cut rate Im = I(X, X₁; Y). The outer maximization over joint input pmfs p(x, x₁) is left to callers.

          Equations
          Instances For
            Used by
              theorem

              InformationTheory.Shannon.Relay.relayCutsetBound_def

              source
              @[simp]
              (Ib Im : ) :
              relayCutsetBound Ib Im = min Ib Im
              Used by
                theorem

                InformationTheory.Shannon.Relay.relay_mac_cut_singleletterize

                source
                {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} {α₁ : Type u_3} {β : Type u_4} [Fintype α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [StandardBorelSpace α] [Fintype α₁] [Nonempty α₁] [MeasurableSpace α₁] [MeasurableSingletonClass α₁] [StandardBorelSpace α₁] [Fintype β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [StandardBorelSpace β] {n : } (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (Xs : Fin nΩα) (X₁s : Fin nΩα₁) (Ys : Fin nΩβ) (hXs : ∀ (i : Fin n), Measurable (Xs i)) (hX₁s : ∀ (i : Fin n), Measurable (X₁s i)) (hYs : ∀ (i : Fin n), Measurable (Ys i)) (h_memo : ChannelCodingConverseGeneral.IsMemorylessChannel μ (fun (i : Fin n) (ω : Ω) => (Xs i ω, X₁s i ω)) Ys) :
                (mutualInfo μ (fun (ω : Ω) (j : Fin n) => (Xs j ω, X₁s j ω)) fun (ω : Ω) (j : Fin n) => Ys j ω).toReal i : Fin n, (mutualInfo μ (fun (ω : Ω) => (Xs i ω, X₁s i ω)) (Ys i)).toReal

                Under a memoryless relay channel (joint input (Xᵢ, X₁ᵢ) to receiver output Yᵢ), the MAC-cut block mutual information I(Xⁿ, X₁ⁿ; Yⁿ) is bounded by the per-letter sum ∑ᵢ I(Xᵢ, X₁ᵢ; Yᵢ). The memoryless structure is a precondition; the inequality (the genuine content) is proven, not assumed. @audit:ok

                Used by
                  theorem

                  InformationTheory.Shannon.Relay.relay_broadcast_cut_singleletterize

                  source
                  {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} {α₁ : Type u_3} {β : Type u_4} {β₁ : Type u_5} [Fintype α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [StandardBorelSpace α] [Fintype α₁] [DecidableEq α₁] [Nonempty α₁] [MeasurableSpace α₁] [MeasurableSingletonClass α₁] [StandardBorelSpace α₁] [Fintype β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [StandardBorelSpace β] [Fintype β₁] [Nonempty β₁] [MeasurableSpace β₁] [MeasurableSingletonClass β₁] [StandardBorelSpace β₁] {n : } (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (Xs : Fin nΩα) (X₁s : Fin nΩα₁) (Ys : Fin nΩβ) (Y₁s : Fin nΩβ₁) (hXs : ∀ (i : Fin n), Measurable (Xs i)) (hX₁s : ∀ (i : Fin n), Measurable (X₁s i)) (hYs : ∀ (i : Fin n), Measurable (Ys i)) (hY₁s : ∀ (i : Fin n), Measurable (Y₁s i)) (h_memo : ChannelCodingConverseGeneral.IsMemorylessChannel μ (fun (i : Fin n) (ω : Ω) => (Xs i ω, X₁s i ω)) fun (i : Fin n) (ω : Ω) => (Y₁s i ω, Ys i ω)) :
                  (condMutualInfo μ (fun (ω : Ω) (j : Fin n) => Xs j ω) (fun (ω : Ω) (j : Fin n) => (Y₁s j ω, Ys j ω)) fun (ω : Ω) (j : Fin n) => X₁s j ω).toReal i : Fin n, (condMutualInfo μ (Xs i) (fun (ω : Ω) => (Y₁s i ω, Ys i ω)) (X₁s i)).toReal

                  The broadcast-cut conditional block mutual information I(Xⁿ; Y₁ⁿ, Yⁿ | X₁ⁿ) is bounded by the per-letter sum ∑ᵢ I(Xᵢ; Y₁ᵢ, Yᵢ | X₁ᵢ), under the memoryless relay channel with joint input (Xᵢ, X₁ᵢ) and joint output (Y₁ᵢ, Yᵢ). @audit:ok

                  Used by
                    theorem

                    InformationTheory.Shannon.Relay.relay_mac_cut_outer_bound

                    source
                    {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} {α₁ : Type u_3} {β : Type u_4} [Fintype α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [StandardBorelSpace α] [Fintype α₁] [Nonempty α₁] [MeasurableSpace α₁] [MeasurableSingletonClass α₁] [StandardBorelSpace α₁] [Fintype β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [StandardBorelSpace β] {M n : } [NeZero M] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (W : ΩFin M) (decoder : (Fin nβ)Fin M) (Xs : Fin nΩα) (X₁s : Fin nΩα₁) (Ys : Fin nΩβ) (hW : Measurable W) (hdecoder : Measurable decoder) (hXs : ∀ (i : Fin n), Measurable (Xs i)) (hX₁s : ∀ (i : Fin n), Measurable (X₁s i)) (hYs : ∀ (i : Fin n), Measurable (Ys i)) (hW_uniform : MeasureTheory.Measure.map W μ = (↑(Fintype.card (Fin M)))⁻¹ MeasureTheory.Measure.count) (hcard : 2 M) (h_markov : IsMarkovChain μ W (fun (ω : Ω) (j : Fin n) => (Xs j ω, X₁s j ω)) fun (ω : Ω) (j : Fin n) => Ys j ω) (h_memo : ChannelCodingConverseGeneral.IsMemorylessChannel μ (fun (i : Fin n) (ω : Ω) => (Xs i ω, X₁s i ω)) Ys) :
                    Real.log M i : Fin n, (mutualInfo μ (fun (ω : Ω) => (Xs i ω, X₁s i ω)) (Ys i)).toReal + Real.binEntropy (MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) decoder) + MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) decoder * Real.log (M - 1)

                    The operational outer bound at the MAC cut of a relay channel (Cover–Thomas): for a uniformly distributed message W decoded from the receiver output Yⁿ, the rate is bounded by the per-letter MAC-cut sum plus a Fano slack, log M ≤ ∑ᵢ I(Xᵢ, X₁ᵢ; Yᵢ) + h(Pe) + Pe · log(M - 1), where Pe is the block decoding error probability.

                    The proof chains destination Fano (shannon_converse_single_shot), the data-processing inequality along the block Markov chain W → (Xⁿ, X₁ⁿ) → Yⁿ (mutualInfo_le_of_markov), and the MAC-cut single-letterization (relay_mac_cut_singleletterize). The Markov and memoryless hypotheses are preconditions (structure / regularity); the per-letter inequality (the genuine content) is proven, not assumed, so neither hypothesis is load-bearing. The outer maximization over joint input pmfs p(x, x₁) — and hence the conversion of the per-letter sum to n · max_p I — is left to callers, which is why the conclusion keeps the explicit per-letter sum. @audit:ok

                    Used by
                      theorem

                      InformationTheory.Shannon.Relay.relay_broadcast_cut_message_telescope

                      source
                      {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} {α₁ : Type u_3} {β : Type u_4} {β₁ : Type u_5} [Fintype α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [StandardBorelSpace α] [Fintype α₁] [Nonempty α₁] [MeasurableSpace α₁] [MeasurableSingletonClass α₁] [StandardBorelSpace α₁] [Fintype β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [StandardBorelSpace β] [Fintype β₁] [Nonempty β₁] [MeasurableSpace β₁] [MeasurableSingletonClass β₁] [StandardBorelSpace β₁] {M n : } [NeZero M] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (W : ΩFin M) (c : RelayCode M n α α₁ β β₁) (Ys : Fin nΩβ) (Y₁s : Fin nΩβ₁) (hW : Measurable W) (hYs : ∀ (i : Fin n), Measurable (Ys i)) (hY₁s : ∀ (i : Fin n), Measurable (Y₁s i)) (h_memo : ∀ (i : Fin n), IsMarkovChain μ (fun (ω : Ω) => (Y₁s i ω, Ys i ω)) (fun (ω : Ω) => (c.encoder (W ω) i, c.relay i fun (j : Fin i) => Y₁s j, ω)) fun (ω : Ω) => (W ω, fun (j : Fin i) => (Y₁s j, ω, Ys j, ω))) :
                      (mutualInfo μ W fun (ω : Ω) (j : Fin n) => Ys j ω).toReal i : Fin n, (condMutualInfo μ (fun (ω : Ω) => c.encoder (W ω) i) (fun (ω : Ω) => (Y₁s i ω, Ys i ω)) fun (ω : Ω) => c.relay i fun (j : Fin i) => Y₁s j, ω).toReal

                      Message-level telescoping at the broadcast cut of a relay channel (Cover–Thomas): the message–output mutual information I(W; Yⁿ) is bounded directly by the per-letter conditional sum ∑ᵢ I(Xᵢ; Y₁ᵢ, Yᵢ | X₁ᵢ), where Xᵢ = encoder(W)ᵢ is the i-th sender symbol and X₁ᵢ = relay i (Y₁^{<i}) is the i-th relay symbol read causally from the relay's past observations.

                      The bound cannot be obtained from relay_broadcast_cut_singleletterize (which single-letterizes the block conditional I(Xⁿ; Y₁ⁿ, Yⁿ | X₁ⁿ)), because routing through the block quantity leaves a chain-rule remainder I(W; X₁ⁿ) ≠ 0 (the relay input X₁ⁿ depends causally on W through the feedback). The proof instead telescopes per-letter with the causal conditioner X₁ᵢ:

                      I(W; Yⁿ) ≤ I(W; (Y₁ⁿ, Yⁿ))                                     -- post-processing
                               = ∑ᵢ [H(Vᵢ | V^{<i}) − H(Vᵢ | W, V^{<i})]              -- chain rule, Vᵢ = (Y₁ᵢ, Yᵢ)
                               ≤ ∑ᵢ [H(Vᵢ | X₁ᵢ) − H(Vᵢ | Xᵢ, X₁ᵢ)]                  -- per-letter (below)
                               = ∑ᵢ I(Xᵢ; Vᵢ | X₁ᵢ).
                      

                      The per-letter step uses two facts:

                      • H(Vᵢ | V^{<i}) ≤ H(Vᵢ | X₁ᵢ) — conditioning reduces entropy: X₁ᵢ = relay i (Y₁^{<i}) is a deterministic function of the past pairs V^{<i}, so conditioning on V^{<i} is a refinement of conditioning on X₁ᵢ. Structural, derived (not assumed).
                      • H(Vᵢ | W, V^{<i}) = H(Vᵢ | Xᵢ, X₁ᵢ) — memorylessness: both Xᵢ = encoder(W)ᵢ (a function of W) and X₁ᵢ (a function of V^{<i}) are deterministic functions of (W, V^{<i}), and given the i-th channel input (Xᵢ, X₁ᵢ) the i-th output Vᵢ is independent of (W, V^{<i}).

                      The single precondition h_memo is exactly the latter independence, Vᵢ ⫫ (W, V^{<i}) | (Xᵢ, X₁ᵢ), the d-separation property of a memoryless relay channel: the i-th output is conditionally independent of the message and the past output pairs given the i-th channel input. It encodes the channel's memoryless structure, not the conclusion (it is true in the operational setup where Vᵢ is fresh channel noise applied to (Xᵢ, X₁ᵢ)), so it is a regularity precondition, not load-bearing — mirroring the h_memo of bc_input_singleletterize. It is not the false independence W ⫫ X₁ⁿ (which fails for a causal relay): the conditioning here is on the i-th input, not on a fictitious second message. @audit:ok

                      Used by
                        theorem

                        InformationTheory.Shannon.Relay.relay_broadcast_cut_outer_bound

                        source
                        {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} {α₁ : Type u_3} {β : Type u_4} {β₁ : Type u_5} [Fintype α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [StandardBorelSpace α] [Fintype α₁] [Nonempty α₁] [MeasurableSpace α₁] [MeasurableSingletonClass α₁] [StandardBorelSpace α₁] [Fintype β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [StandardBorelSpace β] [Fintype β₁] [Nonempty β₁] [MeasurableSpace β₁] [MeasurableSingletonClass β₁] [StandardBorelSpace β₁] {M n : } [NeZero M] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (W : ΩFin M) (c : RelayCode M n α α₁ β β₁) (Ys : Fin nΩβ) (Y₁s : Fin nΩβ₁) (hW : Measurable W) (hYs : ∀ (i : Fin n), Measurable (Ys i)) (hY₁s : ∀ (i : Fin n), Measurable (Y₁s i)) (hW_uniform : MeasureTheory.Measure.map W μ = (↑(Fintype.card (Fin M)))⁻¹ MeasureTheory.Measure.count) (hcard : 2 M) (h_memo : ∀ (i : Fin n), IsMarkovChain μ (fun (ω : Ω) => (Y₁s i ω, Ys i ω)) (fun (ω : Ω) => (c.encoder (W ω) i, c.relay i fun (j : Fin i) => Y₁s j, ω)) fun (ω : Ω) => (W ω, fun (j : Fin i) => (Y₁s j, ω, Ys j, ω))) :
                        Real.log M i : Fin n, (condMutualInfo μ (fun (ω : Ω) => c.encoder (W ω) i) (fun (ω : Ω) => (Y₁s i ω, Ys i ω)) fun (ω : Ω) => c.relay i fun (j : Fin i) => Y₁s j, ω).toReal + Real.binEntropy (MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) c.decoder) + MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) c.decoder * Real.log (M - 1)

                        The operational outer bound at the broadcast cut of a relay channel (Cover–Thomas): for a uniformly distributed message W decoded from the receiver output Yⁿ, the rate is bounded by the per-letter broadcast-cut sum plus a Fano slack, log M ≤ ∑ᵢ I(Xᵢ; Y₁ᵢ, Yᵢ | X₁ᵢ) + h(Pe) + Pe · log(M - 1), where Xᵢ = encoder(W)ᵢ, X₁ᵢ = relay i (Y₁^{<i}), and Pe is the block decoding error probability.

                        The proof chains destination Fano (shannon_converse_single_shot) with the broadcast-cut message-level telescoping (relay_broadcast_cut_message_telescope). The memoryless d-separation hypothesis is a precondition (channel structure / regularity); the per-letter inequality (the genuine content) is proven, not assumed. The outer maximization over joint input pmfs is left to callers, which is why the conclusion keeps the explicit per-letter sum. @audit:ok

                        Used by
                          theorem

                          InformationTheory.Shannon.Relay.relay_cutset_outer_bound

                          source
                          {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} {α₁ : Type u_3} {β : Type u_4} {β₁ : Type u_5} [Fintype α] [Nonempty α] [MeasurableSpace α] [MeasurableSingletonClass α] [StandardBorelSpace α] [Fintype α₁] [Nonempty α₁] [MeasurableSpace α₁] [MeasurableSingletonClass α₁] [StandardBorelSpace α₁] [Fintype β] [Nonempty β] [MeasurableSpace β] [MeasurableSingletonClass β] [StandardBorelSpace β] [Fintype β₁] [Nonempty β₁] [MeasurableSpace β₁] [MeasurableSingletonClass β₁] [StandardBorelSpace β₁] {M n : } [NeZero M] (μ : MeasureTheory.Measure Ω) [MeasureTheory.IsProbabilityMeasure μ] (W : ΩFin M) (c : RelayCode M n α α₁ β β₁) (Ys : Fin nΩβ) (Y₁s : Fin nΩβ₁) (hW : Measurable W) (hYs : ∀ (i : Fin n), Measurable (Ys i)) (hY₁s : ∀ (i : Fin n), Measurable (Y₁s i)) (hW_uniform : MeasureTheory.Measure.map W μ = (↑(Fintype.card (Fin M)))⁻¹ MeasureTheory.Measure.count) (hcard : 2 M) (h_markov_mac : IsMarkovChain μ W (fun (ω : Ω) (j : Fin n) => (c.encoder (W ω) j, c.relay j fun (k : Fin j) => Y₁s k, ω)) fun (ω : Ω) (j : Fin n) => Ys j ω) (h_memo_mac : ChannelCodingConverseGeneral.IsMemorylessChannel μ (fun (i : Fin n) (ω : Ω) => (c.encoder (W ω) i, c.relay i fun (k : Fin i) => Y₁s k, ω)) Ys) (h_memo_bc : ∀ (i : Fin n), IsMarkovChain μ (fun (ω : Ω) => (Y₁s i ω, Ys i ω)) (fun (ω : Ω) => (c.encoder (W ω) i, c.relay i fun (j : Fin i) => Y₁s j, ω)) fun (ω : Ω) => (W ω, fun (j : Fin i) => (Y₁s j, ω, Ys j, ω))) :
                          Real.log M relayCutsetBound (i : Fin n, (condMutualInfo μ (fun (ω : Ω) => c.encoder (W ω) i) (fun (ω : Ω) => (Y₁s i ω, Ys i ω)) fun (ω : Ω) => c.relay i fun (j : Fin i) => Y₁s j, ω).toReal + Real.binEntropy (MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) c.decoder) + MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) c.decoder * Real.log (M - 1)) (i : Fin n, (mutualInfo μ (fun (ω : Ω) => (c.encoder (W ω) i, c.relay i fun (j : Fin i) => Y₁s j, ω)) (Ys i)).toReal + Real.binEntropy (MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) c.decoder) + MeasureFano.errorProb μ W (fun (ω : Ω) (j : Fin n) => Ys j ω) c.decoder * Real.log (M - 1))

                          Cut-set outer bound (relay channel, Cover–Thomas): for a uniformly distributed message W decoded from the receiver output Yⁿ, the rate is bounded by the min of the two cut rates, each taken as its per-letter sum plus a common Fano slack:

                          • the broadcast cut ∑ᵢ I(Xᵢ; Y₁ᵢ, Yᵢ | X₁ᵢ) + h(Pe) + Pe · log(M - 1), and
                          • the MAC cut ∑ᵢ I(Xᵢ, X₁ᵢ; Yᵢ) + h(Pe) + Pe · log(M - 1),

                          where Xᵢ = encoder(W)ᵢ, X₁ᵢ = relay i (Y₁^{<i}), and Pe is the block decoding error probability.

                          The proof combines the two cut bounds (relay_broadcast_cut_outer_bound and relay_mac_cut_outer_bound) via le_min. The memoryless / Markov / causal-relay hypotheses are preconditions (channel structure / regularity); the genuine content is carried by the two single-letterization cut lemmas and is proven, not assumed. The outer maximization over joint input pmfs p(x, x₁) — the textbook n · max_p — is left to callers, which is why the conclusion keeps the explicit per-letter sums. @audit:ok

                          Used by