InformationTheory

InformationTheory.Shannon.Stationary.Basic

source

Stationary processes #

Lean encoding of a stationary (resp. ergodic) discrete-time process via a MeasurePreserving (resp. Ergodic) shift T : Ω → Ω together with a single observable X : Ω → α. Time-i observation is X ∘ T^[i], the block (X_0, …, X_{n-1}) is blockRV n : Ω → (Fin n → α).

These definitions are the foundation for the entropy rate, Birkhoff's ergodic theorem, the log-likelihood chain rule, and the Shannon–McMillan–Breiman theorem.

Main definitions #

Main statements #

structure

InformationTheory.Shannon.StationaryProcess

source
{Ω : Type u_1} [MeasurableSpace Ω] (μ : MeasureTheory.Measure Ω) (α : Type u_3) [MeasurableSpace α] :
Type (max u_1 u_3)

A stationary process: a measure-preserving transformation T : Ω → Ω on (Ω, μ) together with a measurable observable X : Ω → α. The time-i observation is X ∘ T^[i].

  • T : ΩΩ

    The shift / time-evolution map.

  • X : Ωα

    The single observable; later samples are obtained by composing with T^[i].

  • measurePreserving : MeasureTheory.MeasurePreserving self.T μ μ

    T preserves μ.

  • measurable_X : Measurable self.X

    The observable is measurable.

Instances For
    Used by
      def

      InformationTheory.Shannon.StationaryProcess.obs

      source
      {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} [MeasurableSpace α] {μ : MeasureTheory.Measure Ω} (p : StationaryProcess μ α) (i : ) :
      Ωα

      Time-i observation X_i := X ∘ T^[i].

      Equations
      Instances For
        Used by
          theorem

          InformationTheory.Shannon.StationaryProcess.measurable_T

          source
          {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} [MeasurableSpace α] {μ : MeasureTheory.Measure Ω} (p : StationaryProcess μ α) :

          The shift map is measurable (it preserves μ, in particular it is measurable).

          Used by
            theorem

            InformationTheory.Shannon.StationaryProcess.measurable_iterate

            source
            {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} [MeasurableSpace α] {μ : MeasureTheory.Measure Ω} (p : StationaryProcess μ α) (i : ) :

            T^[i] is measurable for any i.

            Used by
              theorem

              InformationTheory.Shannon.StationaryProcess.measurable_obs

              source
              {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} [MeasurableSpace α] {μ : MeasureTheory.Measure Ω} (p : StationaryProcess μ α) (i : ) :

              Each time-i observation is measurable.

              Used by
                def

                InformationTheory.Shannon.StationaryProcess.blockRV

                source
                {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} [MeasurableSpace α] {μ : MeasureTheory.Measure Ω} (p : StationaryProcess μ α) (n : ) :
                ΩFin nα

                Block joint observation (X_0, X_1, …, X_{n-1}) : Ω → (Fin n → α).

                Equations
                Instances For
                  Used by
                    theorem

                    InformationTheory.Shannon.StationaryProcess.measurable_blockRV

                    source
                    {Ω : Type u_1} [MeasurableSpace Ω] {α : Type u_2} [MeasurableSpace α] {μ : MeasureTheory.Measure Ω} (p : StationaryProcess μ α) (n : ) :

                    The block random variable is measurable.

                    Used by
                      structure

                      InformationTheory.Shannon.ErgodicProcess

                      source
                      {Ω : Type u_1} [MeasurableSpace Ω] (μ : MeasureTheory.Measure Ω) (α : Type u_3) [MeasurableSpace α] extends InformationTheory.Shannon.StationaryProcess μ α :
                      Type (max u_1 u_3)

                      An ergodic stationary process: a StationaryProcess whose shift is moreover ergodic. The toStationaryProcess coercion is auto-generated by extends.

                      Instances For
                        Used by