Mixing (mathematics)

In mathematics, mixing is an abstract concept originating from physics: the attempt to describe the irreversible thermodynamic process of mixing in the everyday world: mixing paint, mixing drinks, etc.

Repeated application of the baker's map to points colored red and blue, initially separated. The baker's map is mixing, which can be seen qualitatively as the red and blue points seem to be completely mixed after several iterations.

The concept appears in ergodic theory—the study of stochastic processes and measure-preserving dynamical systems. Several different definitions for mixing exist, including strong mixing, weak mixing and topological mixing, with the last not requiring a measure to be defined. Some of the different definitions of mixing can be arranged in a hierarchical order; thus, strong mixing implies weak mixing. Furthermore, weak mixing (and thus also strong mixing) implies ergodicity: that is, every system that is weakly mixing is also ergodic (and so one says that mixing is a "stronger" notion than ergodicity).

Mixing in stochastic processes

Let ${\displaystyle (X_{t})_{-\infty <t<\infty }}$ be a stochastic process on a probability space ${\displaystyle (\Omega ,{\mathcal {F}},\mathbb {P} )}$. The sequence space into which the process maps can be endowed with a topology, the product topology. The open sets of this topology are called cylinder sets. These cylinder sets generate a σ-algebra, the Borel σ-algebra; this is the smallest σ-algebra that contains the topology.

Define a function ${\displaystyle \alpha }$, called the strong mixing coefficient, as

${\displaystyle \alpha (s)=\sup \left\{|\mathbb {P} (A\cap B)-\mathbb {P} (A)\mathbb {P} (B)|:-\infty <t<\infty ,A\in X_{-\infty }^{t},B\in X_{t+s}^{\infty }\right\}}$

for all ${\displaystyle -\infty <s<\infty }$. The symbol ${\displaystyle X_{a}^{b}}$, with ${\displaystyle -\infty \leq a\leq b\leq \infty }$ denotes a sub-σ-algebra of the σ-algebra; it is the set of cylinder sets that are specified between times a and b, i.e. the σ-algebra generated by ${\displaystyle \{X_{a},X_{a+1},\ldots ,X_{b}\}}$.

The process ${\displaystyle (X_{t})_{-\infty <t<\infty }}$ is said to be strongly mixing if ${\displaystyle \alpha (s)\to 0}$ as ${\displaystyle s\to \infty }$. That is to say, a strongly mixing process is such that, in a way that is uniform over all times ${\displaystyle t}$ and all events, the events before time ${\displaystyle t}$ and the events after time ${\displaystyle t+s}$ tend towards being independent as ${\displaystyle s\to \infty }$; more colloquially, the process, in a strong sense, forgets its history.

Types of mixing

Suppose ${\displaystyle (X_{t})}$ were a stationary Markov process with stationary distribution ${\displaystyle \mathbb {Q} }$ and let ${\displaystyle L^{2}(\mathbb {Q} )}$ denote the space of Borel-measurable functions that are square-integrable with respect to the measure ${\displaystyle \mathbb {Q} }$. Also let

${\displaystyle {\mathcal {E}}_{t}\varphi (x)=\mathbb {E} [\varphi (X_{t})\mid X_{0}=x]}$

denote the conditional expectation operator on ${\displaystyle L^{2}(\mathbb {Q} ).}$ Finally, let

${\displaystyle Z=\left\{\varphi \in L^{2}(\mathbb {Q} ):\int \varphi \,d\mathbb {Q} =0\right\}}$

denote the space of square-integrable functions with mean zero.

The ρ-mixing coefficients of the process {x_t} are

${\displaystyle \rho _{t}=\sup _{\varphi \in Z:\,\|\varphi \|_{2}=1}\|{\mathcal {E}}_{t}\varphi \|_{2}.}$

The process is called ρ-mixing if these coefficients converge to zero as t → ∞, and “ρ-mixing with exponential decay rate” if ρ_t < e^−δt for some δ > 0. For a stationary Markov process, the coefficients ρ_t may either decay at an exponential rate, or be always equal to one.[1]

The α-mixing coefficients of the process {x_t} are

${\displaystyle \alpha _{t}=\sup _{\varphi \in Z:\|\varphi \|_{\infty }=1}\|{\mathcal {E}}_{t}\varphi \|_{1}.}$

The process is called α-mixing if these coefficients converge to zero as t → ∞, it is “α-mixing with exponential decay rate” if α_t < γe^−δt for some δ > 0, and it is α-mixing with a sub-exponential decay rate if α_t < ξ(t) for some non-increasing function ${\displaystyle \xi }$ satisfying

${\displaystyle {\frac {\ln \xi (t)}{t}}\to 0}$

as ${\displaystyle t\to \infty }$.[1]

The α-mixing coefficients are always smaller than the ρ-mixing ones: α_t ≤ ρ_t, therefore if the process is ρ-mixing, it will necessarily be α-mixing too. However, when ρ_t = 1, the process may still be α-mixing, with sub-exponential decay rate.

The β-mixing coefficients are given by

${\displaystyle \beta _{t}=\int \sup _{0\leq \varphi \leq 1}\left|{\mathcal {E}}_{t}\varphi (x)-\int \varphi \,d\mathbb {Q} \right|\,d\mathbb {Q} .}$

The process is called β-mixing if these coefficients converge to zero as t → ∞, it is β-mixing with an exponential decay rate if β_t < γe^−δt for some δ > 0, and it is β-mixing with a sub-exponential decay rate if β_tξ(t) → 0 as t → ∞ for some non-increasing function ${\displaystyle \xi }$ satisfying

${\displaystyle {\frac {\ln \xi (t)}{t}}\to 0}$

as ${\displaystyle t\to \infty }$.[1]

A strictly stationary Markov process is β-mixing if and only if it is an aperiodic recurrent Harris chain. The β-mixing coefficients are always bigger than the α-mixing ones, so if a process is β-mixing it will also be α-mixing. There is no direct relationship between β-mixing and ρ-mixing: neither of them implies the other.

Mixing in dynamical systems

A similar definition can be given using the vocabulary of measure-preserving dynamical systems. Let ${\displaystyle (X,{\mathcal {A}},\mu ,T)}$ be a dynamical system, with T being the time-evolution or shift operator. The system is said to be strong mixing if, for any ${\displaystyle A,B\in {\mathcal {A}}}$, one has

${\displaystyle \lim _{n\to \infty }\mu \left(A\cap T^{-n}B\right)=\mu (A)\mu (B).}$

For shifts parametrized by a continuous variable instead of a discrete integer n, the same definition applies, with ${\displaystyle T^{-n}}$ replaced by ${\displaystyle T_{g}}$ with g being the continuous-time parameter.

To understand the above definition physically, consider a shaker ${\displaystyle M}$ full of an incompressible liquid, which consists of 20% wine and 80% water. If ${\displaystyle A}$ is the region originally occupied by the wine, then, for any region ${\displaystyle B}$ within the shaker, the percentage of wine in ${\displaystyle B}$ after ${\displaystyle n}$ repetitions of the act of stirring is

${\displaystyle {\frac {\mu \left(T^{n}A\cap B\right)}{\mu (B)}}.}$

In such a situation, one would expect that after the liquid is sufficiently stirred (${\displaystyle n\to \infty }$), every region ${\displaystyle B}$ of the shaker will contain approximately 20% wine. This leads to

${\displaystyle \lim _{n\rightarrow \infty }{\frac {\mu \left(T^{n}A\cap B\right)}{\mu (B)}}={\frac {\mu (A)}{\mu (M)}}=\mu (A),}$

where ${\displaystyle \mu (M)=1}$, because measure-preserving dynamical systems are defined on probability spaces, and hence the final expression implies the above definition of strong mixing.

A dynamical system is said to be weak mixing if one has

${\displaystyle \lim _{n\to \infty }{\frac {1}{n}}\sum _{k=0}^{n-1}\left|\mu (A\cap T^{-k}B)-\mu (A)\mu (B)\right|=0.}$

In other words, ${\displaystyle T}$ is strong mixing if ${\displaystyle \mu (A\cap T^{-n}B)-\mu (A)\mu (B)\to 0}$ in the usual sense, weak mixing if

${\displaystyle \left|\mu (A\cap T^{-n}B)-\mu (A)\mu (B)\right|\to 0,}$

in the Cesàro sense, and ergodic if ${\displaystyle \mu \left(A\cap T^{-n}B\right)\to \mu (A)\mu (B)}$ in the Cesàro sense. Hence, strong mixing implies weak mixing, which implies ergodicity. However, the converse is not true: there exist ergodic dynamical systems which are not weakly mixing, and weakly mixing dynamical systems which are not strongly mixing.

For a system that is weak mixing, the shift operator T will have no (non-constant) square-integrable eigenfunctions with associated eigenvalue of one. In general, a shift operator will have a continuous spectrum, and thus will always have eigenfunctions that are generalized functions. However, for the system to be (at least) weak mixing, none of the eigenfunctions with associated eigenvalue of one can be square integrable.

${\displaystyle L^{2}}$ formulation

The properties of ergodicity, weak mixing and strong mixing of a measure-preserving dynamical system can also be characterized by the average of observables. By von Neumann's ergodic theorem, ergodicity of a dynamical system ${\displaystyle (X,{\mathcal {A}},\mu ,T)}$ is equivalent to the property that, for any function ${\displaystyle f\in L^{2}(X,\mu )}$, the sequence ${\displaystyle (f\circ T^{n})_{n\geq 0}}$ converges strongly and in the sense of Cesàro to ${\displaystyle \int _{X}f\,d\mu }$, i.e.,

${\displaystyle \lim _{N\to \infty }\left\|{1 \over N}\sum _{n=0}^{N-1}f\circ T^{n}-\int _{X}f\,d\mu \right\|_{L^{2}(X,\mu )}=0.}$

A dynamical system ${\displaystyle (X,{\mathcal {A}},\mu ,T)}$ is weakly mixing if, for any functions ${\displaystyle f}$ and ${\displaystyle g\in L^{2}(X,\mu ),}$

${\displaystyle \lim _{N\to \infty }{1 \over N}\sum _{n=0}^{N-1}\left|\int _{X}f\circ T^{n}\cdot gd\mu -\int _{X}f\,d\mu \cdot \int _{X}g\,d\mu \right|=0.}$

A dynamical system ${\displaystyle (X,{\mathcal {A}},\mu ,T)}$ is strongly mixing if, for any function ${\displaystyle f\in L^{2}(X,\mu ),}$ the sequence ${\displaystyle (f\circ T^{n})_{n\geq 0}}$ converges weakly to ${\displaystyle \int _{X}f\,d\mu ,}$ i.e., for any function ${\displaystyle g\in L^{2}(X,\mu ),}$

${\displaystyle \lim _{n\to \infty }\int _{X}f\circ T^{n}\cdot g\,d\mu =\int _{X}f\,d\mu \cdot \int _{X}g\,d\mu .}$

Since the system is assumed to be measure preserving, this last line is equivalent to saying that ${\displaystyle \lim _{n\to \infty }\operatorname {Cov} (f\circ T^{n},g)=0,}$ so that the random variables ${\displaystyle f\circ T^{n}}$ and ${\displaystyle g}$ become orthogonal as ${\displaystyle n}$ grows. Actually, since this works for any function ${\displaystyle g,}$ one can informally see mixing as the property that the random variables ${\displaystyle f\circ T^{n}}$ and ${\displaystyle g}$ become independent as ${\displaystyle n}$ grows.

Products of dynamical systems

Given two measured dynamical systems ${\displaystyle (X,\mu ,T)}$ and ${\displaystyle (Y,\nu ,S),}$ one can construct a dynamical system ${\displaystyle (X\times Y,\mu \otimes \nu ,T\times S)}$ on the Cartesian product by defining ${\displaystyle (T\times S)(x,y)=(T(x),S(y)).}$ We then have the following characterizations of weak mixing:

Proposition. A dynamical system ${\displaystyle (X,\mu ,T)}$ is weakly mixing if and only if, for any ergodic dynamical system ${\displaystyle (Y,\nu ,S)}$, the system ${\displaystyle (X\times Y,\mu \otimes \nu ,T\times S)}$ is also ergodic.

Proposition. A dynamical system ${\displaystyle (X,\mu ,T)}$ is weakly mixing if and only if ${\displaystyle (X^{2},\mu \otimes \mu ,T\times T)}$ is also ergodic. If this is the case, then ${\displaystyle (X^{2},\mu \otimes \mu ,T\times T)}$ is also weakly mixing.

Generalizations

The definition given above is sometimes called strong 2-mixing, to distinguish it from higher orders of mixing. A strong 3-mixing system may be defined as a system for which

${\displaystyle \lim _{m,n\to \infty }\mu (A\cap T^{-m}B\cap T^{-m-n}C)=\mu (A)\mu (B)\mu (C)}$

holds for all measurable sets A, B, C. We can define strong k-mixing similarly. A system which is strong k-mixing for all k = 2,3,4,... is called mixing of all orders.

It is unknown whether strong 2-mixing implies strong 3-mixing. It is known that strong m-mixing implies ergodicity.

Examples

Irrational rotations of the circle, and more generally irreducible translations on a torus, are ergodic but neither strongly nor weakly mixing with respect to the Lebesgue measure.

Many maps considered as chaotic are strongly mixing for some well-chosen invariant measure, including: the dyadic map, Arnold's cat map, horseshoe maps, Kolmogorov automorphisms, and the geodesic flow on the unit tangent bundle of compact surfaces of negative curvature.

Topological mixing

A form of mixing may be defined without appeal to a measure, using only the topology of the system. A continuous map ${\displaystyle f:X\to X}$ is said to be topologically transitive if, for every pair of non-empty open sets ${\displaystyle A,B\subset X}$, there exists an integer n such that

${\displaystyle f^{n}(A)\cap B\neq \varnothing }$

where ${\displaystyle f^{n}}$ is the nth iterate of f. In the operator theory, a topologically transitive bounded linear operator (a continuous linear map on a topological vector space) is usually called hypercyclic operator. A related idea is expressed by the wandering set.

Lemma: If X is a complete metric space with no isolated point, then f is topologically transitive if and only if there exists a hypercyclic point ${\displaystyle x\in X}$, that is, a point x such that its orbit ${\displaystyle \{f^{n}(x):n\in \mathbb {N} \}}$ is dense in X.

A system is said to be topologically mixing if, given open sets ${\displaystyle A}$ and ${\displaystyle B}$, there exists an integer N, such that, for all ${\displaystyle n>N}$, one has

${\displaystyle f^{n}(A)\cap B\neq \varnothing .}$

For a continuous-time system, ${\displaystyle f^{n}}$ is replaced by the flow ${\displaystyle \varphi _{g}}$, with g being the continuous parameter, with the requirement that a non-empty intersection hold for all ${\displaystyle \Vert g\Vert >N}$.

A weak topological mixing is one that has no non-constant continuous (with respect to the topology) eigenfunctions of the shift operator.

Topological mixing neither implies, nor is implied by either weak or strong mixing: there are examples of systems that are weak mixing but not topologically mixing, and examples that are topologically mixing but not strong mixing.

References

• Chen, Xiaohong; Hansen, Lars Peter; Carrasco, Marine (2010). "Nonlinearity and temporal dependence". Journal of Econometrics. 155 (2): 155–169. CiteSeerX 10.1.1.597.8777. doi:10.1016/j.jeconom.2009.10.001.CS1 maint: ref=harv (link)
• Achim Klenke, Probability Theory, (2006) Springer ISBN 978-1-84800-047-6
• V. I. Arnold and A. Avez, Ergodic Problems of Classical Mechanics, (1968) W. A. Benjamin, Inc.

1. Chen, Hansen & Carrasco (2010)