Markov kernel

In probability theory, a Markov kernel (or stochastic kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.^[1]

Formal definition

Let $(X,{\mathcal {A}})$ , $(Y,{\mathcal {B}})$ be measurable spaces. A Markov kernel with source $(X,{\mathcal {A}})$ and target $(Y,{\mathcal {B}})$ is a map $\kappa \colon X\times {\mathcal {B}}\to [0,1]$ with the following properties:

The map $x\mapsto \kappa (x,B)$ is ${\mathcal {A}}$ - measureable for every $B\in {\mathcal {B}}$ .
The map $B\mapsto \kappa (x,B)$ is a probability measure on $(Y,{\mathcal {B}})$ for every $x\in X$ .

(i.e. It associates to each point $x\in X$ a probability measure $\kappa (x,.)$ on $(Y,{\mathcal {B}})$ such that, for every measurable set $B\in {\mathcal {B}}$ , the map $x\mapsto \kappa (x,B)$ is measurable with respect to the $\sigma$ -algebra ${\mathcal {A}}$ .)^[2]

Examples

Simple random walk: Take $X=Y=\mathbb {Z}$ and ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {Z} )$ , then the Markov kernel $\kappa$ with

\kappa (x,B)={\frac {1}{2}}\mathbf {1} _{B}(x-1)+{\frac {1}{2}}\mathbf {1} _{B}(x+1),\quad \forall x\in \mathbb {Z} ,\quad \forall B\in {\mathcal {P}}(\mathbb {Z} )

describes the transition rule for the random walk on $\mathbb {Z}$ , where $\mathbf {1}$ is the indicator function.

Galton-Watson process: Take $X=Y=\mathbb {N}$ , ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(\mathbb {N} )$ , then

\kappa (x,B)={\begin{cases}\mathbf {1} _{B}(0)&\quad x=0,\\P[\xi _{1}+\dots +\xi _{x}\in B]&\quad {\text{else,}}\\\end{cases}}

with i.i.d. random variables $\xi _{i}$ .

General Markov processes with finite state space: Take $X=Y$ , ${\mathcal {A}}={\mathcal {B}}={\mathcal {P}}(X)={\mathcal {P}}(Y)$ and $|X|=|Y|=n$ , then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1\leq i,j\leq n}$ with $\Sigma _{j\in Y}K_{ij}=1$ for every $i\in X$ . In the convention of Markov kernels we write

\kappa (i,B)=\Sigma _{j\in B}K_{ij},\quad \forall i\in X,\quad \forall B\in {\mathcal {B}}

Construction of a Markov kernel: If $\nu$ is a finite measure on $(Y,{\mathcal {B}})$ and $k:X\times Y\to \mathbb {R} _{+}$ is a measurable function with respect to the product $\sigma$ -algebra ${\mathcal {A}}\otimes {\mathcal {B}}$ and has the property

\int _{X}k(x,y)\nu (\mathrm {d} y)=1,

for all $x\in X$ , then the mapping $\kappa :X\times {\mathcal {B}}\to [0,1]$

\kappa (x,B)=\int _{B}k(x,y)\nu (\mathrm {d} y),

defines a Markov kernel.^[3]

Properties

Semidirect product

Let $(X,{\mathcal {A}},P)$ be a probability space and $\kappa$ a Markov kernel from $(X,{\mathcal {A}})$ to some $(Y,{\mathcal {B}})$ .

Then there exists a unique measure $Q$ on $(X\times Y,{\mathcal {A}}\otimes {\mathcal {B}})$ , such that

Q(A\times B)=\int _{A}\kappa (x,B)dP(x),\quad \forall A\in {\mathcal {A}},\quad \forall B\in {\mathcal {B}}

Regular conditional distribution

Let $(S,Y)$ be a Borel space, $X$ a $(S,Y)$ - valued random variable on the measure space $(\Omega ,{\mathcal {F}},P)$ and ${\mathcal {G}}\subseteq {\mathcal {F}}$ a sub- $\sigma$ -algebra.

Then there exists a Markov kernel $\kappa$ from $(\Omega ,{\mathcal {G}})$ to $(S,Y)$ , such that $\kappa (.,B)$ is a version of the conditional expectation $E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]$ for every $B\in Y$ , i.e.

P[X\in B|{\mathcal {G}}]=E[\mathbf {1} _{\{X\in B\}}|{\mathcal {G}}]=\kappa (\omega ,B),\quad P-a.s.\forall B\in {\mathcal {G}}

It is called regular conditional distribution of $X$ given ${\mathcal {G}}$ and is not uniquely defined.

References

↑ Reiss, R. D. (1993). "A Course on Point Processes". Springer Series in Statistics. doi:10.1007/978-1-4613-9308-5. ISBN 978-1-4613-9310-8.
↑ Klenke, Achim. Probability Theory: A Comprehensive Course (2 ed.). Springer. p. 180. doi:10.1007/978-1-4471-5361-0.
↑ Erhan, Cinlar (2011). Probability and Stochastics. New York: Springer. pp. 37–38. ISBN 978-0-387-87858-4.

Bauer, Heinz (1996), Probability Theory, de Gruyter, ISBN 3-11-013935-9

§36. Kernels and semigroups of kernels

This article is issued from Wikipedia - version of the 11/30/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.