Independent scholarship · Open accessThe monthly digest

Indexed text attachment

entropy-sources-and-channels-in-higher-categories.tex

Source for Entropy, Sources, and Channels in Higher Categories

View paper · Download original file

Lines 1889–1988 of 4249. Source text is displayed literally.

1889 1890For a stationary source and a letter $a$, let $\FF_n$ be generated by $x_{-n},\dots,x_{-1}$. Then1891\[ M^a_n(x)=\mu(x_0=a\st x_{-n},\dots,x_{-1})=\E\bigl[1_{\{x_0=a\}}\st\FF_n\bigr] \]1892is a closed martingale with values in $[0,1]$. This is the object Khinchin studies.1893 1894\section{Probability spaces and kernels}1895 1896\begin{definition}\label{def:krn}1897$\Krn$ is the category whose objects are standard Borel probability spaces $(X,p)$ and whose morphisms1898$(X,p)\to(Y,q)$ are Markov kernels $k$ with $k\circ p=q$, two kernels being identified when they agree1899$p$-almost surely.1900\end{definition}1901 1902A morphism of $\Krn$ is the same as a \term{coupling} of $p$ and $q$, the joint law $p\rtimes k$ on $X\times Y$.1903Reading the coupling from the other side gives the Bayesian inverse $k^\dagger\colon(Y,q)\to(X,p)$.1904 1905\begin{proposition}\label{prop:dagger}1906\Sli[\cite{CJ19,Fri20,DDGK16}] Bayesian inversion makes $\Krn$ a dagger category:1907$k^{\dagger\dagger}=k$, $(l\circ k)^\dagger=k^\dagger\circ l^\dagger$, $\id^\dagger=\id$. For each $r\in[1,\infty]$ there is a1908functor $L^r\colon\Krn^{\op}\to\Ban$ to Banach spaces and contractions, sending $(X,p)$ to $L^r(p)$ and $k$ to1909$(k^*g)(x)=\int g(y)\,k(dy\st x)$. For $r=2$ it lands in Hilbert spaces and preserves daggers:1910$(k^\dagger)^*=(k^*)^\dagger$.1911\end{proposition}1912 1913A measure-preserving map $f\colon(X,p)\to(Y,q)$ is a morphism of $\Krn$, and $f^*g=g\circ f$ is an isometric1914embedding. Since $f$ is deterministic, $f\circ f^\dagger=\id_Y$. The other composite1915\[ e_f=f^\dagger\circ f\colon(X,p)\to(X,p) \]1916is therefore idempotent and self-adjoint, $e_f\circ e_f=e_f=e_f^\dagger$. As a kernel it does this: given $x$,1917forget everything except $f(x)$, and draw a new point from the conditional distribution given $f(x)$.1918 1919\begin{proposition}[Conditional expectations are idempotents]\label{prop:idem}1920\Sli[\cite{PVB24}]1921\begin{enumerate}[label=(\roman*)]1922\item Every sub-$\sigma$-algebra $\GG\subseteq\FF$ arises, up to null sets, from a measure-preserving map1923$f$, and so defines a dagger idempotent $e_\GG=e_f$ on $(X,p)$ that splits through a deterministic map.1924\item $L^r$ sends $e_\GG$ to the conditional expectation $\E[\,\cdot\st\GG]$; for $r=2$ this is the orthogonal1925projection onto $L^2(\GG)$.1926\item $\GG\subseteq\GG'$ iff $e_\GG\circ e_{\GG'}=e_\GG$, and then also $e_{\GG'}\circ e_\GG=e_\GG$.1927\end{enumerate}1928\end{proposition}1929 1930Part (iii) is the tower property $\E[\E[f\st\GG']\st\GG]=\E[f\st\GG]$. It defines a partial order on the1931idempotents, and a filtration is an increasing sequence $e_1\le e_2\le\cdots$ for it. A closed martingale is1932the image of one function under the sequence of projections $L^r(e_n)$.1933 1934\section{Convergence in mean}1935 1936\begin{theorem}\label{thm:projections}1937\Scl Let $P_1\le P_2\le\cdots$ be an increasing sequence of orthogonal projections on a Hilbert space. Then1938$P_nv\to P_\infty v$ for every vector $v$, where $P_\infty$ is the projection onto the closure of the union of the1939ranges. A decreasing sequence converges in the same sense to the projection onto the intersection of the1940ranges.1941\end{theorem}1942 1943\begin{proof}1944If $v$ lies in the range of some $P_m$ then $P_nv=v$ for $n\ge m$. If $v$ is orthogonal to all the ranges then1945$P_nv=0$. Such vectors span a dense subspace and $\|P_n\|\le1$. For a decreasing sequence apply this to1946$1-P_n$.1947\end{proof}1948 1949\begin{corollary}[L\'evy's theorems, in mean]\label{cor:levy}1950\Scl Let $1\le r<\infty$ and $f\in L^r(p)$. If $\FF_n$ increases to $\FF_\infty$ then1951$\E[f\st\FF_n]\to\E[f\st\FF_\infty]$ in $L^r$. If $\FF_n$ decreases to $\FF_{-\infty}=\bigcap_n\FF_n$ then1952$\E[f\st\FF_n]\to\E[f\st\FF_{-\infty}]$ in $L^r$.1953\end{corollary}1954 1955\begin{proof}1956For $r=2$ this is \cref{thm:projections} with $P_n=\E[\,\cdot\st\FF_n]$, since $\bigcup_nL^2(\FF_n)$ is dense in1957$L^2(\FF_\infty)$. For bounded $f$ the $L^2$ convergence together with the uniform bound $\|f\|_\infty$ gives $L^r$1958convergence for every finite $r$. Bounded functions are dense in $L^r$ and conditional expectations are1959contractions of $L^r$.1960\end{proof}1961 1962So the convergence of closed martingales in mean takes a few lines once conditional expectation is seen as1963a projection. Perrone and Van Belle show that the same argument can be run one level up, in the category1964itself.1965 1966\begin{theorem}[Perrone--Van Belle]\label{thm:PVB}1967\Sli[\cite{PVB24}] The categories $\Krn$ and $\Hilb$ are enriched in topological spaces in such a way that1968\begin{enumerate}[label=(\roman*)]1969\item every monotone net of dagger idempotents converges, in the topology of the hom-space, to its supremum1970if it is increasing and to its infimum if it is decreasing;1971\item the functors $L^r$, $1\le r<\infty$, are enriched, that is, continuous on hom-spaces.1972\end{enumerate}1973Consequently martingales and backward martingales indexed by arbitrary directed sets converge in $L^r$, and1974the same holds for vector-valued martingales with any enriched functor in place of $L^r$.1975\end{theorem}1976 1977The content of (i) for $\Krn$ is that the kernels $e_n$ themselves converge: the conditional distributions1978given $\FF_n$ converge to the conditional distribution given $\FF_\infty$, in a topology that makes1979integration against bounded functions continuous in $L^1$. A martingale is one function watched through1980this convergence. For the source, the kernels are ``the law of $x_0$ given the last $n$ letters'', and the1981limit is the law of $x_0$ given the infinite past.1982 1983\section{Almost sure convergence}1984 1985\begin{theorem}[Doob]\label{thm:doob}1986\Scl[\cite{Doo53}] A martingale with $\sup_n\E|M_n|<\infty$ converges almost surely. In particular, in both1987cases of \cref{cor:levy} the convergence holds almost surely as well.1988\end{theorem}