Indexed text attachment
entropy-sources-and-channels-in-higher-categories.tex
Source for Entropy, Sources, and Channels in Higher Categories
View paper · Download original file
Lines 1901–2000 of 4249. Source text is displayed literally.
1901 1902A morphism of $\Krn$ is the same as a \term{coupling} of $p$ and $q$, the joint law $p\rtimes k$ on $X\times Y$.1903Reading the coupling from the other side gives the Bayesian inverse $k^\dagger\colon(Y,q)\to(X,p)$.1904 1905\begin{proposition}\label{prop:dagger}1906\Sli[\cite{CJ19,Fri20,DDGK16}] Bayesian inversion makes $\Krn$ a dagger category:1907$k^{\dagger\dagger}=k$, $(l\circ k)^\dagger=k^\dagger\circ l^\dagger$, $\id^\dagger=\id$. For each $r\in[1,\infty]$ there is a1908functor $L^r\colon\Krn^{\op}\to\Ban$ to Banach spaces and contractions, sending $(X,p)$ to $L^r(p)$ and $k$ to1909$(k^*g)(x)=\int g(y)\,k(dy\st x)$. For $r=2$ it lands in Hilbert spaces and preserves daggers:1910$(k^\dagger)^*=(k^*)^\dagger$.1911\end{proposition}1912 1913A measure-preserving map $f\colon(X,p)\to(Y,q)$ is a morphism of $\Krn$, and $f^*g=g\circ f$ is an isometric1914embedding. Since $f$ is deterministic, $f\circ f^\dagger=\id_Y$. The other composite1915\[ e_f=f^\dagger\circ f\colon(X,p)\to(X,p) \]1916is therefore idempotent and self-adjoint, $e_f\circ e_f=e_f=e_f^\dagger$. As a kernel it does this: given $x$,1917forget everything except $f(x)$, and draw a new point from the conditional distribution given $f(x)$.1918 1919\begin{proposition}[Conditional expectations are idempotents]\label{prop:idem}1920\Sli[\cite{PVB24}]1921\begin{enumerate}[label=(\roman*)]1922\item Every sub-$\sigma$-algebra $\GG\subseteq\FF$ arises, up to null sets, from a measure-preserving map1923$f$, and so defines a dagger idempotent $e_\GG=e_f$ on $(X,p)$ that splits through a deterministic map.1924\item $L^r$ sends $e_\GG$ to the conditional expectation $\E[\,\cdot\st\GG]$; for $r=2$ this is the orthogonal1925projection onto $L^2(\GG)$.1926\item $\GG\subseteq\GG'$ iff $e_\GG\circ e_{\GG'}=e_\GG$, and then also $e_{\GG'}\circ e_\GG=e_\GG$.1927\end{enumerate}1928\end{proposition}1929 1930Part (iii) is the tower property $\E[\E[f\st\GG']\st\GG]=\E[f\st\GG]$. It defines a partial order on the1931idempotents, and a filtration is an increasing sequence $e_1\le e_2\le\cdots$ for it. A closed martingale is1932the image of one function under the sequence of projections $L^r(e_n)$.1933 1934\section{Convergence in mean}1935 1936\begin{theorem}\label{thm:projections}1937\Scl Let $P_1\le P_2\le\cdots$ be an increasing sequence of orthogonal projections on a Hilbert space. Then1938$P_nv\to P_\infty v$ for every vector $v$, where $P_\infty$ is the projection onto the closure of the union of the1939ranges. A decreasing sequence converges in the same sense to the projection onto the intersection of the1940ranges.1941\end{theorem}1942 1943\begin{proof}1944If $v$ lies in the range of some $P_m$ then $P_nv=v$ for $n\ge m$. If $v$ is orthogonal to all the ranges then1945$P_nv=0$. Such vectors span a dense subspace and $\|P_n\|\le1$. For a decreasing sequence apply this to1946$1-P_n$.1947\end{proof}1948 1949\begin{corollary}[L\'evy's theorems, in mean]\label{cor:levy}1950\Scl Let $1\le r<\infty$ and $f\in L^r(p)$. If $\FF_n$ increases to $\FF_\infty$ then1951$\E[f\st\FF_n]\to\E[f\st\FF_\infty]$ in $L^r$. If $\FF_n$ decreases to $\FF_{-\infty}=\bigcap_n\FF_n$ then1952$\E[f\st\FF_n]\to\E[f\st\FF_{-\infty}]$ in $L^r$.1953\end{corollary}1954 1955\begin{proof}1956For $r=2$ this is \cref{thm:projections} with $P_n=\E[\,\cdot\st\FF_n]$, since $\bigcup_nL^2(\FF_n)$ is dense in1957$L^2(\FF_\infty)$. For bounded $f$ the $L^2$ convergence together with the uniform bound $\|f\|_\infty$ gives $L^r$1958convergence for every finite $r$. Bounded functions are dense in $L^r$ and conditional expectations are1959contractions of $L^r$.1960\end{proof}1961 1962So the convergence of closed martingales in mean takes a few lines once conditional expectation is seen as1963a projection. Perrone and Van Belle show that the same argument can be run one level up, in the category1964itself.1965 1966\begin{theorem}[Perrone--Van Belle]\label{thm:PVB}1967\Sli[\cite{PVB24}] The categories $\Krn$ and $\Hilb$ are enriched in topological spaces in such a way that1968\begin{enumerate}[label=(\roman*)]1969\item every monotone net of dagger idempotents converges, in the topology of the hom-space, to its supremum1970if it is increasing and to its infimum if it is decreasing;1971\item the functors $L^r$, $1\le r<\infty$, are enriched, that is, continuous on hom-spaces.1972\end{enumerate}1973Consequently martingales and backward martingales indexed by arbitrary directed sets converge in $L^r$, and1974the same holds for vector-valued martingales with any enriched functor in place of $L^r$.1975\end{theorem}1976 1977The content of (i) for $\Krn$ is that the kernels $e_n$ themselves converge: the conditional distributions1978given $\FF_n$ converge to the conditional distribution given $\FF_\infty$, in a topology that makes1979integration against bounded functions continuous in $L^1$. A martingale is one function watched through1980this convergence. For the source, the kernels are ``the law of $x_0$ given the last $n$ letters'', and the1981limit is the law of $x_0$ given the infinite past.1982 1983\section{Almost sure convergence}1984 1985\begin{theorem}[Doob]\label{thm:doob}1986\Scl[\cite{Doo53}] A martingale with $\sup_n\E|M_n|<\infty$ converges almost surely. In particular, in both1987cases of \cref{cor:levy} the convergence holds almost surely as well.1988\end{theorem}1989 1990This is the theorem Khinchin quotes. Its proof needs an inequality that controls all times at once, either1991Doob's upcrossing inequality or the maximal inequality1992$\Prob(\sup_n|M_n|>t)\le t^{-1}\sup_n\E|M_n|$. Nothing in \cref{thm:projections} or \cref{thm:PVB} supplies it.1993This is the usual division of labour in analysis: convergence in norm comes from density and uniform1994boundedness, almost everywhere convergence from a maximal inequality. The categorical methods available1995at present reach the first and not the second. For the theorem we are aiming at this does not matter,1996because McMillan's theorem as Khinchin states it is a statement about convergence in mean, and we shall1997prove it in \cref{ch:aep} without \cref{thm:doob}.1998 1999\section{Limits against colimits}2000