Indexed text attachment
entropy-sources-and-channels-in-higher-categories.tex
Source for Entropy, Sources, and Channels in Higher Categories
View paper · Download original file
Lines 1941–2040 of 4249. Source text is displayed literally.
1941\end{theorem}1942 1943\begin{proof}1944If $v$ lies in the range of some $P_m$ then $P_nv=v$ for $n\ge m$. If $v$ is orthogonal to all the ranges then1945$P_nv=0$. Such vectors span a dense subspace and $\|P_n\|\le1$. For a decreasing sequence apply this to1946$1-P_n$.1947\end{proof}1948 1949\begin{corollary}[L\'evy's theorems, in mean]\label{cor:levy}1950\Scl Let $1\le r<\infty$ and $f\in L^r(p)$. If $\FF_n$ increases to $\FF_\infty$ then1951$\E[f\st\FF_n]\to\E[f\st\FF_\infty]$ in $L^r$. If $\FF_n$ decreases to $\FF_{-\infty}=\bigcap_n\FF_n$ then1952$\E[f\st\FF_n]\to\E[f\st\FF_{-\infty}]$ in $L^r$.1953\end{corollary}1954 1955\begin{proof}1956For $r=2$ this is \cref{thm:projections} with $P_n=\E[\,\cdot\st\FF_n]$, since $\bigcup_nL^2(\FF_n)$ is dense in1957$L^2(\FF_\infty)$. For bounded $f$ the $L^2$ convergence together with the uniform bound $\|f\|_\infty$ gives $L^r$1958convergence for every finite $r$. Bounded functions are dense in $L^r$ and conditional expectations are1959contractions of $L^r$.1960\end{proof}1961 1962So the convergence of closed martingales in mean takes a few lines once conditional expectation is seen as1963a projection. Perrone and Van Belle show that the same argument can be run one level up, in the category1964itself.1965 1966\begin{theorem}[Perrone--Van Belle]\label{thm:PVB}1967\Sli[\cite{PVB24}] The categories $\Krn$ and $\Hilb$ are enriched in topological spaces in such a way that1968\begin{enumerate}[label=(\roman*)]1969\item every monotone net of dagger idempotents converges, in the topology of the hom-space, to its supremum1970if it is increasing and to its infimum if it is decreasing;1971\item the functors $L^r$, $1\le r<\infty$, are enriched, that is, continuous on hom-spaces.1972\end{enumerate}1973Consequently martingales and backward martingales indexed by arbitrary directed sets converge in $L^r$, and1974the same holds for vector-valued martingales with any enriched functor in place of $L^r$.1975\end{theorem}1976 1977The content of (i) for $\Krn$ is that the kernels $e_n$ themselves converge: the conditional distributions1978given $\FF_n$ converge to the conditional distribution given $\FF_\infty$, in a topology that makes1979integration against bounded functions continuous in $L^1$. A martingale is one function watched through1980this convergence. For the source, the kernels are ``the law of $x_0$ given the last $n$ letters'', and the1981limit is the law of $x_0$ given the infinite past.1982 1983\section{Almost sure convergence}1984 1985\begin{theorem}[Doob]\label{thm:doob}1986\Scl[\cite{Doo53}] A martingale with $\sup_n\E|M_n|<\infty$ converges almost surely. In particular, in both1987cases of \cref{cor:levy} the convergence holds almost surely as well.1988\end{theorem}1989 1990This is the theorem Khinchin quotes. Its proof needs an inequality that controls all times at once, either1991Doob's upcrossing inequality or the maximal inequality1992$\Prob(\sup_n|M_n|>t)\le t^{-1}\sup_n\E|M_n|$. Nothing in \cref{thm:projections} or \cref{thm:PVB} supplies it.1993This is the usual division of labour in analysis: convergence in norm comes from density and uniform1994boundedness, almost everywhere convergence from a maximal inequality. The categorical methods available1995at present reach the first and not the second. For the theorem we are aiming at this does not matter,1996because McMillan's theorem as Khinchin states it is a statement about convergence in mean, and we shall1997prove it in \cref{ch:aep} without \cref{thm:doob}.1998 1999\section{Limits against colimits}2000 2001A filtration gives two diagrams of Banach spaces: the inclusions $L^r(\FF_n)\hookrightarrow L^r(\FF_{n+1})$, and2002the conditional expectations $L^r(\FF_{n+1})\twoheadrightarrow L^r(\FF_n)$ in the opposite direction. Work in2003the category of Banach spaces and contractions, where a limit of a sequence consists of the bounded2004compatible families.2005 2006\begin{proposition}\label{prop:limcolim}2007\Sob Let $\FF_n$ increase to $\FF_\infty$.2008\begin{enumerate}[label=(\roman*)]2009\item For $1\le r<\infty$, $L^r(\FF_\infty)$ is the colimit of the inclusions.2010\item The limit of the conditional expectations is the space of $L^r$-bounded martingales, with norm2011$\sup_n\|M_n\|_r$.2012\item For $1<r<\infty$ the map $f\mapsto(\E[f\st\FF_n])_n$ from the colimit to the limit is an isometric2013isomorphism.2014\item For $r=1$ it is isometric and in general not surjective. For the filtration generated by a sequence2015of fair coin tosses, the limit is the space of all finite signed measures on the sequence space, and the2016image of the colimit is the subspace of measures absolutely continuous with respect to the coin-tossing2017measure.2018\end{enumerate}2019\end{proposition}2020 2021\begin{proof}2022(i) is the density used in \cref{cor:levy}. (ii) is the definition of a martingale. (iii) is the classical2023fact that for $r>1$ an $L^r$-bounded martingale is closed, with $\|f\|_r=\sup_n\|\E[f\st\FF_n]\|_r$. (iv) On a finite2024$\sigma$-algebra an integrable function is the density of a signed measure, and a martingale is a2025compatible family of signed measures on the algebras of cylinders with bounded total variation, which2026extends uniquely to a signed measure on the sequence space. Closed martingales correspond to measures with a2027density.2028\end{proof}2029 2030For $r=2$ the inclusions and the conditional expectations are adjoint to each other, and (iii) is the2031statement that for a sequence of embedding--projection pairs the colimit of the embeddings is the limit of2032the projections. The same coincidence, for a different kind of object, is what makes recursive domain2033equations solvable \cite{SP82}. For $r=1$ it fails, and the failure is the Lebesgue decomposition: the2034martingales that do not come from a function on $\FF_\infty$ are the singular measures. The likelihood ratio of a2035biased coin against a fair one is the standard example (\cref{exer:likelihood}).2036 2037\section{The mean ergodic theorem}2038 2039\begin{theorem}[von Neumann]\label{thm:vN}2040\Scl[\cite{vN32}] Let $U$ be an isometry of a Hilbert space and $P$ the orthogonal projection onto the space