Independent scholarship · Open accessThe monthly digest

Indexed text attachment

entropy-sources-and-channels-in-higher-categories.tex

Source for Entropy, Sources, and Channels in Higher Categories

View paper · Download original file

Lines 141–240 of 4249. Source text is displayed literally.

141\newcommand{\BZ}{B^{\Z}}142\newcommand{\term}[1]{\textbf{\itshape #1}}143 144% ---------- tikz styles ----------145\tikzset{146  ogbox/.style={draw, thick, rounded corners=2pt, minimum width=1.25cm, minimum height=1.05cm, fill=black!4},147  fw/.style={-{Stealth[length=5pt]}, thick, fwd},148  bw/.style={-{Stealth[length=5pt]}, thick, bwd, densely dashed},149  dec/.style={circle, fill, inner sep=1.7pt},150  nat/.style={circle, draw, thick, fill=white, inner sep=1.7pt},151  leaf/.style={inner sep=2pt, font=\small},152  lab/.style={font=\small, inner sep=1.5pt},153  infoset/.style={densely dashed, thick, accent},154  st/.style={circle, draw, thick, fill=white, inner sep=1.6pt},155  ty/.style={rectangle, draw, thick, fill=black!8, inner sep=2.6pt},156}157\newcommand{\sol}[1]{\par\medskip\noindent\textbf{#1.}\enspace\ignorespaces}158\newcommand{\bookauthor}{Anonymous}159%% ---- end preamble.tex ----160\begin{document}161\frontmatter162%% ---- begin front.tex ----163\begin{titlepage}164\thispagestyle{empty}165\vspace*{1.1in}166\noindent{\Large\scshape\color{accent} Higher Categorical Semantics}\par167\vspace{0.9in}168\noindent{\fontsize{30}{38}\selectfont\bfseries\raggedright Entropy, Sources,\\ and Channels\\ in Higher Categories\par}169\vspace{0.35in}170\noindent{\Large\itshape Cocycles, Markov categories, and coding\\[2pt] in the mathematical foundations of information theory}\par171\vfill172\noindent\begin{tikzpicture}[x=1cm,y=1cm]173  \node[ogbox] (mu) at (0,0) {$\mu$};174  \node[dec] (c) at (1.6,0) {};175  \node[ogbox] (nu) at (3.4,-0.65) {$\nu$};176  \draw[fw] (mu) -- (c);177  \draw[fw] (c) |- (5.4,0.65) node[right,lab,black]{$\AZ$};178  \draw[fw] (c) |- (nu);179  \draw[fw] (nu) -- (5.4,-0.65) node[right,lab,black]{$\BZ$};180\end{tikzpicture}\par181\vspace{0.5in}182\noindent{\large \bookauthor}\par183\vspace{4pt}184\noindent{\small 2026}185\end{titlepage}186 187\thispagestyle{empty}188\vspace*{\fill}189{\small190\noindent\textbf{About this text.}191This monograph is a higher-categorical treatment of the territory mapped by A.~I.~Khinchin's192\emph{Mathematical Foundations of Information Theory} (Dover, 1957; the English translation, by R.~A.~Silverman and193M.~D.~Friedman, of two papers of 1953 and 1956): the entropy of finite schemes and its uniqueness, stationary194and ergodic sources, McMillan's theorem, channels with memory, Feinstein's lemma, and Shannon's coding195theorems. It is an independent work. It reproduces none of that book's text or proofs verbatim; it is not196affiliated with or endorsed by its publisher. \Cref{app:concordance} gives a concordance so that the two can197be read side by side.198 199\medskip200\noindent\textbf{Status of results.}201Every numbered statement carries a tag: \textsf{classical} (standard information theory, probability or202ergodic theory, attributed), \textsf{literature} (established in the categorical or homological literature,203attributed), \textsf{reformulation} (a known result restated in a different language, with the restatement204proved or sketched here), \textsf{observation} (an elementary remark we have not found stated in this form),205or \textsf{programme} (a conjecture or a direction, not a result). No cited theorem is claimed as new.206 207\medskip208\noindent\textbf{Authorship and assistance.}209The text was drafted and typeset with a large language model (Anthropic's Claude) under the direction of210the submitting author, who is responsible for it. Generative AI assisted with drafting, literature recall,211mathematical consistency checks, computation, and typesetting. The numerical examples and a number of finite212structural claims are recomputed by the accompanying script \texttt{verify.py} (\cref{app:computations}).213Proofs and bibliographic details have not been independently refereed; corrections are welcome.214 215\medskip216\noindent Open access. Rights remain with the author.217}218\clearpage219 220\tableofcontents221 222\chapter{Preface}223 224Khinchin's two papers of 1953 and 1956 did for Shannon's theory what Kolmogorov's \emph{Grundbegriffe} had225done for probability: they said what the objects are. A source is a stationary measure on a space of226sequences. A channel is a family of measures on output sequences indexed by input sequences. Entropy is227the function on finite schemes singled out by three properties. The coding theorem is a statement about228these and about nothing else, and it has a proof. The papers are short, they contain almost no examples,229and seventy years on they are still a good place to learn what the theorems say.230 231They were written in the mathematical language available at the time, which was that of measures on232product spaces. This book rewrites them in a language that was not available. The claim is not that233category theory proves Shannon's theorems; for the most part it does not, and the places where it does not234are marked. The claim is that three features of Khinchin's text, each of which he states in words and then235handles by computation, are structure that can now be named.236 237The first is the additivity axiom $H(AB)=H(A)+H_A(B)$. It is the condition for a function on observables to be238a 1-cocycle for the action of observables on functions by conditioning, and the uniqueness theorem computes a239cohomology group: $H^1\iso\R$, generated by entropy (\cref{ch:cocycle}). Symmetry of entropy is then240not an axiom but the commutativity of a monoid; the Tsallis entropies are the coboundaries of a deformed