Independent scholarship · Open accessThe monthly digest

Indexed text attachment

entropy-sources-and-channels-in-higher-categories.tex

Source for Entropy, Sources, and Channels in Higher Categories

View paper · Download original file

Lines 1029–1128 of 4249. Source text is displayed literally.

1029$\alpha$, and write $\delta'$ for the derivative at $\alpha=1$. If $\delta_1c=0$, then $\delta'c$ is a1030$\delta_1$-cocycle, and its class in cohomology vanishes iff $c$ extends to first order to a family of1031cocycles $c+(\alpha-1)c'$. For the conditioning actions and $c=1$,1032\[ \delta'1=-\ln2\cdot H, \]1033so the class of entropy in $H^1(\Omega)$ is the obstruction to deforming the invariant constant $1$ along1034the family of $\alpha$-actions.1035\end{proposition}1036 1037\begin{proof}1038Differentiating $\delta_\alpha\delta_\alpha=0$ at $\alpha=1$ gives $\delta'\delta_1+\delta_1\delta'=0$, so1039$\delta_1(\delta'c)=-\delta'(\delta_1c)=0$. The family $c+(\alpha-1)c'$ is closed to first order iff1040$\delta'c+\delta_1c'=0$. Finally $\frac{d}{d\alpha}\sum_BP(B)^\alpha\big|_{\alpha=1}=\sum_BP(B)\ln P(B)$.1041\end{proof}1042 1043The dimensions fit: $(\dim H^0_\alpha,\dim H^1_\alpha)$ is $(0,0)$ for $\alpha\ne1$ and jumps to $(1,1)$ at1044$\alpha=1$, as cohomology does at a special parameter value, with the alternating sum unchanged.1045 1046\section{A derivation and its defect; entropy modulo a prime}1047 1048The function $\partial(x)=-x\log x$ satisfies the Leibniz rule1049\[ \partial(xy)=x\,\partial(y)+\partial(x)\,y, \]1050but it is not additive, and entropy is its defect of additivity: for a distribution,1051$H(p)=\sum_i\partial(p_i)-\partial\bigl(\sum_ip_i\bigr)$, the last term being $\partial(1)=0$. The chain rule is1052then one line,1053\[ \sum_{i,j}\partial(q_ip_{ij})=\sum_iq_i\sum_j\partial(p_{ij})+\sum_i\partial(q_i)\sum_jp_{ij}. \]1054This way of seeing entropy does not need the real numbers. Let $p$ be a prime. On the integers the Fermat1055quotient $\partial_p(a)=(a-a^p)/p$ satisfies $\partial_p(ab)\equiv a\,\partial_p(b)+\partial_p(a)\,b\pmod p$ and fails to1056be additive.1057 1058\begin{definition}\label{def:modp}1059Let $\pi\in\Fp^n$ with $\sum_i\pi_i=1$, and choose integers $a_i$ representing the $\pi_i$. The1060\term{entropy of $\pi$ modulo $p$} is1061\[ H_p(\pi)=\frac1p\Bigl(1-\sum_ia_i^p\Bigr)\bmod p\ \in\Fp. \]1062\end{definition}1063 1064It is well defined because $a^p$ modulo $p^2$ depends only on $a$ modulo $p$.1065 1066\begin{theorem}\label{thm:modp}1067\Sli[\cite{Kon95,Lei21p}] $H_p$ satisfies the chain rule1068$H_p\bigl(\gamma\circ(\pi^1,\dots,\pi^n)\bigr)=H_p(\gamma)+\sum_i\gamma_iH_p(\pi^i)$, and every function on such1069``distributions'' with values in $\Fp$ satisfying the chain rule is a multiple of it; no regularity1070hypothesis is needed. For odd $p$,1071\[ H_p(x,1-x)=\sum_{k=1}^{p-1}\frac{x^k}{k}, \]1072a truncated logarithm, which satisfies \eqref{eq:FEITH} for all $x,y\in\Fp\setminus\{1\}$.1073\end{theorem}1074 1075The polynomial was found by Kontsevich, who observed that it satisfies the fundamental equation; Leinster1076identified the function it comes from and proved the uniqueness. The real and the mod $p$ theories are1077parallel. Whether there is a single cohomological statement specializing to both is, as far as we know,1078open (\cref{ch:coda}).1079 1080\section{The general setting, and higher degrees}1081 1082We have worked with all partitions of one finite set. Baudot and Bennequin, and in greater generality1083Vigneaux, consider an \term{information structure}: a poset $\Sinf$ of observables in which the product of1084two elements exists whenever they have a common refinement in $\Sinf$, with a functor of laws and a1085presheaf of functions on it. For each $X$ the observables refined by $X$ form a monoid $\Sinf_X$; the1086monoid algebras $\mathcal{A}_X=\R[\Sinf_X]$ form a presheaf of rings $\mathcal{A}$ on $\Sinf$; and the1087functions form a presheaf of $\mathcal{A}$-modules.1088 1089\begin{theorem}\label{thm:topos}1090\Sli[\cite{Vig20}] Information cohomology is $\Ext^\bullet_{\mathcal{A}}(\R,\Fun)$, computed in the abelian1091category of $\mathcal{A}$-modules in the topos of presheaves on $\Sinf$. A relative bar resolution of the1092trivial module $\R$ yields the complex of \cref{def:complex}. If every observable in $\Sinf$ can be written1093as a product of two others in a nondegenerate way, $H^1$ is a product of copies of $\R$, one for each1094connected component of $\Sinf$, each generated by a multiple of entropy.1095\end{theorem}1096 1097The generality matters when not all observables are jointly measurable, as in quantum theory or in the1098contextual situations studied by sheaf-theoretic methods. For our purposes the finite full structure is1099enough.1100 1101About degrees above one much less is known. If the conditioning action is replaced by the trivial action,1102the coboundary of entropy is $H[Y]+H[Z]-H[YZ]$, the mutual information, and the higher alternating sums are1103the multivariate mutual informations \cite{BB15}; these fit together into Hu's theorem that entropy1104extends to a signed measure on the Boolean algebra generated by the observables \cite{Hu62,Yeu08}. For the1105conditioning action itself we know of no general computation of $H^n(\Omega)$ for $n\ge2$.1106 1107\section*{Notes}1108 1109Information cohomology was introduced by Baudot and Bennequin \cite{BB15}; the topos-theoretic foundations,1110the bar resolution, and the treatment of the Tsallis case are due to Vigneaux \cite{Vig20,Vig19}, who also1111treats Gaussian laws, where differential entropy and the dimension appear as cocycles. The fundamental1112equation has a long history, for which see \cite{AD75}; its generalization of degree $\alpha$ is due to1113Dar\'oczy \cite{Dar70}, and the entropies $S_\alpha$ to Havrda and Charv\'at \cite{HC67} and Tsallis1114\cite{Tsa88}. Cathelineau \cite{Cat88} met the equation in the homology of $\mathrm{SL}_2$, which is the1115origin of the connection with polylogarithms pursued in \cite{Kon95,EVG02}. Entropy modulo a prime is1116developed by Leinster \cite{Lei21p}.1117 1118\section*{Exercises}1119 1120\begin{exercise}1121Show that $Y._\alpha(Z._\alpha f)=(YZ)._\alpha f$ for every $\alpha>0$, and that the constant function $1$ is1122invariant under the $\alpha$-action iff $\alpha=1$.1123\end{exercise}1124 1125\begin{exercise}1126Verify directly that $S_\alpha[YZ]=S_\alpha[Y]+Y._\alpha S_\alpha[Z]$. For independent $Y$ and $Z$ deduce1127$S_\alpha[YZ]=S_\alpha[Y]+S_\alpha[Z]-(\alpha-1)S_\alpha[Y]S_\alpha[Z]$.1128\end{exercise}