Independent scholarship · Open accessThe monthly digest

Indexed text attachment

entropy-sources-and-channels-in-higher-categories.tex

Source for Entropy, Sources, and Channels in Higher Categories

View paper · Download original file

Lines 989–1088 of 4249. Source text is displayed literally.

989some $W$. Therefore $f[Y]=cH[Y]$ for all binary $Y$ with one constant $c$. Finally an observable with blocks990$B_1,\dots,B_k$ is the product of the binary observables $\{B_j,B_j^c\}$, and \eqref{eq:cocycle} determines the991value of a cocycle on a product from its values on the factors. Since $f$ and $cH$ agree on binary992observables, they agree everywhere.993\end{proof}994 995\begin{remark}996The hypothesis $|\Omega|\ge3$ is necessary. For $|\Omega|=2$ there is one nontrivial observable $X$, the only997cocycle conditions are $f[1]=0$ and $X.f[X]=0$, and $Z^1$ is the infinite-dimensional space of measurable998functions on $[0,1]$ vanishing at the endpoints (\cref{exer:two}). The chain rule constrains a function only999when there are two different observables to chain.1000\end{remark}1001 1002Compare the hypotheses with those of \cref{thm:khinchin}. Continuity has been weakened to measurability. The1003maximum property (K1) and expansibility (K3) are not assumed. Symmetry is not assumed; it was derived in1004Step~1 from $YZ=ZY$. What is assumed is locality, which says that the quantity attached to an observable1005depends only on its law, and the cocycle condition.1006 1007\section{Deformation}1008 1009For $\alpha>0$ replace \eqref{eq:action} by1010\[ (Y._\alpha f)(P)=\sum_{B\in Y,\ P(B)>0}P(B)^\alpha f\bigl(P|_B\bigr). \]1011The proof of \cref{lem:action} goes through, since $P(B)^\alpha\,P|_B(C)^\alpha=P(B\cap C)^\alpha$. Let $\delta_\alpha$ be1012the coboundary built from this action and $H^\bullet_\alpha(\Omega)$ its cohomology. A constant is no longer1013invariant:1014\[ (\delta_\alpha c)[Y](P)=c\,\Bigl(\sum_BP(B)^\alpha-1\Bigr). \]1015 1016\begin{theorem}\label{thm:tsallis}1017\Sli[\cite{Vig20}; cf.\ \cite{Dar70}] For $\alpha\ne1$ the \term{Tsallis entropy}1018\[ S_\alpha[Y](P)=\frac{1}{\alpha-1}\Bigl(1-\sum_{B\in Y}P(B)^\alpha\Bigr) \]1019is the coboundary of the constant $-1/(\alpha-1)$, hence a 1-cocycle for the $\alpha$-action:1020$S_\alpha[YZ]=S_\alpha[Y]+Y._\alpha S_\alpha[Z]$. For $|\Omega|\ge3$ every measurable $\alpha$-cocycle is a multiple of it,1021so that $H^0_\alpha(\Omega)=0$ and $H^1_\alpha(\Omega)=0$.1022\end{theorem}1023 1024As $\alpha\to1$ the coboundary $\delta_\alpha1$ tends to zero and $S_\alpha$ tends to entropy measured in natural1025units, $-\sum_BP(B)\ln P(B)$. This has a general explanation.1026 1027\begin{proposition}[Entropy as an obstruction]\label{prop:obstruction}1028\Sob Let $\delta_\alpha$ be a family of differentials on a fixed graded vector space, differentiable in1029$\alpha$, and write $\delta'$ for the derivative at $\alpha=1$. If $\delta_1c=0$, then $\delta'c$ is a1030$\delta_1$-cocycle, and its class in cohomology vanishes iff $c$ extends to first order to a family of1031cocycles $c+(\alpha-1)c'$. For the conditioning actions and $c=1$,1032\[ \delta'1=-\ln2\cdot H, \]1033so the class of entropy in $H^1(\Omega)$ is the obstruction to deforming the invariant constant $1$ along1034the family of $\alpha$-actions.1035\end{proposition}1036 1037\begin{proof}1038Differentiating $\delta_\alpha\delta_\alpha=0$ at $\alpha=1$ gives $\delta'\delta_1+\delta_1\delta'=0$, so1039$\delta_1(\delta'c)=-\delta'(\delta_1c)=0$. The family $c+(\alpha-1)c'$ is closed to first order iff1040$\delta'c+\delta_1c'=0$. Finally $\frac{d}{d\alpha}\sum_BP(B)^\alpha\big|_{\alpha=1}=\sum_BP(B)\ln P(B)$.1041\end{proof}1042 1043The dimensions fit: $(\dim H^0_\alpha,\dim H^1_\alpha)$ is $(0,0)$ for $\alpha\ne1$ and jumps to $(1,1)$ at1044$\alpha=1$, as cohomology does at a special parameter value, with the alternating sum unchanged.1045 1046\section{A derivation and its defect; entropy modulo a prime}1047 1048The function $\partial(x)=-x\log x$ satisfies the Leibniz rule1049\[ \partial(xy)=x\,\partial(y)+\partial(x)\,y, \]1050but it is not additive, and entropy is its defect of additivity: for a distribution,1051$H(p)=\sum_i\partial(p_i)-\partial\bigl(\sum_ip_i\bigr)$, the last term being $\partial(1)=0$. The chain rule is1052then one line,1053\[ \sum_{i,j}\partial(q_ip_{ij})=\sum_iq_i\sum_j\partial(p_{ij})+\sum_i\partial(q_i)\sum_jp_{ij}. \]1054This way of seeing entropy does not need the real numbers. Let $p$ be a prime. On the integers the Fermat1055quotient $\partial_p(a)=(a-a^p)/p$ satisfies $\partial_p(ab)\equiv a\,\partial_p(b)+\partial_p(a)\,b\pmod p$ and fails to1056be additive.1057 1058\begin{definition}\label{def:modp}1059Let $\pi\in\Fp^n$ with $\sum_i\pi_i=1$, and choose integers $a_i$ representing the $\pi_i$. The1060\term{entropy of $\pi$ modulo $p$} is1061\[ H_p(\pi)=\frac1p\Bigl(1-\sum_ia_i^p\Bigr)\bmod p\ \in\Fp. \]1062\end{definition}1063 1064It is well defined because $a^p$ modulo $p^2$ depends only on $a$ modulo $p$.1065 1066\begin{theorem}\label{thm:modp}1067\Sli[\cite{Kon95,Lei21p}] $H_p$ satisfies the chain rule1068$H_p\bigl(\gamma\circ(\pi^1,\dots,\pi^n)\bigr)=H_p(\gamma)+\sum_i\gamma_iH_p(\pi^i)$, and every function on such1069``distributions'' with values in $\Fp$ satisfying the chain rule is a multiple of it; no regularity1070hypothesis is needed. For odd $p$,1071\[ H_p(x,1-x)=\sum_{k=1}^{p-1}\frac{x^k}{k}, \]1072a truncated logarithm, which satisfies \eqref{eq:FEITH} for all $x,y\in\Fp\setminus\{1\}$.1073\end{theorem}1074 1075The polynomial was found by Kontsevich, who observed that it satisfies the fundamental equation; Leinster1076identified the function it comes from and proved the uniqueness. The real and the mod $p$ theories are1077parallel. Whether there is a single cohomological statement specializing to both is, as far as we know,1078open (\cref{ch:coda}).1079 1080\section{The general setting, and higher degrees}1081 1082We have worked with all partitions of one finite set. Baudot and Bennequin, and in greater generality1083Vigneaux, consider an \term{information structure}: a poset $\Sinf$ of observables in which the product of1084two elements exists whenever they have a common refinement in $\Sinf$, with a functor of laws and a1085presheaf of functions on it. For each $X$ the observables refined by $X$ form a monoid $\Sinf_X$; the1086monoid algebras $\mathcal{A}_X=\R[\Sinf_X]$ form a presheaf of rings $\mathcal{A}$ on $\Sinf$; and the1087functions form a presheaf of $\mathcal{A}$-modules.1088