Independent scholarship · Open accessThe monthly digest

Indexed text attachment

entropy-sources-and-channels-in-higher-categories.tex

Source for Entropy, Sources, and Channels in Higher Categories

View paper · Download original file

Lines 949–1048 of 4249. Source text is displayed literally.

949for all $x,y\in[0,1)$ with $x+y\le1$. Then $h=c\,\hb$ for a constant $c$, where950$\hb(x)=-x\log x-(1-x)\log(1-x)$.951\end{theorem}952 953Equation \eqref{eq:FEITH} is called the fundamental equation of information theory. The condition954$h(1)=0$ cannot be omitted, since $h(x)=x$ is a solution. The binary entropy satisfies the equation because955both sides compute $H(x,y,1-x-y)$, by splitting off first one outcome and then the other.956 957\begin{theorem}\label{thm:H1}958\Sli[\cite{BB15,Vig20}; in this form proved here] Let $|\Omega|\ge3$. Every 1-cocycle is a constant multiple of959entropy. Hence960\[ H^0(\Omega)\iso\R,\qquad H^1(\Omega)\iso\R\cdot[H]. \]961\end{theorem}962 963\begin{proof}964Let $f$ be a 1-cocycle.965 966\emph{Step 0: normalization.} Taking $Y=Z$ in \eqref{eq:cocycle} and using $YY=Y$ gives $Y.f[Y]=0$, that is,967$\sum_BP(B)\,f[Y](P|_B)=0$ for all $P$. Under $P|_B$ the observable $Y$ is constant, so by locality968$f[Y](P|_B)$ is a number $c_B$ depending only on $B$. Then $\sum_BP(B)c_B=0$ for all $P$ forces $c_B=0$: a969cocycle vanishes at laws under which its argument is deterministic. In particular $f[1]=0$.970 971\emph{Step 1: three points.} Let $\Omega=\{a,b,c\}$ and let $Y_a$ be the partition $\{a\}\,|\,\{b,c\}$, and972similarly $Y_b,Y_c$. By locality $f[Y_a](P)=h_a(P(a))$ for a measurable function $h_a$ on $[0,1]$, and by973Step~0, $h_a(0)=h_a(1)=0$. Evaluate \eqref{eq:symm} for $Y_a,Y_c$ at the law $P=(x,1-x-y,y)$. Conditioning on974$\{a\}$ makes $Y_c$ deterministic; conditioning on $\{b,c\}$ gives $c$ the probability $y/(1-x)$. So975\[ h_a(x)+(1-x)\,h_c\Bigl(\frac{y}{1-x}\Bigr)=h_c(y)+(1-y)\,h_a\Bigl(\frac{x}{1-y}\Bigr). \]976Putting $y=1-x$ gives $h_a(x)=h_c(1-x)$. The same argument for the other two pairs gives977$h_a(x)=h_b(1-x)$ and $h_b(x)=h_c(1-x)$. Hence $h_a=h_b=h_c=h$ with $h(x)=h(1-x)$, and $h$ satisfies978\eqref{eq:FEITH}. By \cref{thm:FEITH}, $h=c\,\hb$, so $f[Y]=cH[Y]$ for the three binary observables. The979atomic observable is $Y_aY_c$, and \eqref{eq:cocycle} gives $f[Y_aY_c]=c\,(H[Y_a]+Y_a.H[Y_c])=cH[Y_aY_c]$.980 981\emph{Step 2: the general case.} Let $W$ be a partition of $\Omega$ into three blocks. The observables982refined by $W$, with the laws of $W$, form a copy of the three-point structure, because all the functions983involved are $W$-local and every distribution on three points is the law of $W$ under some $P$. So there is984a constant $c_W$ with $f[Y]=c_WH[Y]$ for every $Y$ refined by $W$. If two such partitions $W,W'$ both refine985a binary observable $Y$, then $c_W=c_{W'}$, since $H[Y]\ne0$. Any two binary observables $\{A,A^c\}$ and986$\{B,B^c\}$ are connected in this way: if the two are nested, say $A\subsetneq B$, both are refined by987$\{A,B\setminus A,B^c\}$; otherwise $A\cap B$ is a proper nonempty subset of both $A$ and $B$ and988$\{A\cap B,(A\cap B)^c\}$ is nested with each. Since $|\Omega|\ge3$, every binary observable is refined by989some $W$. Therefore $f[Y]=cH[Y]$ for all binary $Y$ with one constant $c$. Finally an observable with blocks990$B_1,\dots,B_k$ is the product of the binary observables $\{B_j,B_j^c\}$, and \eqref{eq:cocycle} determines the991value of a cocycle on a product from its values on the factors. Since $f$ and $cH$ agree on binary992observables, they agree everywhere.993\end{proof}994 995\begin{remark}996The hypothesis $|\Omega|\ge3$ is necessary. For $|\Omega|=2$ there is one nontrivial observable $X$, the only997cocycle conditions are $f[1]=0$ and $X.f[X]=0$, and $Z^1$ is the infinite-dimensional space of measurable998functions on $[0,1]$ vanishing at the endpoints (\cref{exer:two}). The chain rule constrains a function only999when there are two different observables to chain.1000\end{remark}1001 1002Compare the hypotheses with those of \cref{thm:khinchin}. Continuity has been weakened to measurability. The1003maximum property (K1) and expansibility (K3) are not assumed. Symmetry is not assumed; it was derived in1004Step~1 from $YZ=ZY$. What is assumed is locality, which says that the quantity attached to an observable1005depends only on its law, and the cocycle condition.1006 1007\section{Deformation}1008 1009For $\alpha>0$ replace \eqref{eq:action} by1010\[ (Y._\alpha f)(P)=\sum_{B\in Y,\ P(B)>0}P(B)^\alpha f\bigl(P|_B\bigr). \]1011The proof of \cref{lem:action} goes through, since $P(B)^\alpha\,P|_B(C)^\alpha=P(B\cap C)^\alpha$. Let $\delta_\alpha$ be1012the coboundary built from this action and $H^\bullet_\alpha(\Omega)$ its cohomology. A constant is no longer1013invariant:1014\[ (\delta_\alpha c)[Y](P)=c\,\Bigl(\sum_BP(B)^\alpha-1\Bigr). \]1015 1016\begin{theorem}\label{thm:tsallis}1017\Sli[\cite{Vig20}; cf.\ \cite{Dar70}] For $\alpha\ne1$ the \term{Tsallis entropy}1018\[ S_\alpha[Y](P)=\frac{1}{\alpha-1}\Bigl(1-\sum_{B\in Y}P(B)^\alpha\Bigr) \]1019is the coboundary of the constant $-1/(\alpha-1)$, hence a 1-cocycle for the $\alpha$-action:1020$S_\alpha[YZ]=S_\alpha[Y]+Y._\alpha S_\alpha[Z]$. For $|\Omega|\ge3$ every measurable $\alpha$-cocycle is a multiple of it,1021so that $H^0_\alpha(\Omega)=0$ and $H^1_\alpha(\Omega)=0$.1022\end{theorem}1023 1024As $\alpha\to1$ the coboundary $\delta_\alpha1$ tends to zero and $S_\alpha$ tends to entropy measured in natural1025units, $-\sum_BP(B)\ln P(B)$. This has a general explanation.1026 1027\begin{proposition}[Entropy as an obstruction]\label{prop:obstruction}1028\Sob Let $\delta_\alpha$ be a family of differentials on a fixed graded vector space, differentiable in1029$\alpha$, and write $\delta'$ for the derivative at $\alpha=1$. If $\delta_1c=0$, then $\delta'c$ is a1030$\delta_1$-cocycle, and its class in cohomology vanishes iff $c$ extends to first order to a family of1031cocycles $c+(\alpha-1)c'$. For the conditioning actions and $c=1$,1032\[ \delta'1=-\ln2\cdot H, \]1033so the class of entropy in $H^1(\Omega)$ is the obstruction to deforming the invariant constant $1$ along1034the family of $\alpha$-actions.1035\end{proposition}1036 1037\begin{proof}1038Differentiating $\delta_\alpha\delta_\alpha=0$ at $\alpha=1$ gives $\delta'\delta_1+\delta_1\delta'=0$, so1039$\delta_1(\delta'c)=-\delta'(\delta_1c)=0$. The family $c+(\alpha-1)c'$ is closed to first order iff1040$\delta'c+\delta_1c'=0$. Finally $\frac{d}{d\alpha}\sum_BP(B)^\alpha\big|_{\alpha=1}=\sum_BP(B)\ln P(B)$.1041\end{proof}1042 1043The dimensions fit: $(\dim H^0_\alpha,\dim H^1_\alpha)$ is $(0,0)$ for $\alpha\ne1$ and jumps to $(1,1)$ at1044$\alpha=1$, as cohomology does at a special parameter value, with the alternating sum unchanged.1045 1046\section{A derivation and its defect; entropy modulo a prime}1047 1048The function $\partial(x)=-x\log x$ satisfies the Leibniz rule