Indexed text attachment
representation-theory-of-parametrized-neural-models-on-spaces.tex
Source for Representation Theory of Parametrized Neural Models on Spaces
View paper · Download original file
Lines 983–1082 of 1343. Source text is displayed literally.
983sign line is the M\"obius bundle984\cite[\S 1.1]{hatcher-vbkt}. Transport once around the circle985switches sheets. For comparison, the trivial double cover986\(P_+=S^1\times G\) preserves each sheet. The connectedness of987\(P_-\) distinguishes it from \(P_+\). Both admit the zero988equivariant weight, and both resulting decoders output probability989\(1/2\) at every base point and input.990 991\begin{proposition}[Families over the zero observable]992\label{gauge:zero}993The groupoid of smooth families on \(S^1\) with constant994observable \(K(1\mid x)=1/2\) is equivalent to995\(BG\amalg BG\).996\end{proposition}997 998\begin{proof}999The recovery formula for \(r\) forces the equivariant function1000\(w\) to be zero, so the whole groupoid consists of \(G\)-torsors1001and their isomorphisms. Cut the circle at one point and trivialize1002a torsor on the intervening interval. The two ends are glued by1003one element of \(G\) acting by multiplication, and changing the1004trivialization conjugates that element. Here \(G\) is abelian,1005so its two elements determine exactly the trivial and connected1006double covers. A torsor automorphism is multiplication by a1007locally constant sign on the base. The circle is connected, so1008each object has automorphism group \(G\).1009\end{proof}1010 1011At a single base point the corresponding fiber is \(BG\).1012The extra isomorphism class over the circle comes from sign1013monodromy. The statistic \(r\) identifies orbits at each point,1014while the stack retains both their automorphisms and their1015compatibility around a loop.1016 1017\subsection{A family with local weights}1018 1019On \(P_-\), the smooth function \(w([\theta])=\cos(\theta/2)\)1020changes sign under the deck transformation, so it is an1021equivariant weight. The descended family has1022\[1023 r([\theta])=\cos^2(\theta/2)=\tfrac12(1+\cos\theta).1024\]1025Its logits and probabilities are smooth by the local gluing1026above. This family cannot be represented by a single smooth1027weight on the circle in the same architecture. Indeed, any such1028weight \(v:S^1\to\mathbb R\) must satisfy \(v^2=r\) by1029Proposition~\ref{gauge:pointwise}. The sole zero of \(r\) is at1030\([\pi]\), where1031\[1032 r([\pi+t])=\tfrac14 t^2+O(t^4).1033\]1034Thus smoothness forces \(v'([\pi])^2=1/4\). A smooth weight with1035this derivative changes sign at its sole zero. The complement1036of that zero in the circle is connected, and a continuous1037nonvanishing real function on it has constant sign, a1038contradiction. The two chartwise signed weights do define the1039model in the quotient stack, with the nontrivial transition1040sign recording the absence of a global frame. The absolute1041value of the local weight yields a continuous scalar lift,1042which shows the role of the smooth category in the obstruction.1043 1044Information geometry is sensitive to the parameter chart at1045the fixed point. The Fisher information of one Bernoulli1046observation at fixed \(x\) is1047\(K_w(1\mid x)K_w(0\mid x)(\partial_w\ell_w(x))^2\);1048at \(w=0\) it vanishes because \(\partial_w\ell_w(x)=0\).1049For fixed \(x\), the expansion1050\[1051 \ell_w(x)=w^2x-\tfrac13 w^4x^3+O(w^6)1052\]1053gives \(I_w(x)=x^2w^2+O(w^4)\). In the nonnegative coordinate1054\(r=w^2\), the same logit starts as \(rx-\tfrac13r^2x^3\).1055Its derivative from the right gives Fisher information1056\(I_r(x)=x^2/4\) at \(r=0\), which is positive for \(x\neq0\).1057Away from zero the parameters satisfy the pullback identity1058\(I_w(x)=4w^2 I_r(x)\), with the right-hand information evaluated at1059\(r=w^2\). This extends to zero by the displayed expansions.1060The sign automorphism at zero, the monodromy of \(P_-\), and the1061Fisher degeneracy measure three different features of the same1062family. In particular, the finite gauge group has discrete orbits,1063while the map from smooth weights to observable laws loses its1064entire first derivative at the origin.1065\par1066\section{Equivariant comparison and stable features}1067\label{sec:stable}1068 1069At the level of an $\infty$-category, an action of a1070discrete group $G$ on features takes the form of a functor1071from the groupoid $BG$. A layer carrying symmetry specifies1072a natural transformation from its source action to its1073target action. If its parameter and feature objects can be1074combined by a tensor product, the source is the corresponding1075diagonal action.1076 1077For a concrete stable target, take $D(k)$ for a commutative1078ring $k$. A linear layer from $X$ to $Y$ with linear parameter1079$W$ is now a map $W\otimes_k^L X\to Y$. A coherent $G$-action1080on these objects equips the ordinary mapping spectrum1081$\operatorname{map}_{D(k)}(W\otimes_k^L X,Y)$ with conjugation1082transport. In a strict chain model, an element $g$ sends