Indexed text attachment
topos-actions-in-machine-learning-and-ai.tex
Source for Topos Actions in Machine Learning and Artificial Intelligence
View paper · Download original file
Lines 827–926 of 1207. Source text is displayed literally.
827arbitrary $H$-set. An $H$-equivariant map $F:P\times V\to W$828corresponds uniquely to a $K$-equivariant map829$\overline F:P\times(V/N)\to W$.830For topological actions and continuous layers the same831statement holds with the orbit topology on $V/N$.832\end{proposition}833 834\begin{proof}835Equivariance and the hypotheses give836$F(p,nv)=nF(p,v)=F(p,v)$. Therefore $F$ is constant837on the fibers of $(p,v)\mapsto(p,[v])$ and factors.838Normality of $N$ lets $H$ act on the orbits through $K$.839The factor map is consequently equivariant for $K$,840and the converse follows by composing with the quotient.841For continuous actions, $V\to V/N$ is an open surjection842since the saturation of an open set is the union of its843translates. Its product with $P$ is also an open surjection,844so the factor map is continuous by the quotient property.845\end{proof}846 847Suppose now that $N$ is finite and $V,W$ are real linear848representations. For $W$ with trivial $N$-action, an849$N$-invariant \emph{linear} map out of $V$ satisfies850$L=L\Pi_N$, where851\[852 \Pi_N v=\frac{1}{|N|}\sum_{n\in N} n v .853\]854The operator is a projector onto $V^N$, and for an855$H$-representation it commutes with $H$ because $N$ is856normal. The same factorization holds for an affine map857by adding its constant part. The orbit feature $V/N$858instead retains the information needed for all invariant859nonlinear maps. To see the difference, take860$H=C_4$ acting by quarter turns on $V=\mathbb R^2$ and861$N=C_2$ acting by $v\mapsto-v$. This is obtained from862$\chi:C_2\to C_4$ with image the half turn. Here863$\Pi_N=0$. A label on the input set864$\{0,\pm e_1,\pm e_2\}$ indicating whether it is nonzero865is predicted exactly by the invariant square-activation866score $\|v\|^2$. If half of the distribution is at $0$867and half is uniform on the other four points, a predictor868fed only $\Pi_N v$ is constant. Its minimum mean square869error is $1/4$, the variance of the label. The nonlinear870invariant predictor has zero error. The quotient871determines which information may be discarded, and an872implementation still has to choose an orbit encoding.873 874There is a related operation on predictors rather than875input features. If $\mu$ is the joint distribution of876input and label, assume $\mu$ is invariant under877$(v,y)\mapsto(nv,y)$. For a vector score $s$ and a loss878$\ell(y,-)$ convex in the score, define879$\overline s(v)=|N|^{-1}\sum_n s(nv)$.880For measurable scores and a nonnegative loss with finite risk,881Jensen's inequality gives882\begin{align*}883 \mathbb E_\mu[\ell(y,\overline s(v))]884 &\leq \frac1{|N|}\sum_{n\in N}885 \mathbb E_\mu[\ell(y,s(nv))]\\886 &=\mathbb E_\mu[\ell(y,s(v))].887\end{align*}888The averaged predictor descends to $V/N$ by invariance.889If $s$ had an additional $H$-equivariance with $N$ acting890trivially on the output, the factor is $K$-equivariant as891in Proposition~\ref{prop:orbit-layer}.892This is an exact-invariance example of orbit averaging893in data augmentation \cite{chen-augmentation}.894The inequality compares realized predictors. An895architecture must contain the average, for example by896parallel evaluation, before it gives a comparison897within that architecture.898 899\subsection{A filter with a twisted coefficient}900\label{sec:twisted-filter}901 902Let the deployment base be $B=S^1=\mathbb R/(2\pi\mathbb Z)$.903In $\operatorname{Sh}(B)$ let $G$ be the locally constant904group sheaf with fiber $C_3=\mathbb Z/3$ and monodromy905$j\mapsto-j$. Only $0$ is fixed by inversion, so906$\Gamma(B,G)$ is the trivial group. The local groups are907nevertheless nontrivial.908 909Use real regular-representation feature fibers910$V_t=\mathbb R[C_3]$ with basis $(e_j)_{j\in C_3}$.911The monodromy on the bundle $V$ is the orthogonal912permutation $S e_j=e_{-j}$. Let $R e_j=e_{j+1}$913be the group shift in a local lift of the angular914coordinate. Then $SRS^{-1}=R^{-1}$, which makes the915regular action compatible with the group sheaf's916transition. The sheaf of smooth sections of $V$917is an internal $G$-object. Its fiberwise $G$-linear918maps define equivariant linear layers in the relative919classifying topos.920 921\begin{proposition}[The coefficient bundle of equivariant filters]922\label{prop:twisted}923The bundle of fiberwise linear intertwiners from $V$924to itself is $\mathbb R^2\oplus L$ over $S^1$, where925$L$ is the real M\"obius line and $\mathbb R^2$ denotes926a trivial rank-two bundle. A flat filter, meaning